2024-04-25T13:41:11.200 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T13:41:11.200 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184 --verbose --owner scheduled_pdonnell@teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/orig.config.yaml 2024-04-25T13:41:11.254 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:41:11.340 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673184' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMvdbeagEktun0aU5/aZgfKFzqoXJvKfbmf4kMxDhQekZphHuJ9Qn5LqZsrx5X6KmmyGP1HO9na8MYmy1qBmut8= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLLCHTOChGCgKjA9Dk4Qq+P4RLxZdPJQnS7oSBbrUp4gB24kaB7vkEi7jQsdUMYdaNvyDx74CXGobrfXM3nxy5w= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOeVvgRyd8dI/CYsVJVTyLAUKQuDf0EgbXikR3rLl2xyahunGZ/1OHtdGKGp7UucVKsPTiW/cGf2lme2iL9x6tE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T13:41:11.341 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T13:41:11.341 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T13:41:11.341 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T13:41:11.343 INFO:teuthology.task.internal:Checking packages... 2024-04-25T13:41:11.368 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T13:41:11.368 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T13:41:11.368 INFO:teuthology.packaging:ref: None 2024-04-25T13:41:11.368 INFO:teuthology.packaging:tag: None 2024-04-25T13:41:11.368 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:41:11.368 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:41:11.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:41:11.582 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T13:41:11.583 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T13:41:11.590 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T13:41:11.590 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T13:41:11.636 INFO:teuthology.task.internal:Saving configuration 2024-04-25T13:41:11.655 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T13:41:11.703 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:41:11.729 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:35:54.436503', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMvdbeagEktun0aU5/aZgfKFzqoXJvKfbmf4kMxDhQekZphHuJ9Qn5LqZsrx5X6KmmyGP1HO9na8MYmy1qBmut8='} 2024-04-25T13:41:11.752 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:35:54.434705', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLLCHTOChGCgKjA9Dk4Qq+P4RLxZdPJQnS7oSBbrUp4gB24kaB7vkEi7jQsdUMYdaNvyDx74CXGobrfXM3nxy5w='} 2024-04-25T13:41:11.771 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:35:54.437915', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOeVvgRyd8dI/CYsVJVTyLAUKQuDf0EgbXikR3rLl2xyahunGZ/1OHtdGKGp7UucVKsPTiW/cGf2lme2iL9x6tE='} 2024-04-25T13:41:11.771 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T13:41:11.777 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T13:41:11.777 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T13:41:11.777 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T13:41:11.778 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T13:41:11.885 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3a00dc20d0>, signals=[15]) 2024-04-25T13:41:11.886 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T13:41:11.892 INFO:teuthology.task.internal:Opening connections... 2024-04-25T13:41:11.892 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T13:41:11.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:41:11.972 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T13:41:11.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:41:12.048 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T13:41:12.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:41:12.166 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T13:41:12.172 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-04-25T13:41:12.190 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-04-25T13:41:12.191 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:41:12.248 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:41:12.249 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:41:12.249 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:41:12.249 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:41:12.249 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:41:12.249 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-04-25T13:41:12.278 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-25T13:41:12.296 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-25T13:41:12.296 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:41:12.354 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:41:12.355 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-25T13:41:12.378 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-04-25T13:41:12.396 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-04-25T13:41:12.396 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-04-25T13:41:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-04-25T13:41:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-04-25T13:41:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-04-25T13:41:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:41:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-04-25T13:41:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:41:12.453 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:41:12.454 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-04-25T13:41:12.454 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:41:12.454 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:41:12.454 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:41:12.454 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:41:12.454 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:41:12.454 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:41:12.454 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-04-25T13:41:12.478 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T13:41:12.488 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T13:41:12.511 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T13:41:12.511 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:41:12.513 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:41:12.515 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:41:12.531 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T13:41:12.536 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T13:41:12.536 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:41:12.569 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:41:12.572 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:41:12.675 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T13:41:12.751 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T13:41:12.765 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T13:41:12.765 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T13:41:12.765 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T13:41:12.765 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:41:12.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:41:12.781 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-04-25T13:41:12.838 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-437.el9.x86_64 2024-04-25T13:41:12.838 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-437.el9.x86_64 2024-04-25T13:41:12.839 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-04-25T13:41:15.134 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T13:41:16.902 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-25T13:41:21.570 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-25T13:41:23.426 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-04-25T13:41:25.019 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-25T13:41:30.011 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-25T13:41:31.875 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T13:41:31.875 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T13:41:31.972 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T13:41:31.973 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-25T13:41:31.973 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-25T13:41:32.084 DEBUG:teuthology.orchestra.run.smithi078:> echo no | sudo yum reinstall kernel || true 2024-04-25T13:41:32.581 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:41:29 PM UTC. 2024-04-25T13:41:32.699 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T13:41:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:41:32.712 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 255 k 2024-04-25T13:41:32.712 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-04-25T13:41:32.712 INFO:teuthology.orchestra.run.smithi078.stderr:Operation aborted. 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi078.stdout:Is this ok [y/N]: 2024-04-25T13:41:32.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum reinstall -y kernel || true 2024-04-25T13:41:33.262 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 01:41:29 PM UTC. 2024-04-25T13:41:33.376 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T13:41:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:41:33.378 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 255 k 2024-04-25T13:41:33.378 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-04-25T13:41:33.378 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-25T13:41:33.629 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-25T13:41:33.630 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:41:33.630 INFO:teuthology.orchestra.run.smithi078.stdout:Total 1.0 MB/s | 255 kB 00:00 2024-04-25T13:41:33.653 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-25T13:41:33.830 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-25T13:41:33.831 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-25T13:41:33.835 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-25T13:41:33.836 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-25T13:41:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-25T13:41:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:41:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:41:34.716 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:41:34.716 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:41:35.410 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:41:35.410 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:41:35.411 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalled: 2024-04-25T13:41:35.411 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T13:41:35.411 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:41:35.411 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-25T13:41:35.568 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T13:41:35.613 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T13:41:35.614 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T13:41:35.614 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T13:41:35.614 DEBUG:teuthology.parallel:result is None 2024-04-25T13:41:35.614 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T13:41:35.624 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T13:41:35.624 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:41:35.627 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:41:35.630 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:41:35.647 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T13:41:35.653 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T13:41:35.675 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T13:41:35.675 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:41:35.682 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:41:35.687 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:41:35.770 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T13:41:35.777 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T13:41:35.777 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:41:35.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:41:35.793 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:41:35.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:41:35.809 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:41:35.811 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:41:35.835 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:41:35.850 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:41:35.863 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:41:35.872 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:41:35.886 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:41:35.889 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:41:35.927 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:41:35.929 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T13:41:35.936 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T13:41:35.937 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:41:35.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:41:35.941 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:41:35.997 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T13:41:36.006 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T13:41:36.006 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:41:36.009 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:41:36.011 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:41:36.056 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:41:36.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:41:36.149 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:41:36.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:41:36.268 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:41:36.269 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:41:36.332 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:41:36.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:41:36.421 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:41:36.502 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:41:36.546 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:41:36.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:41:36.611 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:41:36.657 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:41:36.701 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:41:36.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:41:36.827 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:41:36.827 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:41:36.893 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-04-25T13:41:36.896 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-25T13:41:36.898 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-04-25T13:41:36.926 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:41:36.935 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:41:36.975 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:41:37.450 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T13:41:37.458 INFO:teuthology.task.internal:Starting timer... 2024-04-25T13:41:37.458 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T13:41:37.495 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T13:41:37.518 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T13:41:37.519 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-04-25T13:41:37.573 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-04-25T13:41:37.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-25T13:41:37.651 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-25T13:41:37.653 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-04-25T13:41:37.720 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T13:41:37.720 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-04-25T13:41:37.746 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-04-25T13:41:37.746 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-25T13:41:37.786 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-25T13:41:37.786 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-04-25T13:41:37.809 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-04-25T13:41:37.810 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive'} 2024-04-25T13:41:37.810 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:41:37.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:41:37.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:41:37.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:41:37.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:41:37.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:41:37.889 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T13:41:37.889 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:41:37.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:41:37.948 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:41:37.979 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T13:41:37.989 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T13:41:38.005 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T13:41:38.006 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi112.front.sepia.ceph.com,smithi129.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T13:45:31.514 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com')] 2024-04-25T13:45:31.516 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-25T13:45:31.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:45:31.593 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-04-25T13:45:31.669 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-25T13:45:31.669 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T13:45:31.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:45:31.747 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T13:45:31.825 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T13:45:31.825 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T13:45:31.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:45:31.901 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-04-25T13:45:31.937 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T13:45:31.938 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T13:45:31.949 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T13:45:31.949 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:45:31.950 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:45:31.952 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:45:31.952 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:45:31.954 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:45:31.955 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:45:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:45:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:45:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:45:31.995 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:45:32.005 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:45:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-04-25T13:45:32.018 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-25T13:45:32.019 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:45:32.028 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-04-25T13:45:32.028 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-25T13:45:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:45:32.042 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:45:32.042 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-04-25T13:45:32.051 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-04-25T13:45:32.054 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:45:32.054 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:45:32.064 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:45:32.077 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:45:32.109 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:45:32.112 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:45:32.112 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T13:45:32.112 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.113 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.113 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.113 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.116 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:45:32.119 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:45:32.120 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-25T13:45:32.120 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.120 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.120 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.120 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.131 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:45:32.134 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:45:32.134 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-25T13:45:32.134 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.134 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.134 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.134 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:45:32.135 INFO:teuthology.run_tasks:Running task install... 2024-04-25T13:45:32.143 DEBUG:teuthology.task.install:project ceph 2024-04-25T13:45:32.144 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T13:45:32.144 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T13:45:32.144 INFO:teuthology.task.install:Using flavor: default 2024-04-25T13:45:32.149 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T13:45:32.149 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T13:45:32.150 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T13:45:32.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:45:32.152 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T13:45:32.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:45:32.153 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T13:45:32.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:45:32.305 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T13:45:32.305 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T13:45:32.409 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T13:45:32.409 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T13:45:32.442 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T13:45:32.442 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T13:45:32.517 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:45:32.518 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:45:32.518 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:45:32.520 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:45:32.520 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:45:32.521 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:45:32.542 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:45:32.542 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:45:32.546 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:45:32.546 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:45:32.606 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:45:32.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:45:32.632 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:45:32.632 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:45:32.632 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:45:32.658 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:45:32.658 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:45:32.678 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:45:32.685 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:45:32.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:45:32.741 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-04-25T13:45:32.743 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-04-25T13:45:32.748 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-04-25T13:45:32.750 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-04-25T13:45:32.798 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-25T13:45:32.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-25T13:45:32.967 INFO:teuthology.orchestra.run.smithi129.stdout:48 files removed 2024-04-25T13:45:32.974 INFO:teuthology.orchestra.run.smithi078.stdout:48 files removed 2024-04-25T13:45:32.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:45:33.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:45:33.086 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-25T13:45:33.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:45:33.580 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 259 kB/s | 84 kB 00:00 2024-04-25T13:45:33.647 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 215 kB/s | 84 kB 00:00 2024-04-25T13:45:33.896 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 159 kB/s | 84 kB 00:00 2024-04-25T13:45:33.955 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-25T13:45:33.998 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-04-25T13:45:34.347 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-25T13:45:34.458 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-25T13:45:34.514 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-04-25T13:45:34.772 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-25T13:45:35.368 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.0 MB 00:00 2024-04-25T13:45:35.542 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.0 MB 00:00 2024-04-25T13:45:35.792 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.0 MB 00:00 2024-04-25T13:45:37.985 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T13:45:38.034 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T13:45:38.228 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T13:45:42.185 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.2 MB 00:00 2024-04-25T13:45:42.294 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-04-25T13:45:42.637 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-04-25T13:45:43.701 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-25T13:45:43.759 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-25T13:45:44.927 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T13:45:45.243 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-25T13:45:46.947 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T13:45:49.844 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-25T13:45:51.936 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T13:45:51.960 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:45:51.961 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:45:51.972 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:45:51.972 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:45:51.973 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:45:51.974 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:45:51.976 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:45:51.978 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:45:51.979 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:45:52.038 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T13:45:52.062 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T13:45:52.063 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T13:45:52.064 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:45:52.065 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T13:45:52.066 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:45:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:45:52.068 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:45:52.069 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:45:52.070 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T13:45:52.071 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:45:52.072 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:45:52.073 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:45:52.074 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:45:52.075 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:45:52.076 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:45:52.077 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T13:45:52.078 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T13:45:52.078 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-25T13:45:52.078 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 4 Packages 2024-04-25T13:45:52.078 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:45:52.078 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 214 M 2024-04-25T13:45:52.079 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T13:45:53.370 INFO:teuthology.orchestra.run.smithi112.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T13:45:54.062 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:45:54.063 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:45:54.073 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:45:54.074 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:45:54.075 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:45:54.076 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:45:54.077 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:45:54.079 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:45:54.080 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:45:54.141 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T13:45:54.150 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:45:54.151 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T13:45:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:45:54.153 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:45:54.154 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:45:54.155 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:45:54.156 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:45:54.157 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:45:54.158 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:45:54.159 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:45:54.160 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:45:54.161 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 4 Packages 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:45:54.162 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 214 M 2024-04-25T13:45:54.163 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-25T13:45:54.362 INFO:teuthology.orchestra.run.smithi112.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 870 kB/s | 863 kB 00:00 2024-04-25T13:45:54.578 INFO:teuthology.orchestra.run.smithi112.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 679 kB/s | 147 kB 00:00 2024-04-25T13:45:54.929 INFO:teuthology.orchestra.run.smithi112.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 2.7 MB/s | 5.4 MB 00:01 2024-04-25T13:45:55.254 INFO:teuthology.orchestra.run.smithi112.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-04-25T13:45:55.480 INFO:teuthology.orchestra.run.smithi078.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-25T13:45:56.146 INFO:teuthology.orchestra.run.smithi112.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 5.5 MB/s | 4.9 MB 00:00 2024-04-25T13:45:56.305 INFO:teuthology.orchestra.run.smithi078.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 863 kB 00:00 2024-04-25T13:45:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.2 MB/s | 147 kB 00:00 2024-04-25T13:45:56.696 INFO:teuthology.orchestra.run.smithi112.stdout:(7/152): ceph-common-19.0.0-3291.g994e2f20.el9. 5.2 MB/s | 20 MB 00:03 2024-04-25T13:45:56.980 INFO:teuthology.orchestra.run.smithi078.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.0 MB/s | 2.3 MB 00:00 2024-04-25T13:45:57.464 INFO:teuthology.orchestra.run.smithi078.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T13:45:57.663 INFO:teuthology.orchestra.run.smithi112.stdout:(8/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 750 kB/s | 2.3 MB 00:03 2024-04-25T13:45:57.788 INFO:teuthology.orchestra.run.smithi112.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 201 kB/s | 25 kB 00:00 2024-04-25T13:45:57.955 INFO:teuthology.orchestra.run.smithi112.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 8.0 MB/s | 10 MB 00:01 2024-04-25T13:45:58.022 INFO:teuthology.orchestra.run.smithi112.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 3.3 MB/s | 224 kB 00:00 2024-04-25T13:45:58.089 INFO:teuthology.orchestra.run.smithi112.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 406 kB/s | 27 kB 00:00 2024-04-25T13:45:58.164 INFO:teuthology.orchestra.run.smithi112.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 421 kB/s | 32 kB 00:00 2024-04-25T13:45:58.314 INFO:teuthology.orchestra.run.smithi112.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.9 MB/s | 756 kB 00:00 2024-04-25T13:45:58.398 INFO:teuthology.orchestra.run.smithi112.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.8 MB/s | 158 kB 00:00 2024-04-25T13:45:58.465 INFO:teuthology.orchestra.run.smithi112.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.9 MB/s | 127 kB 00:00 2024-04-25T13:45:58.540 INFO:teuthology.orchestra.run.smithi112.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 6.5 MB/s | 497 kB 00:00 2024-04-25T13:45:58.547 INFO:teuthology.orchestra.run.smithi078.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 4.5 MB/s | 4.9 MB 00:01 2024-04-25T13:45:59.091 INFO:teuthology.orchestra.run.smithi112.stdout:(18/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 6.0 MB/s | 18 MB 00:02 2024-04-25T13:45:59.166 INFO:teuthology.orchestra.run.smithi112.stdout:(19/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 8.2 MB/s | 5.2 MB 00:00 2024-04-25T13:45:59.191 INFO:teuthology.orchestra.run.smithi112.stdout:(20/152): python3-ceph-argparse-19.0.0-3291.g99 451 kB/s | 45 kB 00:00 2024-04-25T13:45:59.225 INFO:teuthology.orchestra.run.smithi112.stdout:(21/152): python3-ceph-common-19.0.0-3291.g994e 2.3 MB/s | 135 kB 00:00 2024-04-25T13:45:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:(22/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.7 MB/s | 161 kB 00:00 2024-04-25T13:45:59.292 INFO:teuthology.orchestra.run.smithi112.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 4.7 MB/s | 321 kB 00:00 2024-04-25T13:45:59.326 INFO:teuthology.orchestra.run.smithi112.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.9 MB/s | 298 kB 00:00 2024-04-25T13:45:59.351 INFO:teuthology.orchestra.run.smithi112.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.6 MB/s | 100 kB 00:00 2024-04-25T13:45:59.377 INFO:teuthology.orchestra.run.smithi112.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 86 kB 00:00 2024-04-25T13:45:59.469 INFO:teuthology.orchestra.run.smithi112.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.8 MB/s | 169 kB 00:00 2024-04-25T13:45:59.544 INFO:teuthology.orchestra.run.smithi112.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 456 kB/s | 34 kB 00:00 2024-04-25T13:45:59.619 INFO:teuthology.orchestra.run.smithi112.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.8 MB/s | 142 kB 00:00 2024-04-25T13:45:59.678 INFO:teuthology.orchestra.run.smithi112.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 9.3 MB/s | 3.0 MB 00:00 2024-04-25T13:46:00.337 INFO:teuthology.orchestra.run.smithi112.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 4.9 MB/s | 3.5 MB 00:00 2024-04-25T13:46:00.412 INFO:teuthology.orchestra.run.smithi112.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 3.3 MB/s | 249 kB 00:00 2024-04-25T13:46:00.520 INFO:teuthology.orchestra.run.smithi112.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 8.8 MB/s | 7.4 MB 00:00 2024-04-25T13:46:00.537 INFO:teuthology.orchestra.run.smithi112.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 393 kB/s | 49 kB 00:00 2024-04-25T13:46:00.554 INFO:teuthology.orchestra.run.smithi112.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 489 kB/s | 16 kB 00:00 2024-04-25T13:46:00.588 INFO:teuthology.orchestra.run.smithi112.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 5.3 MB/s | 273 kB 00:00 2024-04-25T13:46:00.663 INFO:teuthology.orchestra.run.smithi112.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-25T13:46:00.688 INFO:teuthology.orchestra.run.smithi112.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 182 kB/s | 18 kB 00:00 2024-04-25T13:46:00.939 INFO:teuthology.orchestra.run.smithi112.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 292 kB/s | 80 kB 00:00 2024-04-25T13:46:01.064 INFO:teuthology.orchestra.run.smithi112.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-25T13:46:01.098 INFO:teuthology.orchestra.run.smithi112.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-25T13:46:01.165 INFO:teuthology.orchestra.run.smithi112.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi112.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-04-25T13:46:01.349 INFO:teuthology.orchestra.run.smithi112.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 583 kB/s | 106 kB 00:00 2024-04-25T13:46:01.399 INFO:teuthology.orchestra.run.smithi112.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-25T13:46:01.424 INFO:teuthology.orchestra.run.smithi112.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-25T13:46:01.449 INFO:teuthology.orchestra.run.smithi112.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T13:46:01.508 INFO:teuthology.orchestra.run.smithi112.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-04-25T13:46:01.583 INFO:teuthology.orchestra.run.smithi112.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-04-25T13:46:01.609 INFO:teuthology.orchestra.run.smithi112.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-25T13:46:01.634 INFO:teuthology.orchestra.run.smithi112.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T13:46:01.639 INFO:teuthology.orchestra.run.smithi078.stdout:(7/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x86 5.7 MB/s | 18 MB 00:03 2024-04-25T13:46:01.659 INFO:teuthology.orchestra.run.smithi112.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi112.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-04-25T13:46:01.810 INFO:teuthology.orchestra.run.smithi112.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-04-25T13:46:01.843 INFO:teuthology.orchestra.run.smithi112.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 115 kB/s | 18 kB 00:00 2024-04-25T13:46:01.902 INFO:teuthology.orchestra.run.smithi112.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-25T13:46:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-25T13:46:01.969 INFO:teuthology.orchestra.run.smithi112.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 680 kB/s | 45 kB 00:00 2024-04-25T13:46:02.011 INFO:teuthology.orchestra.run.smithi112.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-04-25T13:46:02.036 INFO:teuthology.orchestra.run.smithi112.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-25T13:46:02.061 INFO:teuthology.orchestra.run.smithi112.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 520 kB/s | 25 kB 00:00 2024-04-25T13:46:02.087 INFO:teuthology.orchestra.run.smithi112.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T13:46:02.112 INFO:teuthology.orchestra.run.smithi112.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T13:46:02.137 INFO:teuthology.orchestra.run.smithi112.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T13:46:02.154 INFO:teuthology.orchestra.run.smithi112.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 849 kB/s | 35 kB 00:00 2024-04-25T13:46:02.388 INFO:teuthology.orchestra.run.smithi112.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-25T13:46:02.472 INFO:teuthology.orchestra.run.smithi112.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T13:46:02.513 INFO:teuthology.orchestra.run.smithi112.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T13:46:02.556 INFO:teuthology.orchestra.run.smithi112.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T13:46:02.581 INFO:teuthology.orchestra.run.smithi078.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 11 MB/s | 10 MB 00:00 2024-04-25T13:46:02.598 INFO:teuthology.orchestra.run.smithi112.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T13:46:02.623 INFO:teuthology.orchestra.run.smithi078.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 609 kB/s | 25 kB 00:00 2024-04-25T13:46:02.957 INFO:teuthology.orchestra.run.smithi112.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T13:46:03.010 INFO:teuthology.orchestra.run.smithi112.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.4 MB/s | 1.0 MB 00:00 2024-04-25T13:46:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-25T13:46:03.307 INFO:teuthology.orchestra.run.smithi112.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-04-25T13:46:03.332 INFO:teuthology.orchestra.run.smithi112.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-04-25T13:46:03.358 INFO:teuthology.orchestra.run.smithi112.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T13:46:03.383 INFO:teuthology.orchestra.run.smithi112.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-04-25T13:46:03.406 INFO:teuthology.orchestra.run.smithi078.stdout:(10/152): ceph-base-19.0.0-3291.g994e2f20.el9.x 652 kB/s | 5.4 MB 00:08 2024-04-25T13:46:03.408 INFO:teuthology.orchestra.run.smithi112.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-25T13:46:03.425 INFO:teuthology.orchestra.run.smithi112.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T13:46:03.459 INFO:teuthology.orchestra.run.smithi112.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-04-25T13:46:03.682 INFO:teuthology.orchestra.run.smithi078.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 817 kB/s | 224 kB 00:00 2024-04-25T13:46:03.717 INFO:teuthology.orchestra.run.smithi112.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-25T13:46:03.742 INFO:teuthology.orchestra.run.smithi112.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 886 kB/s | 250 kB 00:00 2024-04-25T13:46:03.748 INFO:teuthology.orchestra.run.smithi078.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 407 kB/s | 27 kB 00:00 2024-04-25T13:46:03.784 INFO:teuthology.orchestra.run.smithi112.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-04-25T13:46:03.810 INFO:teuthology.orchestra.run.smithi112.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T13:46:03.815 INFO:teuthology.orchestra.run.smithi078.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 476 kB/s | 32 kB 00:00 2024-04-25T13:46:03.835 INFO:teuthology.orchestra.run.smithi112.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-04-25T13:46:03.869 INFO:teuthology.orchestra.run.smithi112.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T13:46:03.894 INFO:teuthology.orchestra.run.smithi112.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T13:46:03.919 INFO:teuthology.orchestra.run.smithi112.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T13:46:03.945 INFO:teuthology.orchestra.run.smithi112.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T13:46:03.970 INFO:teuthology.orchestra.run.smithi112.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T13:46:04.337 INFO:teuthology.orchestra.run.smithi112.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 114 kB/s | 42 kB 00:00 2024-04-25T13:46:04.432 INFO:teuthology.orchestra.run.smithi078.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 1.2 MB/s | 756 kB 00:00 2024-04-25T13:46:04.608 INFO:teuthology.orchestra.run.smithi078.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 903 kB/s | 158 kB 00:00 2024-04-25T13:46:04.645 INFO:teuthology.orchestra.run.smithi112.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T13:46:04.670 INFO:teuthology.orchestra.run.smithi112.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 133 kB/s | 44 kB 00:00 2024-04-25T13:46:04.696 INFO:teuthology.orchestra.run.smithi112.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T13:46:04.716 INFO:teuthology.orchestra.run.smithi078.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.1 MB/s | 127 kB 00:00 2024-04-25T13:46:04.721 INFO:teuthology.orchestra.run.smithi112.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-04-25T13:46:04.755 INFO:teuthology.orchestra.run.smithi112.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-25T13:46:04.780 INFO:teuthology.orchestra.run.smithi112.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T13:46:04.797 INFO:teuthology.orchestra.run.smithi112.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-04-25T13:46:04.824 INFO:teuthology.orchestra.run.smithi112.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.1 MB/s | 94 kB 00:00 2024-04-25T13:46:04.841 INFO:teuthology.orchestra.run.smithi112.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.4 MB/s | 64 kB 00:00 2024-04-25T13:46:04.858 INFO:teuthology.orchestra.run.smithi112.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T13:46:05.017 INFO:teuthology.orchestra.run.smithi112.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-04-25T13:46:05.058 INFO:teuthology.orchestra.run.smithi078.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 1.4 MB/s | 497 kB 00:00 2024-04-25T13:46:05.084 INFO:teuthology.orchestra.run.smithi112.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-04-25T13:46:05.193 INFO:teuthology.orchestra.run.smithi112.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-04-25T13:46:05.235 INFO:teuthology.orchestra.run.smithi112.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-04-25T13:46:05.259 INFO:teuthology.orchestra.run.smithi112.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-25T13:46:05.310 INFO:teuthology.orchestra.run.smithi112.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-04-25T13:46:05.360 INFO:teuthology.orchestra.run.smithi112.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-04-25T13:46:05.410 INFO:teuthology.orchestra.run.smithi112.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T13:46:05.461 INFO:teuthology.orchestra.run.smithi112.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-25T13:46:05.536 INFO:teuthology.orchestra.run.smithi112.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-25T13:46:05.586 INFO:teuthology.orchestra.run.smithi112.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-25T13:46:05.612 INFO:teuthology.orchestra.run.smithi112.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T13:46:05.637 INFO:teuthology.orchestra.run.smithi112.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-04-25T13:46:05.662 INFO:teuthology.orchestra.run.smithi112.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-25T13:46:05.688 INFO:teuthology.orchestra.run.smithi112.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T13:46:05.713 INFO:teuthology.orchestra.run.smithi112.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-25T13:46:05.755 INFO:teuthology.orchestra.run.smithi112.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-25T13:46:05.780 INFO:teuthology.orchestra.run.smithi112.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-25T13:46:05.805 INFO:teuthology.orchestra.run.smithi112.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 707 kB/s | 35 kB 00:00 2024-04-25T13:46:05.830 INFO:teuthology.orchestra.run.smithi112.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T13:46:05.881 INFO:teuthology.orchestra.run.smithi112.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-25T13:46:05.922 INFO:teuthology.orchestra.run.smithi112.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-25T13:46:05.981 INFO:teuthology.orchestra.run.smithi112.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T13:46:05.998 INFO:teuthology.orchestra.run.smithi112.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-25T13:46:06.015 INFO:teuthology.orchestra.run.smithi112.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 693 kB/s | 23 kB 00:00 2024-04-25T13:46:06.032 INFO:teuthology.orchestra.run.smithi112.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-04-25T13:46:06.057 INFO:teuthology.orchestra.run.smithi112.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 470 kB/s | 20 kB 00:00 2024-04-25T13:46:06.116 INFO:teuthology.orchestra.run.smithi112.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T13:46:06.141 INFO:teuthology.orchestra.run.smithi112.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-04-25T13:46:06.167 INFO:teuthology.orchestra.run.smithi112.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T13:46:06.192 INFO:teuthology.orchestra.run.smithi112.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T13:46:06.217 INFO:teuthology.orchestra.run.smithi112.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-25T13:46:06.243 INFO:teuthology.orchestra.run.smithi112.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-04-25T13:46:06.268 INFO:teuthology.orchestra.run.smithi112.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T13:46:06.293 INFO:teuthology.orchestra.run.smithi112.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-04-25T13:46:06.318 INFO:teuthology.orchestra.run.smithi112.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-04-25T13:46:06.344 INFO:teuthology.orchestra.run.smithi112.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-25T13:46:06.369 INFO:teuthology.orchestra.run.smithi112.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-04-25T13:46:06.394 INFO:teuthology.orchestra.run.smithi112.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-25T13:46:06.408 INFO:teuthology.orchestra.run.smithi078.stdout:(18/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 11 MB/s | 43 MB 00:03 2024-04-25T13:46:06.419 INFO:teuthology.orchestra.run.smithi112.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T13:46:06.445 INFO:teuthology.orchestra.run.smithi112.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-25T13:46:06.467 INFO:teuthology.orchestra.run.smithi078.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 777 kB/s | 45 kB 00:00 2024-04-25T13:46:06.470 INFO:teuthology.orchestra.run.smithi112.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-25T13:46:06.495 INFO:teuthology.orchestra.run.smithi112.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-04-25T13:46:06.520 INFO:teuthology.orchestra.run.smithi112.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-25T13:46:06.526 INFO:teuthology.orchestra.run.smithi078.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.3 MB/s | 135 kB 00:00 2024-04-25T13:46:06.546 INFO:teuthology.orchestra.run.smithi112.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T13:46:06.576 INFO:teuthology.orchestra.run.smithi078.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 3.2 MB/s | 161 kB 00:00 2024-04-25T13:46:06.643 INFO:teuthology.orchestra.run.smithi078.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 4.7 MB/s | 321 kB 00:00 2024-04-25T13:46:06.646 INFO:teuthology.orchestra.run.smithi112.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T13:46:06.693 INFO:teuthology.orchestra.run.smithi078.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 5.8 MB/s | 298 kB 00:00 2024-04-25T13:46:06.752 INFO:teuthology.orchestra.run.smithi078.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 100 kB 00:00 2024-04-25T13:46:06.811 INFO:teuthology.orchestra.run.smithi078.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.4 MB/s | 86 kB 00:00 2024-04-25T13:46:07.146 INFO:teuthology.orchestra.run.smithi112.stdout:(148/152): librados2-19.0.0-3291.g994e2f20.el9. 5.5 MB/s | 3.3 MB 00:00 2024-04-25T13:46:07.205 INFO:teuthology.orchestra.run.smithi112.stdout:(149/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-04-25T13:46:07.272 INFO:teuthology.orchestra.run.smithi112.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-04-25T13:46:07.286 INFO:teuthology.orchestra.run.smithi078.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 6.4 MB/s | 3.0 MB 00:00 2024-04-25T13:46:07.336 INFO:teuthology.orchestra.run.smithi078.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 169 kB 00:00 2024-04-25T13:46:07.379 INFO:teuthology.orchestra.run.smithi078.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 818 kB/s | 34 kB 00:00 2024-04-25T13:46:07.397 INFO:teuthology.orchestra.run.smithi112.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-04-25T13:46:07.429 INFO:teuthology.orchestra.run.smithi078.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.8 MB/s | 142 kB 00:00 2024-04-25T13:46:07.829 INFO:teuthology.orchestra.run.smithi078.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 8.8 MB/s | 3.5 MB 00:00 2024-04-25T13:46:08.355 INFO:teuthology.orchestra.run.smithi078.stdout:(31/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 1.6 MB/s | 5.2 MB 00:03 2024-04-25T13:46:08.539 INFO:teuthology.orchestra.run.smithi078.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-25T13:46:08.572 INFO:teuthology.orchestra.run.smithi078.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.1 MB/s | 249 kB 00:00 2024-04-25T13:46:08.597 INFO:teuthology.orchestra.run.smithi078.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 845 kB/s | 49 kB 00:00 2024-04-25T13:46:08.806 INFO:teuthology.orchestra.run.smithi078.stdout:(35/152): ceph-common-19.0.0-3291.g994e2f20.el9 1.4 MB/s | 20 MB 00:13 2024-04-25T13:46:08.831 INFO:teuthology.orchestra.run.smithi078.stdout:(36/152): ceph-prometheus-alerts-19.0.0-3291.g9 63 kB/s | 16 kB 00:00 2024-04-25T13:46:08.856 INFO:teuthology.orchestra.run.smithi078.stdout:(37/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.0 MB/s | 273 kB 00:00 2024-04-25T13:46:08.861 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 857 kB/s | 21 MB 00:25 2024-04-25T13:46:08.923 INFO:teuthology.orchestra.run.smithi078.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 198 kB/s | 18 kB 00:00 2024-04-25T13:46:08.999 INFO:teuthology.orchestra.run.smithi078.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 566 kB/s | 80 kB 00:00 2024-04-25T13:46:09.074 INFO:teuthology.orchestra.run.smithi078.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.3 MB/s | 803 kB 00:00 2024-04-25T13:46:09.099 INFO:teuthology.orchestra.run.smithi078.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T13:46:09.133 INFO:teuthology.orchestra.run.smithi078.stdout:(42/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.3 MB/s | 762 kB 00:00 2024-04-25T13:46:09.158 INFO:teuthology.orchestra.run.smithi078.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-25T13:46:09.191 INFO:teuthology.orchestra.run.smithi078.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-25T13:46:09.217 INFO:teuthology.orchestra.run.smithi078.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-25T13:46:09.242 INFO:teuthology.orchestra.run.smithi078.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-25T13:46:09.276 INFO:teuthology.orchestra.run.smithi078.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T13:46:09.301 INFO:teuthology.orchestra.run.smithi078.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-25T13:46:09.326 INFO:teuthology.orchestra.run.smithi078.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-04-25T13:46:09.360 INFO:teuthology.orchestra.run.smithi078.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T13:46:09.385 INFO:teuthology.orchestra.run.smithi078.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-25T13:46:09.410 INFO:teuthology.orchestra.run.smithi078.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 263 kB/s | 21 kB 00:00 2024-04-25T13:46:09.435 INFO:teuthology.orchestra.run.smithi078.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T13:46:09.461 INFO:teuthology.orchestra.run.smithi078.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 33 kB 00:00 2024-04-25T13:46:09.511 INFO:teuthology.orchestra.run.smithi078.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 238 kB/s | 11 kB 00:00 2024-04-25T13:46:09.537 INFO:teuthology.orchestra.run.smithi078.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 252 kB/s | 18 kB 00:00 2024-04-25T13:46:09.587 INFO:teuthology.orchestra.run.smithi078.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-25T13:46:09.612 INFO:teuthology.orchestra.run.smithi078.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-25T13:46:09.637 INFO:teuthology.orchestra.run.smithi078.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 460 kB/s | 45 kB 00:00 2024-04-25T13:46:09.679 INFO:teuthology.orchestra.run.smithi078.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-25T13:46:09.705 INFO:teuthology.orchestra.run.smithi078.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-25T13:46:09.730 INFO:teuthology.orchestra.run.smithi078.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 283 kB/s | 25 kB 00:00 2024-04-25T13:46:09.755 INFO:teuthology.orchestra.run.smithi078.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-04-25T13:46:09.810 INFO:teuthology.orchestra.run.smithi078.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 188 kB 00:00 2024-04-25T13:46:09.839 INFO:teuthology.orchestra.run.smithi078.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-25T13:46:09.864 INFO:teuthology.orchestra.run.smithi078.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-04-25T13:46:10.039 INFO:teuthology.orchestra.run.smithi078.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.2 MB/s | 1.6 MB 00:00 2024-04-25T13:46:10.106 INFO:teuthology.orchestra.run.smithi078.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T13:46:10.165 INFO:teuthology.orchestra.run.smithi078.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-04-25T13:46:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.7 MB/s | 911 kB 00:00 2024-04-25T13:46:10.215 INFO:teuthology.orchestra.run.smithi078.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T13:46:10.282 INFO:teuthology.orchestra.run.smithi078.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-25T13:46:10.307 INFO:teuthology.orchestra.run.smithi078.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 956 kB/s | 87 kB 00:00 2024-04-25T13:46:10.433 INFO:teuthology.orchestra.run.smithi078.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-25T13:46:10.474 INFO:teuthology.orchestra.run.smithi078.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-25T13:46:10.566 INFO:teuthology.orchestra.run.smithi078.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-04-25T13:46:10.592 INFO:teuthology.orchestra.run.smithi078.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-04-25T13:46:10.617 INFO:teuthology.orchestra.run.smithi078.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 183 kB/s | 26 kB 00:00 2024-04-25T13:46:10.642 INFO:teuthology.orchestra.run.smithi078.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-25T13:46:10.667 INFO:teuthology.orchestra.run.smithi078.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-25T13:46:10.693 INFO:teuthology.orchestra.run.smithi078.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T13:46:10.726 INFO:teuthology.orchestra.run.smithi078.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-25T13:46:10.776 INFO:teuthology.orchestra.run.smithi078.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.7 MB/s | 222 kB 00:00 2024-04-25T13:46:10.844 INFO:teuthology.orchestra.run.smithi078.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-25T13:46:10.869 INFO:teuthology.orchestra.run.smithi078.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 549 kB/s | 77 kB 00:00 2024-04-25T13:46:10.902 INFO:teuthology.orchestra.run.smithi078.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 343 kB/s | 42 kB 00:00 2024-04-25T13:46:10.930 INFO:teuthology.orchestra.run.smithi078.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T13:46:10.955 INFO:teuthology.orchestra.run.smithi078.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-04-25T13:46:10.980 INFO:teuthology.orchestra.run.smithi078.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 279 kB 00:00 2024-04-25T13:46:11.005 INFO:teuthology.orchestra.run.smithi078.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 683 kB/s | 51 kB 00:00 2024-04-25T13:46:11.030 INFO:teuthology.orchestra.run.smithi078.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T13:46:11.089 INFO:teuthology.orchestra.run.smithi078.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 502 kB/s | 42 kB 00:00 2024-04-25T13:46:11.131 INFO:teuthology.orchestra.run.smithi078.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 450 kB/s | 44 kB 00:00 2024-04-25T13:46:11.281 INFO:teuthology.orchestra.run.smithi078.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-04-25T13:46:11.390 INFO:teuthology.orchestra.run.smithi078.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 140 kB/s | 36 kB 00:00 2024-04-25T13:46:11.557 INFO:teuthology.orchestra.run.smithi078.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-25T13:46:11.582 INFO:teuthology.orchestra.run.smithi078.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 159 kB/s | 47 kB 00:00 2024-04-25T13:46:11.607 INFO:teuthology.orchestra.run.smithi078.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-04-25T13:46:11.633 INFO:teuthology.orchestra.run.smithi078.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T13:46:11.658 INFO:teuthology.orchestra.run.smithi078.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T13:46:11.683 INFO:teuthology.orchestra.run.smithi078.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-25T13:46:11.708 INFO:teuthology.orchestra.run.smithi078.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-25T13:46:11.734 INFO:teuthology.orchestra.run.smithi078.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 448 kB/s | 22 kB 00:00 2024-04-25T13:46:11.759 INFO:teuthology.orchestra.run.smithi078.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T13:46:12.201 INFO:teuthology.orchestra.run.smithi078.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 44 kB/s | 19 kB 00:00 2024-04-25T13:46:12.635 INFO:teuthology.orchestra.run.smithi078.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 343 kB/s | 309 kB 00:00 2024-04-25T13:46:12.818 INFO:teuthology.orchestra.run.smithi078.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 494 kB/s | 548 kB 00:01 2024-04-25T13:46:12.835 INFO:teuthology.orchestra.run.smithi078.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-04-25T13:46:13.044 INFO:teuthology.orchestra.run.smithi078.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 216 kB/s | 49 kB 00:00 2024-04-25T13:46:13.069 INFO:teuthology.orchestra.run.smithi078.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 288 kB/s | 67 kB 00:00 2024-04-25T13:46:13.219 INFO:teuthology.orchestra.run.smithi078.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 862 kB/s | 151 kB 00:00 2024-04-25T13:46:13.411 INFO:teuthology.orchestra.run.smithi078.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-25T13:46:13.578 INFO:teuthology.orchestra.run.smithi078.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-04-25T13:46:13.603 INFO:teuthology.orchestra.run.smithi078.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 226 kB/s | 43 kB 00:00 2024-04-25T13:46:13.908 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T13:46:14.154 INFO:teuthology.orchestra.run.smithi078.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 4.4 MB 00:01 2024-04-25T13:46:14.181 INFO:teuthology.orchestra.run.smithi078.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 26 kB/s | 15 kB 00:00 2024-04-25T13:46:14.313 INFO:teuthology.orchestra.run.smithi078.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 44 kB/s | 32 kB 00:00 2024-04-25T13:46:14.355 INFO:teuthology.orchestra.run.smithi078.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 856 kB/s | 172 kB 00:00 2024-04-25T13:46:14.547 INFO:teuthology.orchestra.run.smithi078.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 184 kB/s | 35 kB 00:00 2024-04-25T13:46:14.639 INFO:teuthology.orchestra.run.smithi078.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 931 kB/s | 303 kB 00:00 2024-04-25T13:46:14.714 INFO:teuthology.orchestra.run.smithi078.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.2 MB/s | 206 kB 00:00 2024-04-25T13:46:14.889 INFO:teuthology.orchestra.run.smithi078.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 825 kB/s | 144 kB 00:00 2024-04-25T13:46:15.048 INFO:teuthology.orchestra.run.smithi078.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2024-04-25T13:46:15.156 INFO:teuthology.orchestra.run.smithi078.stdout:(124/152): python3-cherrypy-18.6.1-2.el9.noarch 367 kB/s | 358 kB 00:00 2024-04-25T13:46:15.215 INFO:teuthology.orchestra.run.smithi078.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 106 kB/s | 18 kB 00:00 2024-04-25T13:46:15.332 INFO:teuthology.orchestra.run.smithi078.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 132 kB/s | 23 kB 00:00 2024-04-25T13:46:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 111 kB/s | 19 kB 00:00 2024-04-25T13:46:15.516 INFO:teuthology.orchestra.run.smithi078.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 107 kB/s | 20 kB 00:00 2024-04-25T13:46:15.674 INFO:teuthology.orchestra.run.smithi078.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 292 kB/s | 46 kB 00:00 2024-04-25T13:46:15.850 INFO:teuthology.orchestra.run.smithi078.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.7 MB/s | 2.0 MB 00:01 2024-04-25T13:46:15.875 INFO:teuthology.orchestra.run.smithi078.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 393 kB/s | 79 kB 00:00 2024-04-25T13:46:15.908 INFO:teuthology.orchestra.run.smithi078.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 2.0 MB/s | 1.0 MB 00:00 2024-04-25T13:46:16.017 INFO:teuthology.orchestra.run.smithi078.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 345 kB/s | 58 kB 00:00 2024-04-25T13:46:16.042 INFO:teuthology.orchestra.run.smithi078.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-25T13:46:16.068 INFO:teuthology.orchestra.run.smithi078.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 103 kB/s | 16 kB 00:00 2024-04-25T13:46:16.185 INFO:teuthology.orchestra.run.smithi078.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 538 kB/s | 90 kB 00:00 2024-04-25T13:46:16.218 INFO:teuthology.orchestra.run.smithi078.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 161 kB/s | 28 kB 00:00 2024-04-25T13:46:16.251 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:46:16.252 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:46:16.263 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:46:16.264 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:46:16.265 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:46:16.266 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:46:16.267 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:46:16.270 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:46:16.270 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:46:16.294 INFO:teuthology.orchestra.run.smithi078.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 136 kB/s | 31 kB 00:00 2024-04-25T13:46:16.331 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-25T13:46:16.350 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-25T13:46:16.350 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-04-25T13:46:16.350 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-25T13:46:16.350 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-04-25T13:46:16.350 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:46:16.350 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T13:46:16.351 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:46:16.352 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T13:46:16.353 INFO:teuthology.orchestra.run.smithi129.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T13:46:16.354 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:46:16.355 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:46:16.356 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:46:16.357 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:46:16.358 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:46:16.359 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:46:16.360 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:46:16.361 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:46:16.362 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout:Install 148 Packages 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 4 Packages 2024-04-25T13:46:16.363 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:46:16.364 INFO:teuthology.orchestra.run.smithi078.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-04-25T13:46:16.364 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 214 M 2024-04-25T13:46:16.364 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-04-25T13:46:16.377 INFO:teuthology.orchestra.run.smithi078.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 372 kB/s | 59 kB 00:00 2024-04-25T13:46:16.453 INFO:teuthology.orchestra.run.smithi078.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 225 kB/s | 36 kB 00:00 2024-04-25T13:46:16.528 INFO:teuthology.orchestra.run.smithi078.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 291 kB/s | 51 kB 00:00 2024-04-25T13:46:16.561 INFO:teuthology.orchestra.run.smithi078.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-25T13:46:16.612 INFO:teuthology.orchestra.run.smithi078.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 564 kB/s | 90 kB 00:00 2024-04-25T13:46:16.720 INFO:teuthology.orchestra.run.smithi078.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 126 kB/s | 20 kB 00:00 2024-04-25T13:46:16.779 INFO:teuthology.orchestra.run.smithi078.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-25T13:46:16.838 INFO:teuthology.orchestra.run.smithi078.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.3 MB/s | 427 kB 00:00 2024-04-25T13:46:17.113 INFO:teuthology.orchestra.run.smithi078.stdout:(148/152): librados2-19.0.0-3291.g994e2f20.el9. 10 MB/s | 3.3 MB 00:00 2024-04-25T13:46:17.205 INFO:teuthology.orchestra.run.smithi078.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-04-25T13:46:17.255 INFO:teuthology.orchestra.run.smithi129.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-25T13:46:17.263 INFO:teuthology.orchestra.run.smithi112.stdout:(152/152): ceph-test-19.0.0-3291.g994e2f20.el9. 2.2 MB/s | 43 MB 00:19 2024-04-25T13:46:17.271 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:46:17.271 INFO:teuthology.orchestra.run.smithi112.stdout:Total 8.5 MB/s | 214 MB 00:25 2024-04-25T13:46:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-25T13:46:17.531 INFO:teuthology.orchestra.run.smithi078.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-04-25T13:46:17.635 INFO:teuthology.orchestra.run.smithi129.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.2 MB/s | 863 kB 00:00 2024-04-25T13:46:17.727 INFO:teuthology.orchestra.run.smithi129.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.6 MB/s | 147 kB 00:00 2024-04-25T13:46:18.323 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T13:46:18.410 INFO:teuthology.orchestra.run.smithi129.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 2.3 MB 00:00 2024-04-25T13:46:18.464 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T13:46:18.464 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T13:46:18.619 INFO:teuthology.orchestra.run.smithi129.stdout:(5/152): ceph-common-19.0.0-3291.g994e2f20.el9. 12 MB/s | 20 MB 00:01 2024-04-25T13:46:18.647 INFO:teuthology.orchestra.run.smithi078.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 3.1 MB 00:01 2024-04-25T13:46:18.651 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:46:18.652 INFO:teuthology.orchestra.run.smithi078.stdout:Total 8.7 MB/s | 214 MB 00:24 2024-04-25T13:46:18.694 INFO:teuthology.orchestra.run.smithi129.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.2 MB/s | 5.4 MB 00:01 2024-04-25T13:46:18.786 INFO:teuthology.orchestra.run.smithi129.stdout:(7/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-25T13:46:18.978 INFO:teuthology.orchestra.run.smithi129.stdout:(8/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 14 MB/s | 4.9 MB 00:00 2024-04-25T13:46:19.077 INFO:teuthology.orchestra.run.smithi129.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 255 kB/s | 25 kB 00:00 2024-04-25T13:46:19.762 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-25T13:46:19.903 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-25T13:46:19.903 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-25T13:46:20.172 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T13:46:20.172 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T13:46:20.245 INFO:teuthology.orchestra.run.smithi129.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 6.9 MB/s | 10 MB 00:01 2024-04-25T13:46:20.354 INFO:teuthology.orchestra.run.smithi129.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.0 MB/s | 224 kB 00:00 2024-04-25T13:46:20.454 INFO:teuthology.orchestra.run.smithi129.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 271 kB/s | 27 kB 00:00 2024-04-25T13:46:20.562 INFO:teuthology.orchestra.run.smithi129.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 293 kB/s | 32 kB 00:00 2024-04-25T13:46:20.779 INFO:teuthology.orchestra.run.smithi129.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.4 MB/s | 756 kB 00:00 2024-04-25T13:46:20.913 INFO:teuthology.orchestra.run.smithi129.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.2 MB/s | 158 kB 00:00 2024-04-25T13:46:21.030 INFO:teuthology.orchestra.run.smithi129.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.1 MB/s | 127 kB 00:00 2024-04-25T13:46:21.280 INFO:teuthology.orchestra.run.smithi129.stdout:(17/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 6.9 MB/s | 18 MB 00:02 2024-04-25T13:46:21.314 INFO:teuthology.orchestra.run.smithi129.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 1.7 MB/s | 497 kB 00:00 2024-04-25T13:46:21.415 INFO:teuthology.orchestra.run.smithi129.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 451 kB/s | 45 kB 00:00 2024-04-25T13:46:21.548 INFO:teuthology.orchestra.run.smithi129.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.0 MB/s | 135 kB 00:00 2024-04-25T13:46:21.614 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-25T13:46:21.615 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-25T13:46:21.673 INFO:teuthology.orchestra.run.smithi129.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.3 MB/s | 161 kB 00:00 2024-04-25T13:46:21.765 INFO:teuthology.orchestra.run.smithi129.stdout:(22/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 11 MB/s | 5.2 MB 00:00 2024-04-25T13:46:21.791 INFO:teuthology.orchestra.run.smithi129.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 2.7 MB/s | 321 kB 00:00 2024-04-25T13:46:22.179 INFO:teuthology.orchestra.run.smithi129.stdout:(24/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 14 MB/s | 43 MB 00:03 2024-04-25T13:46:22.208 INFO:teuthology.orchestra.run.smithi129.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 239 kB/s | 100 kB 00:00 2024-04-25T13:46:22.233 INFO:teuthology.orchestra.run.smithi129.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.6 MB/s | 86 kB 00:00 2024-04-25T13:46:22.258 INFO:teuthology.orchestra.run.smithi129.stdout:(27/152): python3-rbd-19.0.0-3291.g994e2f20.el9 605 kB/s | 298 kB 00:00 2024-04-25T13:46:22.284 INFO:teuthology.orchestra.run.smithi129.stdout:(28/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 169 kB 00:00 2024-04-25T13:46:22.342 INFO:teuthology.orchestra.run.smithi129.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.4 MB/s | 142 kB 00:00 2024-04-25T13:46:22.367 INFO:teuthology.orchestra.run.smithi129.stdout:(30/152): ceph-grafana-dashboards-19.0.0-3291.g 314 kB/s | 34 kB 00:00 2024-04-25T13:46:22.668 INFO:teuthology.orchestra.run.smithi129.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 11 MB/s | 3.5 MB 00:00 2024-04-25T13:46:22.734 INFO:teuthology.orchestra.run.smithi129.stdout:(32/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 5.8 MB/s | 3.0 MB 00:00 2024-04-25T13:46:22.760 INFO:teuthology.orchestra.run.smithi129.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.6 MB/s | 249 kB 00:00 2024-04-25T13:46:22.802 INFO:teuthology.orchestra.run.smithi129.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 397 kB/s | 16 kB 00:00 2024-04-25T13:46:22.827 INFO:teuthology.orchestra.run.smithi129.stdout:(35/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 536 kB/s | 49 kB 00:00 2024-04-25T13:46:22.869 INFO:teuthology.orchestra.run.smithi129.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 4.0 MB/s | 273 kB 00:00 2024-04-25T13:46:22.919 INFO:teuthology.orchestra.run.smithi129.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 365 kB/s | 18 kB 00:00 2024-04-25T13:46:23.020 INFO:teuthology.orchestra.run.smithi129.stdout:(38/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 3.9 MB/s | 762 kB 00:00 2024-04-25T13:46:23.178 INFO:teuthology.orchestra.run.smithi129.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 9.1 MB/s | 7.4 MB 00:00 2024-04-25T13:46:23.203 INFO:teuthology.orchestra.run.smithi129.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 283 kB/s | 80 kB 00:00 2024-04-25T13:46:23.304 INFO:teuthology.orchestra.run.smithi129.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-25T13:46:23.371 INFO:teuthology.orchestra.run.smithi129.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-25T13:46:23.504 INFO:teuthology.orchestra.run.smithi129.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-25T13:46:23.529 INFO:teuthology.orchestra.run.smithi129.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-25T13:46:23.554 INFO:teuthology.orchestra.run.smithi129.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 581 kB/s | 106 kB 00:00 2024-04-25T13:46:23.580 INFO:teuthology.orchestra.run.smithi129.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-25T13:46:23.605 INFO:teuthology.orchestra.run.smithi129.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T13:46:23.639 INFO:teuthology.orchestra.run.smithi129.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-25T13:46:23.664 INFO:teuthology.orchestra.run.smithi129.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-04-25T13:46:23.698 INFO:teuthology.orchestra.run.smithi129.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-25T13:46:23.723 INFO:teuthology.orchestra.run.smithi129.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-25T13:46:23.748 INFO:teuthology.orchestra.run.smithi129.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 259 kB/s | 21 kB 00:00 2024-04-25T13:46:23.773 INFO:teuthology.orchestra.run.smithi129.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T13:46:23.798 INFO:teuthology.orchestra.run.smithi129.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-04-25T13:46:23.899 INFO:teuthology.orchestra.run.smithi129.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-04-25T13:46:23.924 INFO:teuthology.orchestra.run.smithi129.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 18 kB 00:00 2024-04-25T13:46:23.949 INFO:teuthology.orchestra.run.smithi129.stdout:(57/152): libaio-devel-0.3.111-13.el9.x86_64.rp 75 kB/s | 11 kB 00:00 2024-04-25T13:46:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-25T13:46:23.991 INFO:teuthology.orchestra.run.smithi129.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T13:46:24.016 INFO:teuthology.orchestra.run.smithi129.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T13:46:24.050 INFO:teuthology.orchestra.run.smithi129.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-25T13:46:24.075 INFO:teuthology.orchestra.run.smithi129.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 310 kB/s | 25 kB 00:00 2024-04-25T13:46:24.101 INFO:teuthology.orchestra.run.smithi129.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 572 kB/s | 47 kB 00:00 2024-04-25T13:46:24.126 INFO:teuthology.orchestra.run.smithi129.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T13:46:24.183 INFO:teuthology.orchestra.run.smithi129.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 188 kB 00:00 2024-04-25T13:46:24.209 INFO:teuthology.orchestra.run.smithi129.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 326 kB/s | 35 kB 00:00 2024-04-25T13:46:24.318 INFO:teuthology.orchestra.run.smithi129.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-25T13:46:24.410 INFO:teuthology.orchestra.run.smithi129.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 8.1 MB/s | 1.6 MB 00:00 2024-04-25T13:46:24.477 INFO:teuthology.orchestra.run.smithi129.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T13:46:24.519 INFO:teuthology.orchestra.run.smithi129.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-04-25T13:46:24.544 INFO:teuthology.orchestra.run.smithi129.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-25T13:46:24.719 INFO:teuthology.orchestra.run.smithi129.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-25T13:46:24.753 INFO:teuthology.orchestra.run.smithi129.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-25T13:46:24.778 INFO:teuthology.orchestra.run.smithi129.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 379 kB/s | 87 kB 00:00 2024-04-25T13:46:24.937 INFO:teuthology.orchestra.run.smithi129.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2024-04-25T13:46:24.962 INFO:teuthology.orchestra.run.smithi129.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-25T13:46:24.987 INFO:teuthology.orchestra.run.smithi129.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-04-25T13:46:25.004 INFO:teuthology.orchestra.run.smithi129.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 388 kB/s | 26 kB 00:00 2024-04-25T13:46:25.021 INFO:teuthology.orchestra.run.smithi129.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 573 kB/s | 19 kB 00:00 2024-04-25T13:46:25.046 INFO:teuthology.orchestra.run.smithi129.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T13:46:25.071 INFO:teuthology.orchestra.run.smithi129.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 522 kB/s | 35 kB 00:00 2024-04-25T13:46:25.155 INFO:teuthology.orchestra.run.smithi129.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-25T13:46:25.239 INFO:teuthology.orchestra.run.smithi129.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-04-25T13:46:25.264 INFO:teuthology.orchestra.run.smithi129.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-04-25T13:46:25.289 INFO:teuthology.orchestra.run.smithi129.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 588 kB/s | 77 kB 00:00 2024-04-25T13:46:25.314 INFO:teuthology.orchestra.run.smithi129.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 566 kB/s | 42 kB 00:00 2024-04-25T13:46:25.340 INFO:teuthology.orchestra.run.smithi129.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T13:46:25.365 INFO:teuthology.orchestra.run.smithi129.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T13:46:25.390 INFO:teuthology.orchestra.run.smithi129.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-25T13:46:25.407 INFO:teuthology.orchestra.run.smithi129.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T13:46:25.423 INFO:teuthology.orchestra.run.smithi129.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 928 kB/s | 54 kB 00:00 2024-04-25T13:46:25.632 INFO:teuthology.orchestra.run.smithi129.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 202 kB/s | 42 kB 00:00 2024-04-25T13:46:25.649 INFO:teuthology.orchestra.run.smithi129.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 198 kB/s | 44 kB 00:00 2024-04-25T13:46:25.850 INFO:teuthology.orchestra.run.smithi129.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T13:46:25.875 INFO:teuthology.orchestra.run.smithi129.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 919 kB/s | 221 kB 00:00 2024-04-25T13:46:25.892 INFO:teuthology.orchestra.run.smithi129.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 149 kB/s | 36 kB 00:00 2024-04-25T13:46:25.908 INFO:teuthology.orchestra.run.smithi129.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 819 kB/s | 47 kB 00:00 2024-04-25T13:46:25.942 INFO:teuthology.orchestra.run.smithi129.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T13:46:25.967 INFO:teuthology.orchestra.run.smithi129.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 771 kB/s | 58 kB 00:00 2024-04-25T13:46:25.992 INFO:teuthology.orchestra.run.smithi129.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-04-25T13:46:26.018 INFO:teuthology.orchestra.run.smithi129.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-25T13:46:26.043 INFO:teuthology.orchestra.run.smithi129.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-04-25T13:46:26.060 INFO:teuthology.orchestra.run.smithi129.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-04-25T13:46:26.073 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T13:46:26.102 INFO:teuthology.orchestra.run.smithi129.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T13:46:26.150 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:46:26.196 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:46:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:46:26.367 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:46:26.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:46:26.402 INFO:teuthology.orchestra.run.smithi129.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-04-25T13:46:26.436 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:46:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:46:26.886 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:46:27.002 INFO:teuthology.orchestra.run.smithi129.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 583 kB/s | 548 kB 00:00 2024-04-25T13:46:27.011 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:46:27.045 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:46:27.086 INFO:teuthology.orchestra.run.smithi129.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-25T13:46:27.113 INFO:teuthology.orchestra.run.smithi129.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 294 kB/s | 309 kB 00:01 2024-04-25T13:46:27.136 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:46:27.174 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:46:27.222 INFO:teuthology.orchestra.run.smithi129.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 358 kB/s | 49 kB 00:00 2024-04-25T13:46:27.239 INFO:teuthology.orchestra.run.smithi129.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 537 kB/s | 67 kB 00:00 2024-04-25T13:46:27.321 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:46:27.372 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:46:27.389 INFO:teuthology.orchestra.run.smithi129.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 907 kB/s | 151 kB 00:00 2024-04-25T13:46:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:46:27.456 INFO:teuthology.orchestra.run.smithi129.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-04-25T13:46:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:46:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:46:27.548 INFO:teuthology.orchestra.run.smithi129.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-25T13:46:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:46:27.606 INFO:teuthology.orchestra.run.smithi129.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-04-25T13:46:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:46:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:46:27.648 INFO:teuthology.orchestra.run.smithi129.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-04-25T13:46:27.673 INFO:teuthology.orchestra.run.smithi129.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 256 kB/s | 32 kB 00:00 2024-04-25T13:46:27.724 INFO:teuthology.orchestra.run.smithi129.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-25T13:46:27.749 INFO:teuthology.orchestra.run.smithi129.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T13:46:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-25T13:46:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:46:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:46:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:46:27.898 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:46:27.951 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:46:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:46:27.985 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:46:28.011 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:46:28.053 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:46:28.091 INFO:teuthology.orchestra.run.smithi129.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 827 kB/s | 303 kB 00:00 2024-04-25T13:46:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:46:28.124 INFO:teuthology.orchestra.run.smithi129.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 795 kB/s | 358 kB 00:00 2024-04-25T13:46:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:46:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:46:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:46:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:46:28.283 INFO:teuthology.orchestra.run.smithi129.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 66 kB/s | 35 kB 00:00 2024-04-25T13:46:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi129.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-25T13:46:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:46:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:46:28.408 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:46:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:46:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:46:28.521 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:46:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:46:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:46:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:46:28.654 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:46:28.672 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:46:28.687 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T13:46:28.731 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T13:46:28.759 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T13:46:28.762 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:46:28.793 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:46:28.829 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:46:28.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:46:28.884 INFO:teuthology.orchestra.run.smithi129.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 259 kB/s | 206 kB 00:00 2024-04-25T13:46:28.909 INFO:teuthology.orchestra.run.smithi129.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 231 kB/s | 144 kB 00:00 2024-04-25T13:46:28.934 INFO:teuthology.orchestra.run.smithi129.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 18 kB/s | 11 kB 00:00 2024-04-25T13:46:28.938 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:46:28.948 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:46:28.989 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:46:29.017 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:46:29.091 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:46:29.101 INFO:teuthology.orchestra.run.smithi129.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 117 kB/s | 19 kB 00:00 2024-04-25T13:46:29.122 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:46:29.143 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:46:29.144 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:46:29.206 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:46:29.228 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:46:29.252 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:46:29.291 INFO:teuthology.orchestra.run.smithi129.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 104 kB/s | 20 kB 00:00 2024-04-25T13:46:29.454 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:46:29.460 INFO:teuthology.orchestra.run.smithi129.stdout:(128/152): python3-jaraco-classes-3.2.1-5.el9.n 31 kB/s | 18 kB 00:00 2024-04-25T13:46:29.485 INFO:teuthology.orchestra.run.smithi129.stdout:(129/152): python3-jaraco-collections-3.0.0-8.e 40 kB/s | 23 kB 00:00 2024-04-25T13:46:29.486 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:46:29.592 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:46:29.662 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:46:29.685 INFO:teuthology.orchestra.run.smithi129.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-04-25T13:46:29.696 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:46:29.710 INFO:teuthology.orchestra.run.smithi129.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 185 kB/s | 46 kB 00:00 2024-04-25T13:46:29.735 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:46:29.736 INFO:teuthology.orchestra.run.smithi129.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 315 kB/s | 79 kB 00:00 2024-04-25T13:46:29.746 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:46:29.769 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:46:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:46:29.818 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:46:29.833 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:46:29.857 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T13:46:29.869 INFO:teuthology.orchestra.run.smithi129.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 314 kB/s | 58 kB 00:00 2024-04-25T13:46:29.877 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:46:29.898 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:46:29.911 INFO:teuthology.orchestra.run.smithi129.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-25T13:46:29.920 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:46:29.953 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:46:29.981 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:46:30.057 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:46:30.069 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:46:30.111 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:46:30.118 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:46:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:46:30.173 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:46:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:46:30.220 INFO:teuthology.orchestra.run.smithi129.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 34 kB/s | 16 kB 00:00 2024-04-25T13:46:30.244 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:46:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:46:30.245 INFO:teuthology.orchestra.run.smithi129.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 85 kB/s | 28 kB 00:00 2024-04-25T13:46:30.262 INFO:teuthology.orchestra.run.smithi129.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 229 kB/s | 90 kB 00:00 2024-04-25T13:46:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T13:46:30.285 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:46:30.321 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:46:30.323 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T13:46:30.352 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T13:46:30.368 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:46:30.387 INFO:teuthology.orchestra.run.smithi129.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-04-25T13:46:30.404 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:46:30.421 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:46:30.447 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:46:30.466 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:46:30.490 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:46:30.540 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:46:30.571 INFO:teuthology.orchestra.run.smithi129.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 195 kB/s | 36 kB 00:00 2024-04-25T13:46:30.624 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:46:30.670 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:46:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:46:30.692 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:46:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:46:30.732 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:46:30.755 INFO:teuthology.orchestra.run.smithi129.stdout:(140/152): python3-repoze-lru-0.7-16.el9.noarch 57 kB/s | 31 kB 00:00 2024-04-25T13:46:30.780 INFO:teuthology.orchestra.run.smithi129.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 353 kB/s | 188 kB 00:00 2024-04-25T13:46:30.827 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:46:30.867 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:46:30.906 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:46:30.913 INFO:teuthology.orchestra.run.smithi129.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-25T13:46:30.932 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:46:30.957 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:46:30.980 INFO:teuthology.orchestra.run.smithi129.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-04-25T13:46:31.006 INFO:teuthology.orchestra.run.smithi129.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 398 kB/s | 90 kB 00:00 2024-04-25T13:46:31.031 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:46:31.039 INFO:teuthology.orchestra.run.smithi129.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T13:46:31.159 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:46:31.198 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:46:31.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:46:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:46:31.260 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:46:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:46:31.331 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:46:31.331 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:46:31.361 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:46:31.361 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:46:31.381 INFO:teuthology.orchestra.run.smithi129.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-25T13:46:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:46:31.403 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:46:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:46:31.441 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T13:46:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:46:31.473 INFO:teuthology.orchestra.run.smithi129.stdout:(147/152): python3-zc-lockfile-2.0-10.el9.noarc 41 kB/s | 20 kB 00:00 2024-04-25T13:46:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:46:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:46:31.573 INFO:teuthology.orchestra.run.smithi129.stdout:(148/152): re2-20211101-3.el9.x86_64.rpm 338 kB/s | 192 kB 00:00 2024-04-25T13:46:31.598 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:46:31.622 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:46:31.651 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:46:31.665 INFO:teuthology.orchestra.run.smithi129.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-04-25T13:46:31.683 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:46:31.686 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:46:31.729 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:46:31.749 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:46:31.790 INFO:teuthology.orchestra.run.smithi129.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T13:46:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:46:31.827 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:46:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:46:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:46:31.906 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:46:31.937 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:46:31.981 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:46:32.023 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:46:32.024 INFO:teuthology.orchestra.run.smithi129.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-25T13:46:32.066 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:46:32.124 INFO:teuthology.orchestra.run.smithi129.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 4.5 MB/s | 3.3 MB 00:00 2024-04-25T13:46:32.131 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:46:32.131 INFO:teuthology.orchestra.run.smithi129.stdout:Total 14 MB/s | 214 MB 00:15 2024-04-25T13:46:32.200 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:46:32.255 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:46:32.297 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:46:32.339 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:46:32.428 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:46:32.469 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:46:32.500 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:46:32.526 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:46:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:46:32.623 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:46:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:46:32.673 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:46:32.707 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:46:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:46:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:46:32.806 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:46:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:46:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:46:32.865 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:46:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:46:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:46:32.951 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:46:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:46:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:46:32.992 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:46:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:46:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:46:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:46:33.171 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:46:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:46:33.209 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:46:33.212 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-04-25T13:46:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:46:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:46:33.285 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:46:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:46:33.355 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-04-25T13:46:33.355 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-04-25T13:46:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:46:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:46:33.923 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:46:34.037 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:46:34.168 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:46:34.209 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:46:34.242 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:46:34.340 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:46:34.401 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:46:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:46:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:46:34.697 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:46:34.726 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:46:34.804 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:46:35.077 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-04-25T13:46:35.077 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-04-25T13:46:35.262 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:46:35.344 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:46:35.372 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:46:35.405 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:46:35.447 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:46:35.559 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:46:35.727 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:46:35.768 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:46:35.859 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:46:35.889 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T13:46:35.925 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:46:36.414 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:46:36.446 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:46:36.517 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:46:36.539 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T13:46:36.783 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:46:36.872 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:46:36.902 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:46:36.933 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:46:37.255 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:46:37.296 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:46:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:46:37.434 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T13:46:37.470 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:46:37.963 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:46:37.991 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:46:38.061 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:46:38.084 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T13:46:38.788 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:38.827 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:39.389 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:39.430 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:46:39.522 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:46:39.522 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:46:39.523 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:46:39.523 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:39.574 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:46:40.294 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:40.339 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:40.895 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:40.943 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:46:41.008 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-04-25T13:46:41.031 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:46:41.031 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:46:41.031 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:46:41.031 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:41.060 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:46:41.078 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:46:41.116 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:46:41.170 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:46:41.212 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:46:41.237 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:46:41.280 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:46:41.608 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:46:41.730 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:46:41.855 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:46:41.881 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:46:41.972 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:46:42.011 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:46:42.207 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:46:42.235 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:46:42.339 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:46:42.361 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:46:42.432 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T13:46:42.454 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:46:42.477 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:46:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:46:42.712 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:46:42.915 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:46:42.961 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:46:42.996 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:46:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:46:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:46:43.138 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:46:43.170 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:46:43.199 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:46:43.341 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:46:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:46:43.435 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:46:43.478 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:46:43.555 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:46:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T13:46:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T13:46:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T13:46:43.724 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:46:43.784 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:46:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:46:43.993 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:46:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:46:44.577 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:46:44.716 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:46:44.920 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:46:44.962 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:46:44.992 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T13:46:45.034 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:46:45.197 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:46:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:46:45.325 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:46:45.398 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:46:45.428 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:46:45.465 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:46:45.512 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:46:45.591 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:46:45.633 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:46:45.769 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:46:45.813 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:46:45.843 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:46:45.885 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:46:45.980 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:46:46.010 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:46:46.050 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:46:46.075 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:46:46.100 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:46:46.176 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:46:46.533 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:46:46.565 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:46:46.593 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:46:46.628 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:46:46.667 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:46:46.698 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:46:46.728 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:46:46.755 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:46:46.788 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:46:46.829 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:46:46.864 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:46:46.951 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:46:46.994 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:46:47.025 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:46:47.064 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:46:47.164 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:46:47.218 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:46:47.947 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:46:47.998 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:46:48.032 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:46:48.132 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:46:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:46:48.279 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:46:48.317 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:46:48.504 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:46:48.540 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:46:48.617 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:46:49.265 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:46:49.389 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:46:50.613 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:46:50.693 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:46:50.721 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:46:50.755 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-25T13:46:50.915 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:51.076 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:46:51.095 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:46:51.110 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:46:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:51.110 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:46:51.111 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:51.117 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:46:51.208 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T13:46:51.238 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T13:46:51.275 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:46:51.764 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:46:51.795 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:46:51.849 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T13:46:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-04-25T13:46:52.348 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-04-25T13:46:52.349 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:46:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:46:52.538 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:52.539 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:46:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:52.706 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:46:52.732 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:46:52.732 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:52.732 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:46:52.732 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:46:52.732 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:46:52.732 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:52.974 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:46:52.996 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:46:52.996 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:52.996 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:46:52.996 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:46:52.996 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:46:52.996 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:46:53.488 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:46:53.488 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:53.489 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:46:53.489 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:46:53.489 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:46:53.489 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:53.588 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:46:53.645 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:46:53.699 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:46:53.883 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:46:53.933 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:46:54.011 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:46:54.093 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:46:54.098 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:46:54.105 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:54.122 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:46:54.122 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:54.123 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:46:54.123 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:46:54.123 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:46:54.123 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:54.151 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:54.230 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:46:54.275 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:46:54.307 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:46:54.335 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:46:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:46:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:46:54.382 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:54.383 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:46:54.383 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:46:54.383 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:46:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:54.708 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T13:46:54.754 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:46:54.778 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:46:54.800 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:46:54.832 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T13:46:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:46:54.847 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T13:46:54.847 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:46:54.847 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:46:54.848 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:46:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:46:54.869 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:54.869 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:46:54.870 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:46:54.870 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:46:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:54.881 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:46:54.893 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T13:46:54.923 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:46:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:46:55.033 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:46:55.088 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:46:55.274 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:46:55.322 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:46:55.391 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:46:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:46:55.611 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:46:55.648 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:46:55.679 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:46:55.707 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:46:55.977 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:46:56.016 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:46:56.158 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:46:56.189 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:46:56.221 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T13:46:56.282 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T13:46:56.312 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:46:56.991 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:46:57.017 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:46:57.118 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:46:57.285 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T13:46:57.311 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:46:57.333 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:46:57.333 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:57.333 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:46:57.333 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:46:57.333 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:46:57.334 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:57.346 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:46:57.378 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:46:57.381 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:46:57.388 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:46:57.489 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T13:46:57.543 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T13:46:58.346 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:46:58.373 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:46:58.474 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:46:58.483 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:46:58.514 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:46:58.536 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:46:58.536 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:58.536 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:46:58.536 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:46:58.536 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:46:58.536 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:58.581 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:58.648 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T13:46:58.648 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:46:58.670 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:46:58.670 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:58.670 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:46:58.670 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:46:58.670 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:46:58.670 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:58.675 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:46:58.698 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:46:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:46:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:46:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:46:58.698 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:58.742 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:46:58.754 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:46:58.852 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T13:46:58.898 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T13:46:58.982 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:46:59.004 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:46:59.004 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:59.004 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:46:59.005 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:46:59.005 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:46:59.005 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:46:59.838 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:46:59.870 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:46:59.893 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:46:59.893 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:59.893 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:46:59.893 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:46:59.893 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:46:59.893 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:46:59.938 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:46:59.960 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:46:59.960 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:46:59.960 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:46:59.960 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:47:00.005 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:47:00.027 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:47:00.028 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:00.028 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:47:00.028 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:47:00.028 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:47:00.028 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:47:00.339 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:47:00.362 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:47:00.362 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:00.362 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:47:00.362 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:47:00.362 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:47:00.362 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:47:03.328 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T13:47:03.370 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T13:47:03.410 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T13:47:03.442 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:47:03.469 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:47:03.652 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T13:47:03.699 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:47:03.729 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T13:47:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:04.626 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T13:47:04.693 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T13:47:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T13:47:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:47:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:47:04.965 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T13:47:05.006 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:47:05.036 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T13:47:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T13:47:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:47:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:47:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:47:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:47:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:47:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:47:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:47:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:47:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:47:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:05.657 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:05.700 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:05.720 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:06.546 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T13:47:06.624 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T13:47:06.624 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-04-25T13:47:06.624 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-04-25T13:47:06.624 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-04-25T13:47:06.624 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-04-25T13:47:06.625 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-04-25T13:47:06.625 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-04-25T13:47:06.625 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-04-25T13:47:06.625 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:06.639 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:47:06.680 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:47:06.761 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:47:06.791 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:47:06.793 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:47:06.793 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:47:06.806 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T13:47:06.806 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:06.806 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:47:06.806 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:06.877 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:47:06.902 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:47:06.902 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:47:06.997 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:08.395 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:47:08.420 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T13:47:08.420 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:08.420 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:47:08.420 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:47:08.420 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:47:08.420 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:47:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T13:47:08.668 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:08.668 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:47:08.668 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:47:08.669 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:47:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:09.130 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:47:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T13:47:09.152 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:09.152 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:47:09.152 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:47:09.152 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:47:09.152 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:47:09.292 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:47:09.347 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:47:09.535 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:47:09.580 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:47:09.650 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:47:09.739 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:47:09.879 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:47:09.913 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:47:09.945 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:47:09.965 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:47:10.408 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:47:10.438 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T13:47:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T13:47:10.532 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T13:47:10.569 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:47:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T13:47:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T13:47:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T13:47:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T13:47:11.600 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T13:47:11.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T13:47:11.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T13:47:11.603 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:47:11.616 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:47:11.617 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:47:11.618 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:47:11.619 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:47:11.624 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T13:47:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:47:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:47:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:47:11.625 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:47:11.625 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T13:47:11.646 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:47:12.609 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T13:47:12.647 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:47:12.749 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T13:47:12.914 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T13:47:12.940 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:47:12.963 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T13:47:12.963 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:12.963 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:47:12.963 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:47:12.964 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:47:12.964 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:13.007 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:47:13.018 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T13:47:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T13:47:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T13:47:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T13:47:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T13:47:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T13:47:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:47:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:47:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:47:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:47:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:47:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:47:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:47:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:47:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:47:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:47:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T13:47:13.173 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T13:47:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:13.692 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.693 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.694 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:47:13.695 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.696 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:47:13.697 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:47:13.698 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:47:13.699 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:47:13.700 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:47:13.701 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:47:13.702 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:47:13.703 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:13.704 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:47:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:47:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:47:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:47:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:47:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:47:13.705 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:47:13.706 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T13:47:13.706 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T13:47:14.078 DEBUG:teuthology.parallel:result is None 2024-04-25T13:47:14.112 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:47:14.152 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:47:14.174 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T13:47:14.174 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:14.174 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:47:14.174 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:47:14.174 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:47:14.174 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:47:14.240 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T13:47:14.240 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:14.240 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:47:14.240 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:14.286 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:47:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T13:47:14.308 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:14.308 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:47:14.308 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:47:14.308 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:47:14.308 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:14.630 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:47:14.651 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T13:47:14.651 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:47:14.651 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:47:14.651 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:47:14.651 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:47:14.651 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:15.218 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:47:15.219 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:15.221 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:47:15.222 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:47:15.223 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:47:15.224 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:47:15.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:47:15.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:47:15.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:47:15.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:47:15.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:47:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:47:15.231 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:47:15.232 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:47:15.232 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:47:15.232 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-25T13:47:15.626 DEBUG:teuthology.parallel:result is None 2024-04-25T13:47:18.941 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T13:47:18.982 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T13:47:19.023 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T13:47:19.055 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:47:19.082 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:47:19.280 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T13:47:19.319 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:47:19.349 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T13:47:19.984 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:20.022 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:20.041 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:47:20.873 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T13:47:20.962 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:47:21.001 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:47:21.092 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:47:21.122 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:47:21.122 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:47:21.206 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:47:21.247 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:47:21.247 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:47:21.340 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:47:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T13:47:27.371 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T13:47:27.372 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T13:47:27.372 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T13:47:27.372 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T13:47:27.372 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T13:47:27.372 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T13:47:27.372 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T13:47:27.372 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T13:47:27.373 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T13:47:27.373 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T13:47:27.373 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T13:47:27.373 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T13:47:27.373 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T13:47:27.373 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T13:47:27.373 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T13:47:27.374 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T13:47:27.391 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:47:27.392 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:47:27.393 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:47:27.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:47:27.395 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:47:27.396 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:47:27.397 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:47:27.398 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:47:27.399 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:47:27.400 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:47:27.400 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:47:27.400 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:47:27.400 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:47:27.400 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T13:47:27.400 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:47:27.401 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T13:47:27.401 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:47:27.401 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:47:27.401 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:47:27.401 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-04-25T13:47:29.579 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.580 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.581 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:47:29.582 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:47:29.583 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:47:29.584 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.585 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:47:29.586 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:47:29.587 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:47:29.588 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:47:29.589 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.590 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:47:29.591 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-25T13:47:29.973 DEBUG:teuthology.parallel:result is None 2024-04-25T13:47:29.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:47:30.214 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T13:47:30.240 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T13:47:30.241 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T13:47:30.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:47:30.385 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:47:30.410 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T13:47:30.410 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T13:47:30.411 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T13:47:30.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:47:30.631 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:47:30.652 INFO:teuthology.orchestra.run.smithi129.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T13:47:30.653 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T13:47:30.653 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T13:47:30.654 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T13:47:30.655 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:47:30.655 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:47:30.685 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:47:30.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:47:30.716 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:47:30.716 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:47:30.743 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T13:47:30.743 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:47:30.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:47:30.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:47:30.836 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:47:30.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:47:30.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:47:30.931 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:47:30.932 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:47:30.955 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:47:31.020 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T13:47:31.021 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:47:31.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:47:31.047 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:47:31.110 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:47:31.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:47:31.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:47:31.203 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:47:31.203 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:47:31.227 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:47:31.291 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T13:47:31.292 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:47:31.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:47:31.315 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:47:31.379 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:47:31.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:47:31.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:47:31.466 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:47:31.466 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:47:31.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:47:31.555 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T13:47:31.656 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2}} 2024-04-25T13:47:31.657 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:47:31.657 INFO:tasks.cephadm:Cluster fsid is 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:47:31.657 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T13:47:31.657 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.78', 'mon.b': '172.21.15.112', 'mon.c': '172.21.15.129'} 2024-04-25T13:47:31.658 INFO:tasks.cephadm:First mon is mon.a on smithi078 2024-04-25T13:47:31.658 INFO:tasks.cephadm:First mgr is x 2024-04-25T13:47:31.658 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T13:47:31.658 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-04-25T13:47:31.682 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-04-25T13:47:31.706 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-04-25T13:47:31.730 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T13:47:31.730 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:47:31.866 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T13:47:32.001 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:47:32.003 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T13:47:32.003 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:32.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 13:47 /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:32.592 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:33.108 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 13:47 /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:33.108 DEBUG:teuthology.orchestra.run.smithi129:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:33.515 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 13:47 /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:33.516 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:33.534 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:33.553 DEBUG:teuthology.orchestra.run.smithi129:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:47:33.586 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T13:47:33.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T13:47:33.588 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T13:47:33.596 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:47:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:48:27.427 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T13:48:27.427 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:48:27.427 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T13:48:27.427 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-04-25T13:48:27.427 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T13:48:27.428 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-25T13:48:27.428 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T13:48:55.307 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-04-25T13:48:55.308 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T13:48:55.308 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T13:48:55.308 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T13:48:55.429 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:48:55.430 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:48:55.430 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T13:48:55.430 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-04-25T13:48:55.430 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T13:48:55.430 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T13:48:55.430 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:48:55.453 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-04-25T13:48:55.494 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-04-25T13:48:55.519 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-04-25T13:48:55.552 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-04-25T13:48:55.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-04-25T13:48:55.598 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-04-25T13:48:55.626 INFO:tasks.cephadm:Writing seed config... 2024-04-25T13:48:55.627 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T13:48:55.627 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T13:48:55.627 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T13:48:55.628 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T13:48:55.629 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T13:48:55.630 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T13:48:55.631 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:48:55.631 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T13:48:55.646 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5cff6f4a-030a-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T13:48:55.646 DEBUG:teuthology.orchestra.run.smithi078:mon.a> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T13:48:55.689 DEBUG:teuthology.orchestra.run.smithi078:mgr.x> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T13:48:55.736 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T13:48:55.736 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.78 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:48:55.927 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:48:55.927 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', '5cff6f4a-030a-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.78', '--skip-admin-label'] 2024-04-25T13:48:55.928 INFO:teuthology.orchestra.run.smithi078.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T13:48:55.928 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying podman|docker is present... 2024-04-25T13:48:55.953 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:48:55.953 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying lvm2 is present... 2024-04-25T13:48:55.953 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying time synchronization is in place... 2024-04-25T13:48:55.960 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:48:55.960 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:48:55.966 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:48:55.966 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T13:48:55.972 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-04-25T13:48:55.978 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-04-25T13:48:55.978 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:48:55.979 INFO:teuthology.orchestra.run.smithi078.stdout:Repeating the final host check... 2024-04-25T13:48:56.018 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:48:56.019 INFO:teuthology.orchestra.run.smithi078.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T13:48:56.019 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl is present 2024-04-25T13:48:56.019 INFO:teuthology.orchestra.run.smithi078.stdout:lvcreate is present 2024-04-25T13:48:56.025 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:48:56.025 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:48:56.031 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:48:56.031 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T13:48:56.037 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-04-25T13:48:56.043 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-04-25T13:48:56.043 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:48:56.043 INFO:teuthology.orchestra.run.smithi078.stdout:Host looks OK 2024-04-25T13:48:56.043 INFO:teuthology.orchestra.run.smithi078.stdout:Cluster fsid: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:48:56.043 INFO:teuthology.orchestra.run.smithi078.stdout:Acquiring lock 140395057856464 on /run/cephadm/5cff6f4a-030a-11ef-bc93-c7b262605968.lock 2024-04-25T13:48:56.043 INFO:teuthology.orchestra.run.smithi078.stdout:Lock 140395057856464 acquired on /run/cephadm/5cff6f4a-030a-11ef-bc93-c7b262605968.lock 2024-04-25T13:48:56.044 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 3300 ... 2024-04-25T13:48:56.044 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 6789 ... 2024-04-25T13:48:56.044 INFO:teuthology.orchestra.run.smithi078.stdout:Base mon IP(s) is [172.21.15.78:3300, 172.21.15.78:6789], mon addrv is [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-04-25T13:48:56.047 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.78 metric 100 2024-04-25T13:48:56.047 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.78 metric 100 2024-04-25T13:48:56.050 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T13:48:56.050 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T13:48:56.050 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T13:48:56.052 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T13:48:56.052 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T13:48:56.052 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:48:56.052 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T13:48:56.052 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 fe80::93cc:cc59:36ef:6ab2/64 scope link noprefixroute 2024-04-25T13:48:56.052 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:48:56.053 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-04-25T13:48:56.053 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-04-25T13:48:56.053 INFO:teuthology.orchestra.run.smithi078.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T13:48:56.053 INFO:teuthology.orchestra.run.smithi078.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T13:48:56.054 INFO:teuthology.orchestra.run.smithi078.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:48:56.548 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T13:48:56.548 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T13:48:56.548 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T13:48:56.549 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T13:48:56.549 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T13:48:56.549 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T13:48:56.549 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T13:48:57.703 INFO:teuthology.orchestra.run.smithi078.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T13:48:57.703 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T13:48:57.703 INFO:teuthology.orchestra.run.smithi078.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T13:48:58.667 INFO:teuthology.orchestra.run.smithi078.stdout:stat: stdout 167 167 2024-04-25T13:48:58.667 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial keys... 2024-04-25T13:48:59.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBKXypmhzJxOxAAguxusMo03smYuUoHgdZKjA== 2024-04-25T13:49:00.308 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBLXypmMnWKMBAARkpvFKMIeezTAPZDYGb0Vg== 2024-04-25T13:49:01.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBMXypmAA0vJhAAi4DKVZPzM98QfdyAUA1Meg== 2024-04-25T13:49:01.165 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial monmap... 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool for a [v2:172.21.15.78:3300,v1:172.21.15.78:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: set fsid to 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:49:06.464 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mon... 2024-04-25T13:49:12.854 INFO:teuthology.orchestra.run.smithi078.stdout:create mon.a on 2024-04-25T13:49:13.076 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T13:49:13.215 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T13:49:13.375 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968.target. 2024-04-25T13:49:13.376 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968.target. 2024-04-25T13:49:13.592 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:49:13.592 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T13:49:13.754 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968.target.wants/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@.service. 2024-04-25T13:49:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 podman[28774]: 2024-04-25 13:49:13.956230003 +0000 UTC m=+0.061997971 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:49:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 podman[28774]: 2024-04-25 13:49:14.469998056 +0000 UTC m=+0.575766066 container create 0ec57aa23363394d3360149fdbc1c836514dfeaed894dc17c2846a4639595c45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:49:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 podman[28774]: 2024-04-25 13:49:14.655746307 +0000 UTC m=+0.761514275 container init 0ec57aa23363394d3360149fdbc1c836514dfeaed894dc17c2846a4639595c45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T13:49:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 podman[28774]: 2024-04-25 13:49:14.658710207 +0000 UTC m=+0.764478179 container start 0ec57aa23363394d3360149fdbc1c836514dfeaed894dc17c2846a4639595c45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T13:49:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 ceph-mon[28793]: mkfs 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:14.743 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:49:14.743 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T13:49:14.750 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:49:14.750 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T13:49:14.750 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T13:49:14.750 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:49:14.751 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon to start... 2024-04-25T13:49:14.751 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon... 2024-04-25T13:49:15.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 ceph-mon[28793]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:49:15.025 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 bash[28774]: 0ec57aa23363394d3360149fdbc1c836514dfeaed894dc17c2846a4639595c45 2024-04-25T13:49:15.025 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:14 smithi078 systemd[1]: Started Ceph mon.a for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout id: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout services: 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.498377s) 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T13:49:15.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout data: 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.132+0000 7f7b1c6eb700 1 Processor -- start 2024-04-25T13:49:15.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.132+0000 7f7b1c6eb700 1 -- start start 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.133+0000 7f7b1c6eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b14109de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.133+0000 7f7b1c6eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1410a3d0 con 0x7f7b141099d0 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.133+0000 7f7b1a487700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b14109de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.133+0000 7f7b1a487700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b14109de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38656/0 (socket says 172.21.15.78:38656) 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.133+0000 7f7b1a487700 1 -- 172.21.15.78:0/216917800 learned_addr learned my addr 172.21.15.78:0/216917800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.133+0000 7f7b1a487700 1 -- 172.21.15.78:0/216917800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1410ab00 con 0x7f7b141099d0 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.134+0000 7f7b1a487700 1 --2- 172.21.15.78:0/216917800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b14109de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b040098a0 tx=0x7f7b04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fcb5a85603d091 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.134+0000 7f7b19485700 1 -- 172.21.15.78:0/216917800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b04004160 con 0x7f7b141099d0 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.134+0000 7f7b19485700 1 -- 172.21.15.78:0/216917800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b04004300 con 0x7f7b141099d0 2024-04-25T13:49:15.750 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.134+0000 7f7b19485700 1 -- 172.21.15.78:0/216917800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b040036a0 con 0x7f7b141099d0 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.134+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/216917800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 msgr2=0x7f7b14109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.134+0000 7f7b1c6eb700 1 --2- 172.21.15.78:0/216917800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b14109de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b040098a0 tx=0x7f7b04009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.135+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/216917800 shutdown_connections 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.135+0000 7f7b1c6eb700 1 --2- 172.21.15.78:0/216917800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b14109de0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.135+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/216917800 >> 172.21.15.78:0/216917800 conn(0x7f7b14105090 msgr2=0x7f7b141074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.135+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/216917800 shutdown_connections 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.135+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/216917800 wait complete. 2024-04-25T13:49:15.751 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.135+0000 7f7b1c6eb700 1 Processor -- start 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.135+0000 7f7b1c6eb700 1 -- start start 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1c6eb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b1419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1c6eb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1410b030 con 0x7f7b141099d0 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1a487700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b1419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1a487700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b1419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38672/0 (socket says 172.21.15.78:38672) 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1a487700 1 -- 172.21.15.78:0/3393804286 learned_addr learned my addr 172.21.15.78:0/3393804286 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1a487700 1 -- 172.21.15.78:0/3393804286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1419d9c0 con 0x7f7b141099d0 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1a487700 1 --2- 172.21.15.78:0/3393804286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b1419d480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b0b7fe700 1 -- 172.21.15.78:0/3393804286 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b04003db0 con 0x7f7b141099d0 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b0b7fe700 1 -- 172.21.15.78:0/3393804286 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b04011630 con 0x7f7b141099d0 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1419dc50 con 0x7f7b141099d0 2024-04-25T13:49:15.752 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b0b7fe700 1 -- 172.21.15.78:0/3393804286 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b04011950 con 0x7f7b141099d0 2024-04-25T13:49:15.753 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.136+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b141a0860 con 0x7f7b141099d0 2024-04-25T13:49:15.753 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.137+0000 7f7b0b7fe700 1 -- 172.21.15.78:0/3393804286 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b04011af0 con 0x7f7b141099d0 2024-04-25T13:49:15.753 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.137+0000 7f7b0b7fe700 1 -- 172.21.15.78:0/3393804286 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b0401c450 con 0x7f7b141099d0 2024-04-25T13:49:15.753 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.137+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b14064cf0 con 0x7f7b141099d0 2024-04-25T13:49:15.753 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.140+0000 7f7b0b7fe700 1 -- 172.21.15.78:0/3393804286 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7b04004260 con 0x7f7b141099d0 2024-04-25T13:49:15.753 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.209+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7b1402e810 con 0x7f7b141099d0 2024-04-25T13:49:15.753 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.210+0000 7f7b0b7fe700 1 -- 172.21.15.78:0/3393804286 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7b0401b450 con 0x7f7b141099d0 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.210+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 msgr2=0x7f7b1419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.210+0000 7f7b1c6eb700 1 --2- 172.21.15.78:0/3393804286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b1419d480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b04000c00 tx=0x7f7b04004750 comp rx=0 tx=0).stop 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.210+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 shutdown_connections 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.210+0000 7f7b1c6eb700 1 --2- 172.21.15.78:0/3393804286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b141099d0 0x7f7b1419d480 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.210+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 >> 172.21.15.78:0/3393804286 conn(0x7f7b14105090 msgr2=0x7f7b14193b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.211+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 shutdown_connections 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:15.211+0000 7f7b1c6eb700 1 -- 172.21.15.78:0/3393804286 wait complete. 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:mon is available 2024-04-25T13:49:15.754 INFO:teuthology.orchestra.run.smithi078.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T13:49:16.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:49:16.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: monmap epoch 1 2024-04-25T13:49:16.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:16.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: last_changed 2024-04-25T13:49:01.492952+0000 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: min_mon_release 19 (squid) 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: election_strategy: 1 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: fsmap 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: mgrmap e1: no daemons active 2024-04-25T13:49:16.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:15 smithi078 ceph-mon[28793]: from='client.? 172.21.15.78:0/3393804286' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T13:49:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:49:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:49:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-04-25T13:49:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:49:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:49:16.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.123+0000 7f27c1333700 1 Processor -- start 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.123+0000 7f27c1333700 1 -- start start 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27c1333700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc108690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27c1333700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc07d600 con 0x7f27bc108280 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27baffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc108690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27baffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc108690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38676/0 (socket says 172.21.15.78:38676) 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27baffd700 1 -- 172.21.15.78:0/2374684038 learned_addr learned my addr 172.21.15.78:0/2374684038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:16.772 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27baffd700 1 -- 172.21.15.78:0/2374684038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc108bd0 con 0x7f27bc108280 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27baffd700 1 --2- 172.21.15.78:0/2374684038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc108690 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0098a0 tx=0x7f27ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80931bf671d74689 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27ba7fc700 1 -- 172.21.15.78:0/2374684038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ac004160 con 0x7f27bc108280 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27ba7fc700 1 -- 172.21.15.78:0/2374684038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ac004300 con 0x7f27bc108280 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.124+0000 7f27ba7fc700 1 -- 172.21.15.78:0/2374684038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ac0036a0 con 0x7f27bc108280 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.125+0000 7f27c1333700 1 -- 172.21.15.78:0/2374684038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 msgr2=0x7f27bc108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.125+0000 7f27c1333700 1 --2- 172.21.15.78:0/2374684038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc108690 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0098a0 tx=0x7f27ac009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.125+0000 7f27c1333700 1 -- 172.21.15.78:0/2374684038 shutdown_connections 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.125+0000 7f27c1333700 1 --2- 172.21.15.78:0/2374684038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc108690 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.125+0000 7f27c1333700 1 -- 172.21.15.78:0/2374684038 >> 172.21.15.78:0/2374684038 conn(0x7f27bc103f30 msgr2=0x7f27bc1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.125+0000 7f27c1333700 1 -- 172.21.15.78:0/2374684038 shutdown_connections 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.125+0000 7f27c1333700 1 -- 172.21.15.78:0/2374684038 wait complete. 2024-04-25T13:49:16.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.126+0000 7f27c1333700 1 Processor -- start 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.126+0000 7f27c1333700 1 -- start start 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.126+0000 7f27c1333700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.126+0000 7f27c1333700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc108f80 con 0x7f27bc108280 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.126+0000 7f27baffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.126+0000 7f27baffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38678/0 (socket says 172.21.15.78:38678) 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.126+0000 7f27baffd700 1 -- 172.21.15.78:0/189707219 learned_addr learned my addr 172.21.15.78:0/189707219 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27baffd700 1 -- 172.21.15.78:0/189707219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc19fb70 con 0x7f27bc108280 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27baffd700 1 --2- 172.21.15.78:0/189707219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc19f630 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f27ac000c00 tx=0x7f27ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27b8ff9700 1 -- 172.21.15.78:0/189707219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ac003db0 con 0x7f27bc108280 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27b8ff9700 1 -- 172.21.15.78:0/189707219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ac011630 con 0x7f27bc108280 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc19fe00 con 0x7f27bc108280 2024-04-25T13:49:16.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27b8ff9700 1 -- 172.21.15.78:0/189707219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ac011950 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a2a10 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27b8ff9700 1 -- 172.21.15.78:0/189707219 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f27ac011af0 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.127+0000 7f27b8ff9700 1 -- 172.21.15.78:0/189707219 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f27ac01c450 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.128+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc064cf0 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.130+0000 7f27b8ff9700 1 -- 172.21.15.78:0/189707219 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f27ac004260 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.197+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f27bc1a2be0 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.201+0000 7f27b8ff9700 1 -- 172.21.15.78:0/189707219 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f27ac01b450 con 0x7f27bc108280 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.202+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 msgr2=0x7f27bc19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.202+0000 7f27c1333700 1 --2- 172.21.15.78:0/189707219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc19f630 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f27ac000c00 tx=0x7f27ac004750 comp rx=0 tx=0).stop 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.202+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 shutdown_connections 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.202+0000 7f27c1333700 1 --2- 172.21.15.78:0/189707219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27bc108280 0x7f27bc19f630 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:16.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.202+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 >> 172.21.15.78:0/189707219 conn(0x7f27bc103f30 msgr2=0x7f27bc195900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:16.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.202+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 shutdown_connections 2024-04-25T13:49:16.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:16.202+0000 7f27c1333700 1 -- 172.21.15.78:0/189707219 wait complete. 2024-04-25T13:49:16.776 INFO:teuthology.orchestra.run.smithi078.stdout:Generating new minimal ceph.conf... 2024-04-25T13:49:16.972 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:16 smithi078 ceph-mon[28793]: from='client.? 172.21.15.78:0/189707219' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:49:16.972 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:16 smithi078 ceph-mon[28793]: from='client.? 172.21.15.78:0/189707219' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T13:49:18.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:17 smithi078 ceph-mon[28793]: from='client.? 172.21.15.78:0/1937829779' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:49:18.334 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.163+0000 7fde5afd2700 1 Processor -- start 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.163+0000 7fde5afd2700 1 -- start start 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.163+0000 7fde5afd2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde54108d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.163+0000 7fde5afd2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde54109310 con 0x7fde54108910 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde58d6e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde54108d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde58d6e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde54108d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48206/0 (socket says 172.21.15.78:48206) 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde58d6e700 1 -- 172.21.15.78:0/1250238521 learned_addr learned my addr 172.21.15.78:0/1250238521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde58d6e700 1 -- 172.21.15.78:0/1250238521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde54109a40 con 0x7fde54108910 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde58d6e700 1 --2- 172.21.15.78:0/1250238521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde54108d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fde480098a0 tx=0x7fde48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc8b1b3daf5376e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde537fe700 1 -- 172.21.15.78:0/1250238521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde48004160 con 0x7fde54108910 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde537fe700 1 -- 172.21.15.78:0/1250238521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde48004300 con 0x7fde54108910 2024-04-25T13:49:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.164+0000 7fde537fe700 1 -- 172.21.15.78:0/1250238521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde48011af0 con 0x7fde54108910 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.165+0000 7fde5afd2700 1 -- 172.21.15.78:0/1250238521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 msgr2=0x7fde54108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.165+0000 7fde5afd2700 1 --2- 172.21.15.78:0/1250238521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde54108d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fde480098a0 tx=0x7fde48009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.165+0000 7fde5afd2700 1 -- 172.21.15.78:0/1250238521 shutdown_connections 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.165+0000 7fde5afd2700 1 --2- 172.21.15.78:0/1250238521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde54108d20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.165+0000 7fde5afd2700 1 -- 172.21.15.78:0/1250238521 >> 172.21.15.78:0/1250238521 conn(0x7fde54103f90 msgr2=0x7fde54106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.165+0000 7fde5afd2700 1 -- 172.21.15.78:0/1250238521 shutdown_connections 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.165+0000 7fde5afd2700 1 -- 172.21.15.78:0/1250238521 wait complete. 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde5afd2700 1 Processor -- start 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde5afd2700 1 -- start start 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde5afd2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde5407d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde5afd2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde5410cdf0 con 0x7fde54108910 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde58d6e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde5407d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde58d6e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde5407d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48210/0 (socket says 172.21.15.78:48210) 2024-04-25T13:49:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde58d6e700 1 -- 172.21.15.78:0/1937829779 learned_addr learned my addr 172.21.15.78:0/1937829779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde58d6e700 1 -- 172.21.15.78:0/1937829779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5407d710 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde58d6e700 1 --2- 172.21.15.78:0/1937829779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde5407d1d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde48011890 tx=0x7fde48011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde51ffb700 1 -- 172.21.15.78:0/1937829779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde480039e0 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.166+0000 7fde51ffb700 1 -- 172.21.15.78:0/1937829779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde48003b80 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.167+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde541a0b40 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.167+0000 7fde51ffb700 1 -- 172.21.15.78:0/1937829779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde480246e0 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.167+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde541a1020 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.167+0000 7fde51ffb700 1 -- 172.21.15.78:0/1937829779 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde48022020 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.167+0000 7fde51ffb700 1 -- 172.21.15.78:0/1937829779 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde4802dc40 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.168+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde54064cf0 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.170+0000 7fde51ffb700 1 -- 172.21.15.78:0/1937829779 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fde48043e50 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.237+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fde541a18f0 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.238+0000 7fde51ffb700 1 -- 172.21.15.78:0/1937829779 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fde48032020 con 0x7fde54108910 2024-04-25T13:49:18.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.238+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 msgr2=0x7fde5407d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:18.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.238+0000 7fde5afd2700 1 --2- 172.21.15.78:0/1937829779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde5407d1d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde48011890 tx=0x7fde48011970 comp rx=0 tx=0).stop 2024-04-25T13:49:18.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.238+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 shutdown_connections 2024-04-25T13:49:18.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.238+0000 7fde5afd2700 1 --2- 172.21.15.78:0/1937829779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde54108910 0x7fde5407d1d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:18.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.239+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 >> 172.21.15.78:0/1937829779 conn(0x7fde54103f90 msgr2=0x7fde54104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:18.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.239+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 shutdown_connections 2024-04-25T13:49:18.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:17.239+0000 7fde5afd2700 1 -- 172.21.15.78:0/1937829779 wait complete. 2024-04-25T13:49:18.338 INFO:teuthology.orchestra.run.smithi078.stdout:Restarting the monitor... 2024-04-25T13:49:18.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:18 smithi078 systemd[1]: Stopping Ceph mon.a for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:49:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:18 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a[28789]: 2024-04-25T13:49:18.597+0000 7f10d431a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:49:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:18 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a[28789]: 2024-04-25T13:49:18.597+0000 7f10d431a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T13:49:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:18 smithi078 podman[29036]: 2024-04-25 13:49:18.751402164 +0000 UTC m=+0.247543087 container died 0ec57aa23363394d3360149fdbc1c836514dfeaed894dc17c2846a4639595c45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T13:49:19.356 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 podman[29036]: 2024-04-25 13:49:19.097609157 +0000 UTC m=+0.593750076 container cleanup 0ec57aa23363394d3360149fdbc1c836514dfeaed894dc17c2846a4639595c45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T13:49:19.356 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 bash[29036]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:49:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 podman[29050]: 2024-04-25 13:49:19.356791682 +0000 UTC m=+0.607705788 container remove 0ec57aa23363394d3360149fdbc1c836514dfeaed894dc17c2846a4639595c45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T13:49:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 systemd[1]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T13:49:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 systemd[1]: Stopped Ceph mon.a for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:49:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 systemd[1]: Starting Ceph mon.a for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 podman[29143]: 2024-04-25 13:49:19.628637185 +0000 UTC m=+0.017861148 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 podman[29143]: 2024-04-25 13:49:19.732719764 +0000 UTC m=+0.121943713 container create 70d96a743c37bd08859a2a5ee84654eca3cf862a27547d231188572f64ba1e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 podman[29143]: 2024-04-25 13:49:19.926612413 +0000 UTC m=+0.315836363 container init 70d96a743c37bd08859a2a5ee84654eca3cf862a27547d231188572f64ba1e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 podman[29143]: 2024-04-25 13:49:19.929609658 +0000 UTC m=+0.318833608 container start 70d96a743c37bd08859a2a5ee84654eca3cf862a27547d231188572f64ba1e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: pidfile_write: ignore empty --pid-file 2024-04-25T13:49:20.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: load: jerasure load: lrc 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Git sha 0 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: DB SUMMARY 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: DB Session ID: GFX9N9PGCJXBNZAX8TII 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T13:49:20.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104080 ; 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.env: 0x558b35034500 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.info_log: 0x558b36d41fc0 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:49:20.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.statistics: (nil) 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.use_fsync: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:49:20.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.db_log_dir: 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.wal_dir: 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.write_buffer_manager: 0x558b36d128c0 2024-04-25T13:49:20.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:49:20.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:49:20.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:49:20.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:49:20.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:49:20.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:49:20.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.unordered_write: 0 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.row_cache: None 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.wal_filter: None 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:49:20.008 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.wal_compression: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:49:20.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:49:20.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_open_files: -1 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Compression algorithms supported: 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kZSTD supported: 0 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:49:20.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.merge_operator: 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_filter: None 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:49:20.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558b36e481a0) 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_type: 0 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_index_type: 0 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_shortening: 1 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:49:20.013 INFO:journalctl@ceph.mon.a.smithi078.stdout: checksum: 4 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: no_block_cache: 0 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache: 0x558b36d451f0 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_options: 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: capacity : 536870912 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_shard_bits : 4 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: strict_capacity_limit : 0 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_compressed: (nil) 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: persistent_cache: (nil) 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size: 4096 2024-04-25T13:49:20.014 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size_deviation: 10 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_restart_interval: 16 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_block_restart_interval: 1 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: metadata_block_size: 4096 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: partition_filters: 0 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: use_delta_encoding: 1 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: filter_policy: bloomfilter 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: whole_key_filtering: 1 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: verify_compression: 0 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: format_version: 5 2024-04-25T13:49:20.015 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_index_compression: 1 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_align: 0 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout: max_auto_readahead_size: 262144 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout: prepopulate_block_cache: 0 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression: NoCompression 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:49:20.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.num_levels: 7 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:49:20.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:49:20.018 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:49:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:49:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:49:20.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.ttl: 2592000 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:49:20.022 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:49:20.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 29edab51-bca3-4f87-8e87-8a5b12cab1d4 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714052959968360, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714052959969553, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97254, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89648, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714052959, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "29edab51-bca3-4f87-8e87-8a5b12cab1d4", "db_session_id": "GFX9N9PGCJXBNZAX8TII", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714052959969635, "job": 1, "event": "recovery_finished"} 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558b36d29c00 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: DB pointer 0x558b3bb1c000 2024-04-25T13:49:20.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** DB Stats ** 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T13:49:20.025 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: L0 2/0 98.84 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: Sum 2/0 98.84 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:49:20.026 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative compaction: 0.00 GB write, 29.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval compaction: 0.00 GB write, 29.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:49:20.027 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache BinnedLRUCache@0x558b36d451f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: starting mon.a rank 0 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???) e1 preinit fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???).mds e1 new map 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???).mds e1 print_map 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: e1 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: legacy client fscid: -1 2024-04-25T13:49:20.028 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout: No filesystems configured 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:19 smithi078 ceph-mon[29163]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 bash[29143]: 70d96a743c37bd08859a2a5ee84654eca3cf862a27547d231188572f64ba1e5c 2024-04-25T13:49:20.029 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 systemd[1]: Started Ceph mon.a for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:49:20.029 INFO:teuthology.orchestra.run.smithi078.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T13:49:20.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:49:20.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: monmap epoch 1 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: last_changed 2024-04-25T13:49:01.492952+0000 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: min_mon_release 19 (squid) 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: election_strategy: 1 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: fsmap 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:49:20.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:20 smithi078 ceph-mon[29163]: mgrmap e1: no daemons active 2024-04-25T13:49:21.088 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.440+0000 7f681b436700 1 Processor -- start 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.440+0000 7f681b436700 1 -- start start 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.441+0000 7f681b436700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f6814108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.441+0000 7f681b436700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68141092d0 con 0x7f68141088d0 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.441+0000 7f68191d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f6814108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.441+0000 7f68191d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f6814108ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48226/0 (socket says 172.21.15.78:48226) 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.441+0000 7f68191d2700 1 -- 172.21.15.78:0/3696655162 learned_addr learned my addr 172.21.15.78:0/3696655162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.441+0000 7f68191d2700 1 -- 172.21.15.78:0/3696655162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6814109a00 con 0x7f68141088d0 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.441+0000 7f68191d2700 1 --2- 172.21.15.78:0/3696655162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f6814108ce0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f68100098a0 tx=0x7f6810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb42d6a2b52af5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f680bfff700 1 -- 172.21.15.78:0/3696655162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6810004160 con 0x7f68141088d0 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f680bfff700 1 -- 172.21.15.78:0/3696655162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6810004300 con 0x7f68141088d0 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f680bfff700 1 -- 172.21.15.78:0/3696655162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6810011af0 con 0x7f68141088d0 2024-04-25T13:49:21.089 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f681b436700 1 -- 172.21.15.78:0/3696655162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 msgr2=0x7f6814108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f681b436700 1 --2- 172.21.15.78:0/3696655162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f6814108ce0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f68100098a0 tx=0x7f6810009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f681b436700 1 -- 172.21.15.78:0/3696655162 shutdown_connections 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f681b436700 1 --2- 172.21.15.78:0/3696655162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f6814108ce0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f681b436700 1 -- 172.21.15.78:0/3696655162 >> 172.21.15.78:0/3696655162 conn(0x7f6814103fb0 msgr2=0x7f6814106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.442+0000 7f681b436700 1 -- 172.21.15.78:0/3696655162 shutdown_connections 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.443+0000 7f681b436700 1 -- 172.21.15.78:0/3696655162 wait complete. 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.443+0000 7f681b436700 1 Processor -- start 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.443+0000 7f681b436700 1 -- start start 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.443+0000 7f681b436700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f681407c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.443+0000 7f681b436700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f681410cdb0 con 0x7f68141088d0 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f68191d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f681407c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f68191d2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f681407c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48236/0 (socket says 172.21.15.78:48236) 2024-04-25T13:49:21.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f68191d2700 1 -- 172.21.15.78:0/1368500274 learned_addr learned my addr 172.21.15.78:0/1368500274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f68191d2700 1 -- 172.21.15.78:0/1368500274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681407ce40 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f68191d2700 1 --2- 172.21.15.78:0/1368500274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f681407c900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6810011890 tx=0x7f6810011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f680a7fc700 1 -- 172.21.15.78:0/1368500274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68100039e0 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f680a7fc700 1 -- 172.21.15.78:0/1368500274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6810003b80 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68141a0b20 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f680a7fc700 1 -- 172.21.15.78:0/1368500274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68100246e0 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.444+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68141a0fa0 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.445+0000 7f680a7fc700 1 -- 172.21.15.78:0/1368500274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6810022020 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.445+0000 7f680a7fc700 1 -- 172.21.15.78:0/1368500274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f681002dcb0 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.445+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6814064cf0 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.448+0000 7f680a7fc700 1 -- 172.21.15.78:0/1368500274 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f681001aa70 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.515+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f68141a13c0 con 0x7f68141088d0 2024-04-25T13:49:21.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.517+0000 7f680a7fc700 1 -- 172.21.15.78:0/1368500274 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f681001a430 con 0x7f68141088d0 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.518+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 msgr2=0x7f681407c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.518+0000 7f681b436700 1 --2- 172.21.15.78:0/1368500274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f681407c900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6810011890 tx=0x7f6810011970 comp rx=0 tx=0).stop 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.518+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 shutdown_connections 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.518+0000 7f681b436700 1 --2- 172.21.15.78:0/1368500274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68141088d0 0x7f681407c900 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.518+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 >> 172.21.15.78:0/1368500274 conn(0x7f6814103fb0 msgr2=0x7f6814105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.518+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 shutdown_connections 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:20.518+0000 7f681b436700 1 -- 172.21.15.78:0/1368500274 wait complete. 2024-04-25T13:49:21.092 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T13:49:21.106 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:49:21.106 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mgr... 2024-04-25T13:49:21.107 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T13:49:21.107 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T13:49:21.348 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:49:21.348 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T13:49:21.500 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968.target.wants/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@.service. 2024-04-25T13:49:21.823 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:21 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1368500274' entity='client.admin' 2024-04-25T13:49:21.976 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:49:21.976 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T13:49:21.984 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:49:21.985 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T13:49:21.985 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T13:49:21.985 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:49:21.995 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:49:21.995 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T13:49:22.003 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:49:22.003 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T13:49:22.004 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T13:49:22.004 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T13:49:22.004 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr to start... 2024-04-25T13:49:22.004 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr... 2024-04-25T13:49:22.082 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:21 smithi078 systemd[1]: Started Ceph mgr.x for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:49:22.384 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:22 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:22.081+0000 7fc39a8bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:49:22.384 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:22 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:22.275+0000 7fc39a8bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:49:22.384 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:22 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:22.383+0000 7fc39a8bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:49:22.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:22 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2894971629' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:49:22.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:22 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:22.704+0000 7fc39a8bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:49:22.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:22 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:22.793+0000 7fc39a8bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:49:23.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:23.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:49:23.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:49:23.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:49:23.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:49:23.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:49:23.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:23.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:49:23.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:23.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:49:23.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:49:23.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:49:23.066 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:49:23.067 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:49:14.716776+0000", 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.391+0000 7f5f1b79f700 1 Processor -- start 2024-04-25T13:49:23.068 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.392+0000 7f5f1b79f700 1 -- start start 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.392+0000 7f5f1b79f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1410c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.392+0000 7f5f1b79f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1407cd70 con 0x7f5f1410a4b0 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.392+0000 7f5f1953b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1410c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.392+0000 7f5f1953b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1410c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48264/0 (socket says 172.21.15.78:48264) 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.392+0000 7f5f1953b700 1 -- 172.21.15.78:0/737186688 learned_addr learned my addr 172.21.15.78:0/737186688 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.393+0000 7f5f1953b700 1 -- 172.21.15.78:0/737186688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1410ce10 con 0x7f5f1410a4b0 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.393+0000 7f5f1953b700 1 --2- 172.21.15.78:0/737186688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1410c8d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b5ca1bf8fa7f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.393+0000 7f5f0bfff700 1 -- 172.21.15.78:0/737186688 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10004160 con 0x7f5f1410a4b0 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.393+0000 7f5f0bfff700 1 -- 172.21.15.78:0/737186688 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5f10004300 con 0x7f5f1410a4b0 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.393+0000 7f5f0bfff700 1 -- 172.21.15.78:0/737186688 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10011af0 con 0x7f5f1410a4b0 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.394+0000 7f5f1b79f700 1 -- 172.21.15.78:0/737186688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 msgr2=0x7f5f1410c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.394+0000 7f5f1b79f700 1 --2- 172.21.15.78:0/737186688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1410c8d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.394+0000 7f5f1b79f700 1 -- 172.21.15.78:0/737186688 shutdown_connections 2024-04-25T13:49:23.069 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.394+0000 7f5f1b79f700 1 --2- 172.21.15.78:0/737186688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1410c8d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.394+0000 7f5f1b79f700 1 -- 172.21.15.78:0/737186688 >> 172.21.15.78:0/737186688 conn(0x7f5f14103fb0 msgr2=0x7f5f14106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.394+0000 7f5f1b79f700 1 -- 172.21.15.78:0/737186688 shutdown_connections 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.394+0000 7f5f1b79f700 1 -- 172.21.15.78:0/737186688 wait complete. 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1b79f700 1 Processor -- start 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1b79f700 1 -- start start 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1b79f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1b79f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1410dd10 con 0x7f5f1410a4b0 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1953b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1953b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1419dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48270/0 (socket says 172.21.15.78:48270) 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1953b700 1 -- 172.21.15.78:0/2894971629 learned_addr learned my addr 172.21.15.78:0/2894971629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1953b700 1 -- 172.21.15.78:0/2894971629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1419e270 con 0x7f5f1410a4b0 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f1953b700 1 --2- 172.21.15.78:0/2894971629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1419dd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009fb0 tx=0x7f5f10011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:23.070 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.395+0000 7f5f0a7fc700 1 -- 172.21.15.78:0/2894971629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f100036a0 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.396+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1419e500 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.396+0000 7f5f0a7fc700 1 -- 172.21.15.78:0/2894971629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5f1001a430 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.396+0000 7f5f0a7fc700 1 -- 172.21.15.78:0/2894971629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f10019550 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.396+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f141a1100 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.396+0000 7f5f0a7fc700 1 -- 172.21.15.78:0/2894971629 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f10019be0 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.396+0000 7f5f0a7fc700 1 -- 172.21.15.78:0/2894971629 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f10024730 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.396+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f14064cf0 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.399+0000 7f5f0a7fc700 1 -- 172.21.15.78:0/2894971629 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5f10043e50 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.471+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5f141a12d0 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.472+0000 7f5f0a7fc700 1 -- 172.21.15.78:0/2894971629 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5f10032020 con 0x7f5f1410a4b0 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.472+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 msgr2=0x7f5f1419dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.472+0000 7f5f1b79f700 1 --2- 172.21.15.78:0/2894971629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1419dd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009fb0 tx=0x7f5f10011940 comp rx=0 tx=0).stop 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.472+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 shutdown_connections 2024-04-25T13:49:23.071 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.472+0000 7f5f1b79f700 1 --2- 172.21.15.78:0/2894971629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f1410a4b0 0x7f5f1419dd30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:23.072 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.472+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 >> 172.21.15.78:0/2894971629 conn(0x7f5f14103fb0 msgr2=0x7f5f141058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:23.072 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.473+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 shutdown_connections 2024-04-25T13:49:23.072 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:22.473+0000 7f5f1b79f700 1 -- 172.21.15.78:0/2894971629 wait complete. 2024-04-25T13:49:23.072 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (1/15)... 2024-04-25T13:49:24.321 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:23 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:23.985+0000 7fc39a8bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:49:24.321 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:24.153+0000 7fc39a8bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:49:24.321 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:24.321+0000 7fc39a8bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:49:24.925 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:24.621+0000 7fc39a8bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:49:24.926 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:24.717+0000 7fc39a8bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:49:25.183 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:24.925+0000 7fc39a8bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:49:25.183 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:25 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:25.128+0000 7fc39a8bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:49:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:25 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/251184031' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:49:26.181 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:25 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:25.757+0000 7fc39a8bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:26.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:26.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:49:26.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:26.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:49:26.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:49:26.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:49:26.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:49:26.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:49:26.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:49:26.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:49:26.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:26.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:49:14.716776+0000", 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.490+0000 7f0f1c895700 1 Processor -- start 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.490+0000 7f0f1c895700 1 -- start start 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.490+0000 7f0f1c895700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1810e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.490+0000 7f0f1c895700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810eec0 con 0x7f0f1807d490 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.490+0000 7f0f1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1810e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.490+0000 7f0f1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1810e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48286/0 (socket says 172.21.15.78:48286) 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.490+0000 7f0f1659c700 1 -- 172.21.15.78:0/3104448002 learned_addr learned my addr 172.21.15.78:0/3104448002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.491+0000 7f0f1659c700 1 -- 172.21.15.78:0/3104448002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1810f040 con 0x7f0f1807d490 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.491+0000 7f0f1659c700 1 --2- 172.21.15.78:0/3104448002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1810e980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f080098a0 tx=0x7f0f08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d66ef722c63569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:26.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.491+0000 7f0f1559a700 1 -- 172.21.15.78:0/3104448002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08004160 con 0x7f0f1807d490 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.491+0000 7f0f1559a700 1 -- 172.21.15.78:0/3104448002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f08004300 con 0x7f0f1807d490 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.491+0000 7f0f1559a700 1 -- 172.21.15.78:0/3104448002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08011af0 con 0x7f0f1807d490 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.492+0000 7f0f1c895700 1 -- 172.21.15.78:0/3104448002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 msgr2=0x7f0f1810e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.492+0000 7f0f1c895700 1 --2- 172.21.15.78:0/3104448002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1810e980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f080098a0 tx=0x7f0f08009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.492+0000 7f0f1c895700 1 -- 172.21.15.78:0/3104448002 shutdown_connections 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.492+0000 7f0f1c895700 1 --2- 172.21.15.78:0/3104448002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1810e980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.492+0000 7f0f1c895700 1 -- 172.21.15.78:0/3104448002 >> 172.21.15.78:0/3104448002 conn(0x7f0f18103f50 msgr2=0x7f0f181063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.492+0000 7f0f1c895700 1 -- 172.21.15.78:0/3104448002 shutdown_connections 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.492+0000 7f0f1c895700 1 -- 172.21.15.78:0/3104448002 wait complete. 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.493+0000 7f0f1c895700 1 Processor -- start 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.493+0000 7f0f1c895700 1 -- start start 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.493+0000 7f0f1c895700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:26.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.493+0000 7f0f1c895700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810ff50 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.493+0000 7f0f1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.493+0000 7f0f1659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1819fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48296/0 (socket says 172.21.15.78:48296) 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.493+0000 7f0f1659c700 1 -- 172.21.15.78:0/251184031 learned_addr learned my addr 172.21.15.78:0/251184031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.494+0000 7f0f1659c700 1 -- 172.21.15.78:0/251184031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a0430 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.494+0000 7f0f1659c700 1 --2- 172.21.15.78:0/251184031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1819fef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f080117b0 tx=0x7f0f08011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.494+0000 7f0f077fe700 1 -- 172.21.15.78:0/251184031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08003af0 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.494+0000 7f0f077fe700 1 -- 172.21.15.78:0/251184031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f08020070 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.494+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a06c0 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.494+0000 7f0f077fe700 1 -- 172.21.15.78:0/251184031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f08019630 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.494+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a32d0 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.495+0000 7f0f077fe700 1 -- 172.21.15.78:0/251184031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f08019c30 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.495+0000 7f0f077fe700 1 -- 172.21.15.78:0/251184031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0f0800b720 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.495+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18064cf0 con 0x7f0f1807d490 2024-04-25T13:49:26.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.497+0000 7f0f077fe700 1 -- 172.21.15.78:0/251184031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0f0801a9c0 con 0x7f0f1807d490 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.570+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0f181a3960 con 0x7f0f1807d490 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.571+0000 7f0f077fe700 1 -- 172.21.15.78:0/251184031 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0f0801a430 con 0x7f0f1807d490 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.572+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 msgr2=0x7f0f1819fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.572+0000 7f0f1c895700 1 --2- 172.21.15.78:0/251184031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1819fef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f080117b0 tx=0x7f0f08011b70 comp rx=0 tx=0).stop 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.572+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 shutdown_connections 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.572+0000 7f0f1c895700 1 --2- 172.21.15.78:0/251184031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f1807d490 0x7f0f1819fef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.572+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 >> 172.21.15.78:0/251184031 conn(0x7f0f18103f50 msgr2=0x7f0f18104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.572+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 shutdown_connections 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:25.572+0000 7f0f1c895700 1 -- 172.21.15.78:0/251184031 wait complete. 2024-04-25T13:49:26.328 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (2/15)... 2024-04-25T13:49:26.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:26 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:26.181+0000 7fc39a8bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:49:26.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:26 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:26.279+0000 7fc39a8bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:49:26.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:26 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:26.491+0000 7fc39a8bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:49:27.339 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:27.010+0000 7fc39a8bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:49:27.339 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:27.098+0000 7fc39a8bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:49:27.680 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:27.339+0000 7fc39a8bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:49:27.680 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:27.471+0000 7fc39a8bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:49:27.680 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:27.565+0000 7fc39a8bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:49:27.681 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:27.680+0000 7fc39a8bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:49:28.440 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:27.982+0000 7fc39a8bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:49:28.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:28.590+0000 7fc39a8bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:28.861+0000 7fc39a8bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/240157159' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: Activating manager daemon x 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: mgrmap e2: x(active, starting, since 0.00352628s) 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:49:29.147 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: Manager daemon x is now available 2024-04-25T13:49:29.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:49:29.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:49:29.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' 2024-04-25T13:49:29.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' 2024-04-25T13:49:29.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:28 smithi078 ceph-mon[29163]: from='mgr.14100 172.21.15.78:0/2116022105' entity='mgr.x' 2024-04-25T13:49:29.452 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:29.452 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:29.453 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:49:29.454 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:49:29.462 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:49:29.462 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:49:29.462 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:49:29.462 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:29.462 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:49:29.462 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:49:29.462 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:29.463 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:49:14.716776+0000", 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:29.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.720+0000 7f584cb92700 1 Processor -- start 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.720+0000 7f584cb92700 1 -- start start 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.720+0000 7f584cb92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f5848108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.720+0000 7f584cb92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58481092d0 con 0x7f58481088d0 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.720+0000 7f584659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f5848108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.720+0000 7f584659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f5848108ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54644/0 (socket says 172.21.15.78:54644) 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.720+0000 7f584659c700 1 -- 172.21.15.78:0/694975202 learned_addr learned my addr 172.21.15.78:0/694975202 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.721+0000 7f584659c700 1 -- 172.21.15.78:0/694975202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5848109a00 con 0x7f58481088d0 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.721+0000 7f584659c700 1 --2- 172.21.15.78:0/694975202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f5848108ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f583800bb40 tx=0x7f583800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaaf978731585cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.721+0000 7f584559a700 1 -- 172.21.15.78:0/694975202 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f583800dbe0 con 0x7f58481088d0 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.721+0000 7f584559a700 1 -- 172.21.15.78:0/694975202 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f58380044c0 con 0x7f58481088d0 2024-04-25T13:49:29.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.721+0000 7f584559a700 1 -- 172.21.15.78:0/694975202 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5838010470 con 0x7f58481088d0 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.722+0000 7f584cb92700 1 -- 172.21.15.78:0/694975202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 msgr2=0x7f5848108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.722+0000 7f584cb92700 1 --2- 172.21.15.78:0/694975202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f5848108ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f583800bb40 tx=0x7f583800be50 comp rx=0 tx=0).stop 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.722+0000 7f584cb92700 1 -- 172.21.15.78:0/694975202 shutdown_connections 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.722+0000 7f584cb92700 1 --2- 172.21.15.78:0/694975202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f5848108ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.722+0000 7f584cb92700 1 -- 172.21.15.78:0/694975202 >> 172.21.15.78:0/694975202 conn(0x7f5848103fb0 msgr2=0x7f5848106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.722+0000 7f584cb92700 1 -- 172.21.15.78:0/694975202 shutdown_connections 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.722+0000 7f584cb92700 1 -- 172.21.15.78:0/694975202 wait complete. 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.723+0000 7f584cb92700 1 Processor -- start 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.723+0000 7f584cb92700 1 -- start start 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.723+0000 7f584cb92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f584819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.723+0000 7f584cb92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f584807c0e0 con 0x7f58481088d0 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f584659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f584819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:29.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f584659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f584819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54658/0 (socket says 172.21.15.78:54658) 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f584659c700 1 -- 172.21.15.78:0/240157159 learned_addr learned my addr 172.21.15.78:0/240157159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f584659c700 1 -- 172.21.15.78:0/240157159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584819bff0 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f584659c700 1 --2- 172.21.15.78:0/240157159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f584819bab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5838003b90 tx=0x7f5838003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f58377fe700 1 -- 172.21.15.78:0/240157159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5838027070 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584819c280 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f58377fe700 1 -- 172.21.15.78:0/240157159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5838010c00 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f58377fe700 1 -- 172.21.15.78:0/240157159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5838022be0 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.724+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584819ee90 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.725+0000 7f58377fe700 1 -- 172.21.15.78:0/240157159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f583802a070 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.725+0000 7f58377fe700 1 -- 172.21.15.78:0/240157159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f583801f070 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.725+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848064cf0 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.728+0000 7f58377fe700 1 -- 172.21.15.78:0/240157159 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5838015070 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.858+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f584819f530 con 0x7f58481088d0 2024-04-25T13:49:29.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.859+0000 7f58377fe700 1 -- 172.21.15.78:0/240157159 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5838010db0 con 0x7f58481088d0 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.860+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 msgr2=0x7f584819bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.860+0000 7f584cb92700 1 --2- 172.21.15.78:0/240157159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f584819bab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5838003b90 tx=0x7f5838003dc0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.860+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 shutdown_connections 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.860+0000 7f584cb92700 1 --2- 172.21.15.78:0/240157159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481088d0 0x7f584819bab0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.860+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 >> 172.21.15.78:0/240157159 conn(0x7f5848103fb0 msgr2=0x7f5848191d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.860+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 shutdown_connections 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:28.860+0000 7f584cb92700 1 -- 172.21.15.78:0/240157159 wait complete. 2024-04-25T13:49:29.468 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (3/15)... 2024-04-25T13:49:31.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:30 smithi078 ceph-mon[29163]: mgrmap e3: x(active, since 1.00771s) 2024-04-25T13:49:32.193 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:31 smithi078 ceph-mon[29163]: mgrmap e4: x(active, since 2s) 2024-04-25T13:49:32.674 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:32.674 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:32.675 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:49:32.676 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:49:32.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:49:32.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:49:32.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:49:14.716776+0000", 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.907+0000 7f73ef59a700 1 Processor -- start 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.907+0000 7f73ef59a700 1 -- start start 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73ef59a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8108cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73ef59a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73e81092e0 con 0x7f73e81088e0 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73ed336700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8108cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73ed336700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8108cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54720/0 (socket says 172.21.15.78:54720) 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73ed336700 1 -- 172.21.15.78:0/783651209 learned_addr learned my addr 172.21.15.78:0/783651209 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73ed336700 1 -- 172.21.15.78:0/783651209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e8109a10 con 0x7f73e81088e0 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73ed336700 1 --2- 172.21.15.78:0/783651209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8108cf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098a0 tx=0x7f73e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861d00e11bd7b926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73e7fff700 1 -- 172.21.15.78:0/783651209 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e0004160 con 0x7f73e81088e0 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73e7fff700 1 -- 172.21.15.78:0/783651209 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f73e0004300 con 0x7f73e81088e0 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.908+0000 7f73e7fff700 1 -- 172.21.15.78:0/783651209 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e0011af0 con 0x7f73e81088e0 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.909+0000 7f73ef59a700 1 -- 172.21.15.78:0/783651209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 msgr2=0x7f73e8108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.909+0000 7f73ef59a700 1 --2- 172.21.15.78:0/783651209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8108cf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f73e00098a0 tx=0x7f73e0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.909+0000 7f73ef59a700 1 -- 172.21.15.78:0/783651209 shutdown_connections 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.909+0000 7f73ef59a700 1 --2- 172.21.15.78:0/783651209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8108cf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.909+0000 7f73ef59a700 1 -- 172.21.15.78:0/783651209 >> 172.21.15.78:0/783651209 conn(0x7f73e8103f20 msgr2=0x7f73e81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:32.689 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.909+0000 7f73ef59a700 1 -- 172.21.15.78:0/783651209 shutdown_connections 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.909+0000 7f73ef59a700 1 -- 172.21.15.78:0/783651209 wait complete. 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.910+0000 7f73ef59a700 1 Processor -- start 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.910+0000 7f73ef59a700 1 -- start start 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.910+0000 7f73ef59a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8105f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.910+0000 7f73ef59a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73e810cdc0 con 0x7f73e81088e0 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.910+0000 7f73ed336700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8105f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73ed336700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8105f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54724/0 (socket says 172.21.15.78:54724) 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73ed336700 1 -- 172.21.15.78:0/1045707954 learned_addr learned my addr 172.21.15.78:0/1045707954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73ed336700 1 -- 172.21.15.78:0/1045707954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e81064d0 con 0x7f73e81088e0 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73ed336700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8105f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009fb0 tx=0x7f73e0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73e67fc700 1 -- 172.21.15.78:0/1045707954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e00036a0 con 0x7f73e81088e0 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73e67fc700 1 -- 172.21.15.78:0/1045707954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f73e0003cd0 con 0x7f73e81088e0 2024-04-25T13:49:32.690 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e8104720 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73e67fc700 1 -- 172.21.15.78:0/1045707954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73e001a430 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.911+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e8104c00 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.912+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e8064cf0 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.912+0000 7f73e67fc700 1 -- 172.21.15.78:0/1045707954 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f73e001a6a0 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.912+0000 7f73e67fc700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f73d003d5e0 0x7f73d003faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.912+0000 7f73e67fc700 1 -- 172.21.15.78:0/1045707954 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f73e0052450 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.913+0000 7f73ecb35700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f73d003d5e0 0x7f73d003faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.913+0000 7f73ecb35700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f73d003d5e0 0x7f73d003faf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f73dc006fd0 tx=0x7f73dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:31.918+0000 7f73e67fc700 1 -- 172.21.15.78:0/1045707954 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73e001aa40 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.192+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f73e8196f30 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.193+0000 7f73e67fc700 1 -- 172.21.15.78:0/1045707954 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f73e0020240 con 0x7f73e81088e0 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f73d003d5e0 msgr2=0x7f73d003faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f73d003d5e0 0x7f73d003faf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f73dc006fd0 tx=0x7f73dc006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:32.691 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 msgr2=0x7f73e8105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8105f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009fb0 tx=0x7f73e0011890 comp rx=0 tx=0).stop 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 shutdown_connections 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f73d003d5e0 0x7f73d003faf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 --2- 172.21.15.78:0/1045707954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e81088e0 0x7f73e8105f90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 >> 172.21.15.78:0/1045707954 conn(0x7f73e8103f20 msgr2=0x7f73e81942a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 shutdown_connections 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:32.195+0000 7f73ef59a700 1 -- 172.21.15.78:0/1045707954 wait complete. 2024-04-25T13:49:32.692 INFO:teuthology.orchestra.run.smithi078.stdout:mgr is available 2024-04-25T13:49:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:32 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1045707954' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:49:33.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:33.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:49:33.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:33.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:49:33.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:49:33.986 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.098+0000 7fe41cfa6700 1 Processor -- start 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.098+0000 7fe41cfa6700 1 -- start start 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.098+0000 7fe41cfa6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe41810c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.098+0000 7fe41cfa6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe41807cd70 con 0x7fe41810a480 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe41810c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe41810c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54738/0 (socket says 172.21.15.78:54738) 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41659c700 1 -- 172.21.15.78:0/2592778401 learned_addr learned my addr 172.21.15.78:0/2592778401 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41659c700 1 -- 172.21.15.78:0/2592778401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41810cde0 con 0x7fe41810a480 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41659c700 1 --2- 172.21.15.78:0/2592778401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe41810c8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe410009e80 tx=0x7fe41000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce0d9d268760d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:33.987 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41559a700 1 -- 172.21.15.78:0/2592778401 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe410017070 con 0x7fe41810a480 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41559a700 1 -- 172.21.15.78:0/2592778401 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe41000b8c0 con 0x7fe41810a480 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.099+0000 7fe41559a700 1 -- 172.21.15.78:0/2592778401 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe410004490 con 0x7fe41810a480 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.100+0000 7fe41cfa6700 1 -- 172.21.15.78:0/2592778401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 msgr2=0x7fe41810c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.100+0000 7fe41cfa6700 1 --2- 172.21.15.78:0/2592778401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe41810c8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe410009e80 tx=0x7fe41000b330 comp rx=0 tx=0).stop 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.100+0000 7fe41cfa6700 1 -- 172.21.15.78:0/2592778401 shutdown_connections 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.100+0000 7fe41cfa6700 1 --2- 172.21.15.78:0/2592778401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe41810c8a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.100+0000 7fe41cfa6700 1 -- 172.21.15.78:0/2592778401 >> 172.21.15.78:0/2592778401 conn(0x7fe418103f80 msgr2=0x7fe4181063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.100+0000 7fe41cfa6700 1 -- 172.21.15.78:0/2592778401 shutdown_connections 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.100+0000 7fe41cfa6700 1 -- 172.21.15.78:0/2592778401 wait complete. 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41cfa6700 1 Processor -- start 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41cfa6700 1 -- start start 2024-04-25T13:49:33.988 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41cfa6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe418109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41cfa6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe41810dce0 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe418109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe418109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54754/0 (socket says 172.21.15.78:54754) 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41659c700 1 -- 172.21.15.78:0/1099242856 learned_addr learned my addr 172.21.15.78:0/1099242856 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.101+0000 7fe41659c700 1 -- 172.21.15.78:0/1099242856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41810a140 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.102+0000 7fe41659c700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe418109c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe410004220 tx=0x7fe41000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.102+0000 7fe4077fe700 1 -- 172.21.15.78:0/1099242856 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe410020020 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.102+0000 7fe4077fe700 1 -- 172.21.15.78:0/1099242856 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe410012430 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.102+0000 7fe4077fe700 1 -- 172.21.15.78:0/1099242856 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe410017040 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.102+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe418108390 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.102+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4181088d0 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.103+0000 7fe4077fe700 1 -- 172.21.15.78:0/1099242856 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7fe410023020 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.103+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe418064cf0 con 0x7fe41810a480 2024-04-25T13:49:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.103+0000 7fe4077fe700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fe40003d4d0 0x7fe40003f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.103+0000 7fe4077fe700 1 -- 172.21.15.78:0/1099242856 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe41000f050 con 0x7fe41810a480 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.103+0000 7fe415d9b700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fe40003d4d0 0x7fe40003f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.104+0000 7fe415d9b700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fe40003d4d0 0x7fe40003f9e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe40c006fd0 tx=0x7fe40c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.108+0000 7fe4077fe700 1 -- 172.21.15.78:0/1099242856 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe41002a020 con 0x7fe41810a480 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.303+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe418108fc0 con 0x7fe41810a480 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.304+0000 7fe4077fe700 1 -- 172.21.15.78:0/1099242856 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fe4100102f0 con 0x7fe41810a480 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fe40003d4d0 msgr2=0x7fe40003f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fe40003d4d0 0x7fe40003f9e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe40c006fd0 tx=0x7fe40c006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 msgr2=0x7fe418109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe418109c00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe410004220 tx=0x7fe41000bd50 comp rx=0 tx=0).stop 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 shutdown_connections 2024-04-25T13:49:33.990 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fe40003d4d0 0x7fe40003f9e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.991 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 --2- 172.21.15.78:0/1099242856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe41810a480 0x7fe418109c00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:33.991 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 >> 172.21.15.78:0/1099242856 conn(0x7fe418103f80 msgr2=0x7fe4181058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:33.991 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 shutdown_connections 2024-04-25T13:49:33.991 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:33.306+0000 7fe41cfa6700 1 -- 172.21.15.78:0/1099242856 wait complete. 2024-04-25T13:49:33.991 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling cephadm module... 2024-04-25T13:49:34.093 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:33 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1099242856' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:49:34.889 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:34 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2362267188' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T13:49:35.181 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:34 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: ignoring --setuser ceph since I am not root 2024-04-25T13:49:35.182 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:34 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: ignoring --setgroup ceph since I am not root 2024-04-25T13:49:35.182 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:35.071+0000 7f66265b4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:49:35.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:35.261+0000 7f66265b4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:49:35.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:35.363+0000 7f66265b4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.374+0000 7f405233d700 1 Processor -- start 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.374+0000 7f405233d700 1 -- start start 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.374+0000 7f405233d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c108d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.374+0000 7f405233d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f404c1092f0 con 0x7f404c1088f0 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c108d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c108d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54760/0 (socket says 172.21.15.78:54760) 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404bfff700 1 -- 172.21.15.78:0/50127079 learned_addr learned my addr 172.21.15.78:0/50127079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404bfff700 1 -- 172.21.15.78:0/50127079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c109a20 con 0x7f404c1088f0 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404bfff700 1 --2- 172.21.15.78:0/50127079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c108d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f403c0098a0 tx=0x7f403c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1803c87a60aefd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:35.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404affd700 1 -- 172.21.15.78:0/50127079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c004160 con 0x7f404c1088f0 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404affd700 1 -- 172.21.15.78:0/50127079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f403c004300 con 0x7f404c1088f0 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.375+0000 7f404affd700 1 -- 172.21.15.78:0/50127079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c011af0 con 0x7f404c1088f0 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.376+0000 7f405233d700 1 -- 172.21.15.78:0/50127079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 msgr2=0x7f404c108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.376+0000 7f405233d700 1 --2- 172.21.15.78:0/50127079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c108d00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f403c0098a0 tx=0x7f403c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.376+0000 7f405233d700 1 -- 172.21.15.78:0/50127079 shutdown_connections 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.376+0000 7f405233d700 1 --2- 172.21.15.78:0/50127079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c108d00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.376+0000 7f405233d700 1 -- 172.21.15.78:0/50127079 >> 172.21.15.78:0/50127079 conn(0x7f404c103f50 msgr2=0x7f404c1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.376+0000 7f405233d700 1 -- 172.21.15.78:0/50127079 shutdown_connections 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.376+0000 7f405233d700 1 -- 172.21.15.78:0/50127079 wait complete. 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f405233d700 1 Processor -- start 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f405233d700 1 -- start start 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f405233d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c07d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f405233d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f404c10cdd0 con 0x7f404c1088f0 2024-04-25T13:49:35.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f404bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c07d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f404bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c07d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54774/0 (socket says 172.21.15.78:54774) 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f404bfff700 1 -- 172.21.15.78:0/2362267188 learned_addr learned my addr 172.21.15.78:0/2362267188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.377+0000 7f404bfff700 1 -- 172.21.15.78:0/2362267188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c07d6d0 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.378+0000 7f404bfff700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c07d190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f403c009fb0 tx=0x7f403c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.378+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c0036a0 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.378+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f403c003cd0 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.378+0000 7f405233d700 1 -- 172.21.15.78:0/2362267188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c1a0b60 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.378+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c01a430 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.378+0000 7f405233d700 1 -- 172.21.15.78:0/2362267188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1a0fe0 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.379+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f403c01a6f0 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.379+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c064cf0 con 0x7f404c1088f0 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.379+0000 7f40497fa700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f403403d4d0 0x7f403403f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:35.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.379+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f403c0524e0 con 0x7f404c1088f0 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.379+0000 7f404b7fe700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f403403d4d0 0x7f403403f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.380+0000 7f404b7fe700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f403403d4d0 0x7f403403f9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.384+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f403c017e90 con 0x7f404c1088f0 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.639+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f404c0642a0 con 0x7f404c1088f0 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.874+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f403c024560 con 0x7f404c1088f0 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.876+0000 7f40497fa700 1 -- 172.21.15.78:0/2362267188 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f403c02d3f0 con 0x7f404c1088f0 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f403403d4d0 msgr2=0x7f403403f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f403403d4d0 0x7f403403f9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 msgr2=0x7f404c07d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c07d190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f403c009fb0 tx=0x7f403c011890 comp rx=0 tx=0).stop 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 shutdown_connections 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7f403403d4d0 0x7f403403f9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:35.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 --2- 172.21.15.78:0/2362267188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f404c1088f0 0x7f404c07d190 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:35.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 >> 172.21.15.78:0/2362267188 conn(0x7f404c103f50 msgr2=0x7f404c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:35.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 shutdown_connections 2024-04-25T13:49:35.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:34.877+0000 7f4032ffd700 1 -- 172.21.15.78:0/2362267188 wait complete. 2024-04-25T13:49:35.784 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:35.695+0000 7f66265b4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:49:36.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:35 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2362267188' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T13:49:36.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:35 smithi078 ceph-mon[29163]: mgrmap e5: x(active, since 6s) 2024-04-25T13:49:36.101 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:35.785+0000 7f66265b4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:49:36.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:49:36.773 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.895+0000 7fa37a535700 1 Processor -- start 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.895+0000 7fa37a535700 1 -- start start 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.895+0000 7fa37a535700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37410c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.895+0000 7fa37a535700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa37407cd70 con 0x7fa37410a4b0 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.895+0000 7fa373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37410c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.896+0000 7fa373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37410c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54818/0 (socket says 172.21.15.78:54818) 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.896+0000 7fa373fff700 1 -- 172.21.15.78:0/98688651 learned_addr learned my addr 172.21.15.78:0/98688651 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:36.774 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.896+0000 7fa373fff700 1 -- 172.21.15.78:0/98688651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37410ce10 con 0x7fa37410a4b0 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.896+0000 7fa373fff700 1 --2- 172.21.15.78:0/98688651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37410c8d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0098a0 tx=0x7fa35c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d090ee5c75b396 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.896+0000 7fa372ffd700 1 -- 172.21.15.78:0/98688651 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa35c004160 con 0x7fa37410a4b0 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.896+0000 7fa372ffd700 1 -- 172.21.15.78:0/98688651 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa35c004300 con 0x7fa37410a4b0 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.896+0000 7fa372ffd700 1 -- 172.21.15.78:0/98688651 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa35c011af0 con 0x7fa37410a4b0 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.897+0000 7fa37a535700 1 -- 172.21.15.78:0/98688651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 msgr2=0x7fa37410c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.897+0000 7fa37a535700 1 --2- 172.21.15.78:0/98688651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37410c8d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0098a0 tx=0x7fa35c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.897+0000 7fa37a535700 1 -- 172.21.15.78:0/98688651 shutdown_connections 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.897+0000 7fa37a535700 1 --2- 172.21.15.78:0/98688651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37410c8d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.897+0000 7fa37a535700 1 -- 172.21.15.78:0/98688651 >> 172.21.15.78:0/98688651 conn(0x7fa374103fb0 msgr2=0x7fa374106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.897+0000 7fa37a535700 1 -- 172.21.15.78:0/98688651 shutdown_connections 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.897+0000 7fa37a535700 1 -- 172.21.15.78:0/98688651 wait complete. 2024-04-25T13:49:36.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa37a535700 1 Processor -- start 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa37a535700 1 -- start start 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa37a535700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa37a535700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa37410dd10 con 0x7fa37410a4b0 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37419dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54822/0 (socket says 172.21.15.78:54822) 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa373fff700 1 -- 172.21.15.78:0/815603484 learned_addr learned my addr 172.21.15.78:0/815603484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.898+0000 7fa373fff700 1 -- 172.21.15.78:0/815603484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37419e270 con 0x7fa37410a4b0 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.899+0000 7fa373fff700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37419dd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0117b0 tx=0x7fa35c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.899+0000 7fa3717fa700 1 -- 172.21.15.78:0/815603484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa35c003af0 con 0x7fa37410a4b0 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.899+0000 7fa3717fa700 1 -- 172.21.15.78:0/815603484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa35c020070 con 0x7fa37410a4b0 2024-04-25T13:49:36.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.899+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37419e500 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.899+0000 7fa3717fa700 1 -- 172.21.15.78:0/815603484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa35c019630 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.899+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3741a1100 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.900+0000 7fa3717fa700 1 -- 172.21.15.78:0/815603484 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fa35c0198a0 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.900+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa374064cf0 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.900+0000 7fa3717fa700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 0x7fa36003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.900+0000 7fa3717fa700 1 -- 172.21.15.78:0/815603484 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa35c0528d0 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.900+0000 7fa3737fe700 1 -- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 msgr2=0x7fa36003fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3823505283 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.900+0000 7fa3737fe700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 0x7fa36003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:35.910+0000 7fa3717fa700 1 -- 172.21.15.78:0/815603484 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa35c024e70 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.101+0000 7fa3737fe700 1 -- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 msgr2=0x7fa36003fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3823505283 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.101+0000 7fa3737fe700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 0x7fa36003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.172+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa3741a1780 con 0x7fa37410a4b0 2024-04-25T13:49:36.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.173+0000 7fa3717fa700 1 -- 172.21.15.78:0/815603484 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fa35c0522f0 con 0x7fa37410a4b0 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.176+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 msgr2=0x7fa36003fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.176+0000 7fa37a535700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 0x7fa36003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.176+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 msgr2=0x7fa37419dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.176+0000 7fa37a535700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37419dd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0117b0 tx=0x7fa35c011b70 comp rx=0 tx=0).stop 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.178+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 shutdown_connections 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.178+0000 7fa37a535700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7fa36003d5c0 0x7fa36003fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.178+0000 7fa37a535700 1 --2- 172.21.15.78:0/815603484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa37410a4b0 0x7fa37419dd30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.178+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 >> 172.21.15.78:0/815603484 conn(0x7fa374103fb0 msgr2=0x7fa3741058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.178+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 shutdown_connections 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:36.179+0000 7fa37a535700 1 -- 172.21.15.78:0/815603484 wait complete. 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-04-25T13:49:36.778 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 5... 2024-04-25T13:49:37.025 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:36 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/815603484' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:49:37.025 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:36 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:36.994+0000 7f66265b4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:49:37.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:37.173+0000 7f66265b4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:49:37.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:37.348+0000 7f66265b4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:49:37.937 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:37.640+0000 7f66265b4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:49:37.937 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:37.734+0000 7f66265b4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:49:37.937 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:37.937+0000 7f66265b4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:49:38.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:38 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:38.131+0000 7f66265b4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:49:38.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:38 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:38.721+0000 7f66265b4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:49:39.444 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:39.145+0000 7f66265b4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:49:39.444 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:39.241+0000 7f66265b4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:49:39.444 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:39.444+0000 7f66265b4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:49:40.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:39.961+0000 7f66265b4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:49:40.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:40.054+0000 7f66265b4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:49:40.525 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:40.297+0000 7f66265b4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:49:40.526 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:40.430+0000 7f66265b4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:49:40.526 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:40.525+0000 7f66265b4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:49:40.940 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:40.642+0000 7f66265b4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:49:40.940 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:40.940+0000 7f66265b4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:49:41.806 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:41.536+0000 7f66265b4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:49:41.806 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:49:41.753+0000 7f66265b4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:49:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: Active manager daemon x restarted 2024-04-25T13:49:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: Activating manager daemon x 2024-04-25T13:49:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T13:49:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: mgrmap e6: x(active, starting, since 0.00376176s) 2024-04-25T13:49:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:49:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: Manager daemon x is now available 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:42.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:41 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:43.060 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:42 smithi078 ceph-mon[29163]: Found migration_current of "None". Setting to last migration. 2024-04-25T13:49:43.060 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:42 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:49:43.061 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:42 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:49:43.061 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:42 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:43.061 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:42 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:43.061 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:42 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:43.061 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:42 smithi078 ceph-mon[29163]: mgrmap e7: x(active, since 1.00648s) 2024-04-25T13:49:43.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:49:43.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T13:49:43.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:49:43.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:49:43.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.237+0000 7ff4e3246700 1 Processor -- start 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.237+0000 7ff4e3246700 1 -- start start 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.237+0000 7ff4e3246700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc10c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.237+0000 7ff4e3246700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc10cb70 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.237+0000 7ff4e0fe2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc10c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.237+0000 7ff4e0fe2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc10c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48368/0 (socket says 172.21.15.78:48368) 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.237+0000 7ff4e0fe2700 1 -- 172.21.15.78:0/3740213002 learned_addr learned my addr 172.21.15.78:0/3740213002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.238+0000 7ff4e0fe2700 1 -- 172.21.15.78:0/3740213002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc10ccf0 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.238+0000 7ff4e0fe2700 1 --2- 172.21.15.78:0/3740213002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc10c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00098a0 tx=0x7ff4d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac3812de79e9076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.238+0000 7ff4db7fe700 1 -- 172.21.15.78:0/3740213002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4d0004160 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.238+0000 7ff4db7fe700 1 -- 172.21.15.78:0/3740213002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4d0004300 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.238+0000 7ff4db7fe700 1 -- 172.21.15.78:0/3740213002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4d0011af0 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.239+0000 7ff4e3246700 1 -- 172.21.15.78:0/3740213002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 msgr2=0x7ff4dc10c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:43.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.239+0000 7ff4e3246700 1 --2- 172.21.15.78:0/3740213002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc10c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00098a0 tx=0x7ff4d0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.239+0000 7ff4e3246700 1 -- 172.21.15.78:0/3740213002 shutdown_connections 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.239+0000 7ff4e3246700 1 --2- 172.21.15.78:0/3740213002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc10c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.239+0000 7ff4e3246700 1 -- 172.21.15.78:0/3740213002 >> 172.21.15.78:0/3740213002 conn(0x7ff4dc103f10 msgr2=0x7ff4dc1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.239+0000 7ff4e3246700 1 -- 172.21.15.78:0/3740213002 shutdown_connections 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.239+0000 7ff4e3246700 1 -- 172.21.15.78:0/3740213002 wait complete. 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.240+0000 7ff4e3246700 1 Processor -- start 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.240+0000 7ff4e3246700 1 -- start start 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.240+0000 7ff4e3246700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.240+0000 7ff4e3246700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc10dd10 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.240+0000 7ff4e0fe2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.240+0000 7ff4e0fe2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48374/0 (socket says 172.21.15.78:48374) 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.240+0000 7ff4e0fe2700 1 -- 172.21.15.78:0/258090220 learned_addr learned my addr 172.21.15.78:0/258090220 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:43.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.241+0000 7ff4e0fe2700 1 -- 172.21.15.78:0/258090220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc19e280 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.241+0000 7ff4e0fe2700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc19dd40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009fb0 tx=0x7ff4d0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.241+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4d00036a0 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.241+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc19e510 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.241+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4d001a430 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.241+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4d0019550 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.241+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc1a1110 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.242+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7ff4d0019800 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.243+0000 7ff4d9ffb700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.243+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 --> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4cc040190 con 0x7ff4cc03d570 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.243+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4d0052660 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.243+0000 7ff4dbfff700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 msgr2=0x7ff4cc03fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3823505283 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.243+0000 7ff4dbfff700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.444+0000 7ff4dbfff700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 msgr2=0x7ff4cc03fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3823505283 2024-04-25T13:49:43.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.444+0000 7ff4dbfff700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.845+0000 7ff4dbfff700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 msgr2=0x7ff4cc03fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3823505283 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:37.845+0000 7ff4dbfff700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:38.646+0000 7ff4dbfff700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 msgr2=0x7ff4cc03fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3823505283 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:38.646+0000 7ff4dbfff700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:40.247+0000 7ff4dbfff700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 msgr2=0x7ff4cc03fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3823505283 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:40.247+0000 7ff4dbfff700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:41.757+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7ff4d000fd80 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:41.757+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 msgr2=0x7ff4cc03fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:41.757+0000 7ff4d9ffb700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.758+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7ff4d00547d0 con 0x7ff4dc07d5f0 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.759+0000 7ff4d9ffb700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7ff4cc040eb0 0x7ff4cc0432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.759+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4d002ee50 con 0x7ff4cc040eb0 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.759+0000 7ff4dbfff700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7ff4cc040eb0 0x7ff4cc0432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.760+0000 7ff4dbfff700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7ff4cc040eb0 0x7ff4cc0432d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8003a10 tx=0x7ff4c80092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:43.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.760+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff4d002ee50 con 0x7ff4cc040eb0 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.768+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff4dc064cf0 con 0x7ff4cc040eb0 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4d9ffb700 1 -- 172.21.15.78:0/258090220 <== mgr.14120 v2:172.21.15.78:6800/1759244565 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff4dc064cf0 con 0x7ff4cc040eb0 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7ff4cc040eb0 msgr2=0x7ff4cc0432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7ff4cc040eb0 0x7ff4cc0432d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8003a10 tx=0x7ff4c80092b0 comp rx=0 tx=0).stop 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 msgr2=0x7ff4dc19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc19dd40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009fb0 tx=0x7ff4d0011940 comp rx=0 tx=0).stop 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 shutdown_connections 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7ff4cc040eb0 0x7ff4cc0432d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4dc07d5f0 0x7ff4dc19dd40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 --2- 172.21.15.78:0/258090220 >> [v2:172.21.15.78:6800/3823505283,v1:172.21.15.78:6801/3823505283] conn(0x7ff4cc03d570 0x7ff4cc03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 >> 172.21.15.78:0/258090220 conn(0x7ff4dc103f10 msgr2=0x7ff4dc104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 shutdown_connections 2024-04-25T13:49:43.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:42.769+0000 7ff4e3246700 1 -- 172.21.15.78:0/258090220 wait complete. 2024-04-25T13:49:43.285 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 5 is available 2024-04-25T13:49:43.285 INFO:teuthology.orchestra.run.smithi078.stdout:Setting orchestrator backend to cephadm... 2024-04-25T13:49:43.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:43 smithi078 ceph-mon[29163]: [25/Apr/2024:13:49:42] ENGINE Bus STARTING 2024-04-25T13:49:43.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:43 smithi078 ceph-mon[29163]: [25/Apr/2024:13:49:42] ENGINE Serving on http://172.21.15.78:8765 2024-04-25T13:49:43.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:43 smithi078 ceph-mon[29163]: [25/Apr/2024:13:49:42] ENGINE Serving on https://172.21.15.78:7150 2024-04-25T13:49:43.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:43 smithi078 ceph-mon[29163]: [25/Apr/2024:13:49:42] ENGINE Bus STARTED 2024-04-25T13:49:43.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:43 smithi078 ceph-mon[29163]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:49:43.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:43 smithi078 ceph-mon[29163]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27bd78e700 1 Processor -- start 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27bd78e700 1 -- start start 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27bd78e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b810c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27bd78e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810cae0 con 0x7f27b807cc60 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27b6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b810c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27b6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b810c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48424/0 (socket says 172.21.15.78:48424) 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27b6ffd700 1 -- 172.21.15.78:0/3383239498 learned_addr learned my addr 172.21.15.78:0/3383239498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:44.442 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.684+0000 7f27b6ffd700 1 -- 172.21.15.78:0/3383239498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b810cc60 con 0x7f27b807cc60 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27b6ffd700 1 --2- 172.21.15.78:0/3383239498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b810c4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f27a80098a0 tx=0x7f27a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7750b45ec9cdd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27b5ffb700 1 -- 172.21.15.78:0/3383239498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a8004160 con 0x7f27b807cc60 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27b5ffb700 1 -- 172.21.15.78:0/3383239498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f27a8004300 con 0x7f27b807cc60 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27b5ffb700 1 -- 172.21.15.78:0/3383239498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a8011af0 con 0x7f27b807cc60 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27bd78e700 1 -- 172.21.15.78:0/3383239498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 msgr2=0x7f27b810c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27bd78e700 1 --2- 172.21.15.78:0/3383239498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b810c4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f27a80098a0 tx=0x7f27a8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27bd78e700 1 -- 172.21.15.78:0/3383239498 shutdown_connections 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27bd78e700 1 --2- 172.21.15.78:0/3383239498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b810c4d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27bd78e700 1 -- 172.21.15.78:0/3383239498 >> 172.21.15.78:0/3383239498 conn(0x7f27b8103f80 msgr2=0x7f27b81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27bd78e700 1 -- 172.21.15.78:0/3383239498 shutdown_connections 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.685+0000 7f27bd78e700 1 -- 172.21.15.78:0/3383239498 wait complete. 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.686+0000 7f27bd78e700 1 Processor -- start 2024-04-25T13:49:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.686+0000 7f27bd78e700 1 -- start start 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27bd78e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27bd78e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810dce0 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27b6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27b6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48438/0 (socket says 172.21.15.78:48438) 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27b6ffd700 1 -- 172.21.15.78:0/3993390422 learned_addr learned my addr 172.21.15.78:0/3993390422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27b6ffd700 1 -- 172.21.15.78:0/3993390422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b819e240 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27b6ffd700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b819dd00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27a80117b0 tx=0x7f27a8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a8003af0 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f27a8020070 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b819e4d0 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a8019630 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.687+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b81a10e0 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.688+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b8064cf0 con 0x7f27b807cc60 2024-04-25T13:49:44.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.688+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7f27a80198f0 con 0x7f27b807cc60 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.688+0000 7f279ffff700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f27a003d450 0x7f27a003f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.688+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27a8052780 con 0x7f27b807cc60 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.689+0000 7f27b67fc700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f27a003d450 0x7f27a003f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.689+0000 7f27b67fc700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f27a003d450 0x7f27a003f960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27ac006fd0 tx=0x7f27ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.694+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27a8016260 con 0x7f27b807cc60 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.813+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f27a802dcf0 con 0x7f27b807cc60 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.907+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f27b8106010 con 0x7f27a003d450 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.913+0000 7f279ffff700 1 -- 172.21.15.78:0/3993390422 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f27b8106010 con 0x7f27a003d450 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f27a003d450 msgr2=0x7f27a003f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f27a003d450 0x7f27a003f960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27ac006fd0 tx=0x7f27ac006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 msgr2=0x7f27b819dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b819dd00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27a80117b0 tx=0x7f27a8011b70 comp rx=0 tx=0).stop 2024-04-25T13:49:44.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 shutdown_connections 2024-04-25T13:49:44.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f27a003d450 0x7f27a003f960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:44.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 --2- 172.21.15.78:0/3993390422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b807cc60 0x7f27b819dd00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:44.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 >> 172.21.15.78:0/3993390422 conn(0x7f27b8103f80 msgr2=0x7f27b81058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:44.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 shutdown_connections 2024-04-25T13:49:44.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:43.915+0000 7f27bd78e700 1 -- 172.21.15.78:0/3993390422 wait complete. 2024-04-25T13:49:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:44 smithi078 ceph-mon[29163]: mgrmap e8: x(active, since 2s) 2024-04-25T13:49:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:44 smithi078 ceph-mon[29163]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:49:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:44 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:44 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.830+0000 7f6535df4700 1 Processor -- start 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.830+0000 7f6535df4700 1 -- start start 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.830+0000 7f6535df4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f6530109db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.830+0000 7f6535df4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f653010a3a0 con 0x7f65301099a0 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.830+0000 7f652f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f6530109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.830+0000 7f652f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f6530109db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48440/0 (socket says 172.21.15.78:48440) 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.830+0000 7f652f7fe700 1 -- 172.21.15.78:0/3083862257 learned_addr learned my addr 172.21.15.78:0/3083862257 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.831+0000 7f652f7fe700 1 -- 172.21.15.78:0/3083862257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653010aad0 con 0x7f65301099a0 2024-04-25T13:49:45.632 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.831+0000 7f652f7fe700 1 --2- 172.21.15.78:0/3083862257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f6530109db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f65200098a0 tx=0x7f6520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d09e1cbb3743d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.831+0000 7f652effd700 1 -- 172.21.15.78:0/3083862257 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6520004160 con 0x7f65301099a0 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.831+0000 7f652effd700 1 -- 172.21.15.78:0/3083862257 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6520004300 con 0x7f65301099a0 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.831+0000 7f652effd700 1 -- 172.21.15.78:0/3083862257 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6520011af0 con 0x7f65301099a0 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 -- 172.21.15.78:0/3083862257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 msgr2=0x7f6530109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 --2- 172.21.15.78:0/3083862257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f6530109db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f65200098a0 tx=0x7f6520009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 -- 172.21.15.78:0/3083862257 shutdown_connections 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 --2- 172.21.15.78:0/3083862257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f6530109db0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 -- 172.21.15.78:0/3083862257 >> 172.21.15.78:0/3083862257 conn(0x7f6530105060 msgr2=0x7f65301074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 -- 172.21.15.78:0/3083862257 shutdown_connections 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 -- 172.21.15.78:0/3083862257 wait complete. 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.832+0000 7f6535df4700 1 Processor -- start 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f6535df4700 1 -- start start 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f6535df4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f653019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:45.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f6535df4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f653010de80 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f652f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f653019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f652f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f653019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48456/0 (socket says 172.21.15.78:48456) 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f652f7fe700 1 -- 172.21.15.78:0/2182595244 learned_addr learned my addr 172.21.15.78:0/2182595244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f652f7fe700 1 -- 172.21.15.78:0/2182595244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653019e2a0 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.833+0000 7f652f7fe700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f653019dd60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6520011890 tx=0x7f6520011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.834+0000 7f652d7fa700 1 -- 172.21.15.78:0/2182595244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65200039e0 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.834+0000 7f652d7fa700 1 -- 172.21.15.78:0/2182595244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6520003b80 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.834+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653019e530 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.834+0000 7f652d7fa700 1 -- 172.21.15.78:0/2182595244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65200246e0 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.834+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301a1130 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.835+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6530064cf0 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.835+0000 7f652d7fa700 1 -- 172.21.15.78:0/2182595244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f6520022020 con 0x7f65301099a0 2024-04-25T13:49:45.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.835+0000 7f652d7fa700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f651c03d570 0x7f651c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.835+0000 7f652d7fa700 1 -- 172.21.15.78:0/2182595244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f65200525d0 con 0x7f65301099a0 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.835+0000 7f6526ffd700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f651c03d570 0x7f651c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.835+0000 7f6526ffd700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f651c03d570 0x7f651c03fa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6518006fd0 tx=0x7f6518006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:44.840+0000 7f652d7fa700 1 -- 172.21.15.78:0/2182595244 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65200094f0 con 0x7f65301099a0 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.039+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6530107050 con 0x7f651c03d570 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.039+0000 7f652d7fa700 1 -- 172.21.15.78:0/2182595244 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6530107050 con 0x7f651c03d570 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f651c03d570 msgr2=0x7f651c03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f651c03d570 0x7f651c03fa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6518006fd0 tx=0x7f6518006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 msgr2=0x7f653019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f653019dd60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6520011890 tx=0x7f6520011970 comp rx=0 tx=0).stop 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 shutdown_connections 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f651c03d570 0x7f651c03fa80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:45.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 --2- 172.21.15.78:0/2182595244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65301099a0 0x7f653019dd60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:45.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 >> 172.21.15.78:0/2182595244 conn(0x7f6530105060 msgr2=0x7f6530106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:45.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.041+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 shutdown_connections 2024-04-25T13:49:45.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:45.042+0000 7f6535df4700 1 -- 172.21.15.78:0/2182595244 wait complete. 2024-04-25T13:49:45.636 INFO:teuthology.orchestra.run.smithi078.stdout:Generating ssh key... 2024-04-25T13:49:46.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-mon[29163]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: Generating public/private rsa key pair. 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: Your identification has been saved in /tmp/tmpw5dugpn4/key. 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: Your public key has been saved in /tmp/tmpw5dugpn4/key.pub. 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: The key fingerprint is: 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: SHA256:NoQmrdx2w8icJJHKHp+ntMKaIUjRToswYVce5D1XojE ceph-5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: The key's randomart image is: 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: +---[RSA 3072]----+ 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |.. .+= E . . | 2024-04-25T13:49:46.540 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |..o +oo.+ o | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |oo +oo=+.. | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |..O..X =o | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: | + =o.B S | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |o . +..o o | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |+. . + | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |.oo o | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: |o. . | 2024-04-25T13:49:46.541 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:49:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: +----[SHA256]-----+ 2024-04-25T13:49:47.129 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd7d9e700 1 Processor -- start 2024-04-25T13:49:47.129 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd7d9e700 1 -- start start 2024-04-25T13:49:47.129 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd7d9e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd010c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd7d9e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fafd010cb10 con 0x7fafd007d650 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd5b3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd010c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd5b3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd010c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48472/0 (socket says 172.21.15.78:48472) 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd5b3a700 1 -- 172.21.15.78:0/533047081 learned_addr learned my addr 172.21.15.78:0/533047081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd5b3a700 1 -- 172.21.15.78:0/533047081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd010cc90 con 0x7fafd007d650 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd5b3a700 1 --2- 172.21.15.78:0/533047081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd010c540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0098a0 tx=0x7fafcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77f42a3a9e5f8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.077+0000 7fafd4b38700 1 -- 172.21.15.78:0/533047081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc004160 con 0x7fafd007d650 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd4b38700 1 -- 172.21.15.78:0/533047081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafcc004300 con 0x7fafd007d650 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd4b38700 1 -- 172.21.15.78:0/533047081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc011af0 con 0x7fafd007d650 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd7d9e700 1 -- 172.21.15.78:0/533047081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 msgr2=0x7fafd010c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd7d9e700 1 --2- 172.21.15.78:0/533047081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd010c540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0098a0 tx=0x7fafcc009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:47.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd7d9e700 1 -- 172.21.15.78:0/533047081 shutdown_connections 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd7d9e700 1 --2- 172.21.15.78:0/533047081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd010c540 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd7d9e700 1 -- 172.21.15.78:0/533047081 >> 172.21.15.78:0/533047081 conn(0x7fafd0103f30 msgr2=0x7fafd01063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd7d9e700 1 -- 172.21.15.78:0/533047081 shutdown_connections 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.078+0000 7fafd7d9e700 1 -- 172.21.15.78:0/533047081 wait complete. 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.079+0000 7fafd7d9e700 1 Processor -- start 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.079+0000 7fafd7d9e700 1 -- start start 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.079+0000 7fafd7d9e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.079+0000 7fafd7d9e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fafd010dd10 con 0x7fafd007d650 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.079+0000 7fafd5b3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.079+0000 7fafd5b3a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd019dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48484/0 (socket says 172.21.15.78:48484) 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.079+0000 7fafd5b3a700 1 -- 172.21.15.78:0/155624771 learned_addr learned my addr 172.21.15.78:0/155624771 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.080+0000 7fafd5b3a700 1 -- 172.21.15.78:0/155624771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd019e230 con 0x7fafd007d650 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.080+0000 7fafd5b3a700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd019dcf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009fb0 tx=0x7fafcc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.080+0000 7fafc6ffd700 1 -- 172.21.15.78:0/155624771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc0036a0 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.080+0000 7fafc6ffd700 1 -- 172.21.15.78:0/155624771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafcc01a430 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.080+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd019e4c0 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.080+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd01a10d0 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.080+0000 7fafc6ffd700 1 -- 172.21.15.78:0/155624771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc019550 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.081+0000 7fafc6ffd700 1 -- 172.21.15.78:0/155624771 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fafcc019850 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.081+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd0064cf0 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.081+0000 7fafc6ffd700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fafbc03d520 0x7fafbc03fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.081+0000 7fafc6ffd700 1 -- 172.21.15.78:0/155624771 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fafcc0528c0 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.081+0000 7fafd5339700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fafbc03d520 0x7fafbc03fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.082+0000 7fafd5339700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fafbc03d520 0x7fafbc03fa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fafc0006fd0 tx=0x7fafc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.088+0000 7fafc6ffd700 1 -- 172.21.15.78:0/155624771 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafcc017e90 con 0x7fafd007d650 2024-04-25T13:49:47.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.286+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fafd0104f80 con 0x7fafbc03d520 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.539+0000 7fafc6ffd700 1 -- 172.21.15.78:0/155624771 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fafd0104f80 con 0x7fafbc03d520 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.541+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fafbc03d520 msgr2=0x7fafbc03fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.541+0000 7fafd7d9e700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fafbc03d520 0x7fafbc03fa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fafc0006fd0 tx=0x7fafc0006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.541+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 msgr2=0x7fafd019dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.541+0000 7fafd7d9e700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd019dcf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009fb0 tx=0x7fafcc011940 comp rx=0 tx=0).stop 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.542+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 shutdown_connections 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.542+0000 7fafd7d9e700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fafbc03d520 0x7fafbc03fa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.542+0000 7fafd7d9e700 1 --2- 172.21.15.78:0/155624771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafd007d650 0x7fafd019dcf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.542+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 >> 172.21.15.78:0/155624771 conn(0x7fafd0103f30 msgr2=0x7fafd0104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.542+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 shutdown_connections 2024-04-25T13:49:47.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:46.542+0000 7fafd7d9e700 1 -- 172.21.15.78:0/155624771 wait complete. 2024-04-25T13:49:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:47 smithi078 ceph-mon[29163]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:49:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:47 smithi078 ceph-mon[29163]: Generating ssh key... 2024-04-25T13:49:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:47 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:47 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:48.318 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd3Yzu82ZzBYEC1PE6cL9OWdwfgg+3B2TDbvElxpOot/xYXueUys3GSwO2zVMt7LbbC4vHUH0Vk8WEj7aLjoL9HnT2Ej0LBl6zrwAh4fq38EGzsFomo3U22AA1aMh82p6Sf0W8T2VMYVP4f0r7l/J357ydN7gtTzcHy0ODBzHYDRXsu3ftU1Whzn05XHFZWn8br/fZsWlUXQ6etrGZFd5dMf38YMLWtAVoxAa+K3851kMLuWV34whw8UiQ9nHLwXaFGK4hl4X8RsCtrgBqijjdKn6Vccpc6mbCupT36sxjwlJpVHN+qRKsBdhlAAdFbzjVs91UUw3ot4TeRlEZscxxteOyZzY4xLNbzV2amf+DDLycs3sj8JchGHGuVvvzlYO/7jKJDFuuSQ+HkMHsq++NQLKBnAGSWEaF31wd79pEeoXG+ni3JyfPM7d2+UAa6sbjGs0Q/hEL92saNbASRplUxAQX0+D/FCmWJGmYov5uEexgzexoNZgxITW2T2tdt5M= ceph-5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:49:48.318 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.541+0000 7fc58b3c8700 1 Processor -- start 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.541+0000 7fc58b3c8700 1 -- start start 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc58b3c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58410c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc58b3c8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410cb10 con 0x7fc58407cc40 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc589164700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58410c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc589164700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58410c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55538/0 (socket says 172.21.15.78:55538) 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc589164700 1 -- 172.21.15.78:0/1597717386 learned_addr learned my addr 172.21.15.78:0/1597717386 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc589164700 1 -- 172.21.15.78:0/1597717386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58410cc90 con 0x7fc58407cc40 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc589164700 1 --2- 172.21.15.78:0/1597717386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58410c520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc5740098a0 tx=0x7fc574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff1c72774ce30ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc57bfff700 1 -- 172.21.15.78:0/1597717386 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc574004160 con 0x7fc58407cc40 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc57bfff700 1 -- 172.21.15.78:0/1597717386 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc574004300 con 0x7fc58407cc40 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.542+0000 7fc57bfff700 1 -- 172.21.15.78:0/1597717386 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc574011af0 con 0x7fc58407cc40 2024-04-25T13:49:48.319 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.543+0000 7fc58b3c8700 1 -- 172.21.15.78:0/1597717386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 msgr2=0x7fc58410c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.543+0000 7fc58b3c8700 1 --2- 172.21.15.78:0/1597717386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58410c520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc5740098a0 tx=0x7fc574009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.543+0000 7fc58b3c8700 1 -- 172.21.15.78:0/1597717386 shutdown_connections 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.543+0000 7fc58b3c8700 1 --2- 172.21.15.78:0/1597717386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58410c520 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.543+0000 7fc58b3c8700 1 -- 172.21.15.78:0/1597717386 >> 172.21.15.78:0/1597717386 conn(0x7fc584103fb0 msgr2=0x7fc584106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.543+0000 7fc58b3c8700 1 -- 172.21.15.78:0/1597717386 shutdown_connections 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.543+0000 7fc58b3c8700 1 -- 172.21.15.78:0/1597717386 wait complete. 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc58b3c8700 1 Processor -- start 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc58b3c8700 1 -- start start 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc58b3c8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc58b3c8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410dd10 con 0x7fc58407cc40 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc589164700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc589164700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58419dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55546/0 (socket says 172.21.15.78:55546) 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc589164700 1 -- 172.21.15.78:0/2009827836 learned_addr learned my addr 172.21.15.78:0/2009827836 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:48.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc589164700 1 -- 172.21.15.78:0/2009827836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58419e230 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.544+0000 7fc589164700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58419dcf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5740117b0 tx=0x7fc574011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.545+0000 7fc57a7fc700 1 -- 172.21.15.78:0/2009827836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc574003af0 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.545+0000 7fc57a7fc700 1 -- 172.21.15.78:0/2009827836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc574020070 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.545+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58419e4c0 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.545+0000 7fc57a7fc700 1 -- 172.21.15.78:0/2009827836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc574019630 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.545+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a10d0 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.546+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc584064cf0 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.546+0000 7fc57a7fc700 1 -- 172.21.15.78:0/2009827836 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fc5740198a0 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.546+0000 7fc57a7fc700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fc57003d5c0 0x7fc57003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.546+0000 7fc57a7fc700 1 -- 172.21.15.78:0/2009827836 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc5740528f0 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.546+0000 7fc588963700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fc57003d5c0 0x7fc57003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.547+0000 7fc588963700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fc57003d5c0 0x7fc57003fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc580006fd0 tx=0x7fc580006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.551+0000 7fc57a7fc700 1 -- 172.21.15.78:0/2009827836 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc574016260 con 0x7fc58407cc40 2024-04-25T13:49:48.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.752+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc5841a12a0 con 0x7fc57003d5c0 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.753+0000 7fc57a7fc700 1 -- 172.21.15.78:0/2009827836 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc5841a12a0 con 0x7fc57003d5c0 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fc57003d5c0 msgr2=0x7fc57003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fc57003d5c0 0x7fc57003fad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc580006fd0 tx=0x7fc580006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 msgr2=0x7fc58419dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58419dcf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc5740117b0 tx=0x7fc574011b70 comp rx=0 tx=0).stop 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 shutdown_connections 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fc57003d5c0 0x7fc57003fad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 --2- 172.21.15.78:0/2009827836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc58407cc40 0x7fc58419dcf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 >> 172.21.15.78:0/2009827836 conn(0x7fc584103fb0 msgr2=0x7fc584105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 shutdown_connections 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:47.755+0000 7fc58b3c8700 1 -- 172.21.15.78:0/2009827836 wait complete. 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T13:49:48.322 INFO:teuthology.orchestra.run.smithi078.stdout:Adding host smithi078... 2024-04-25T13:49:48.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:48 smithi078 ceph-mon[29163]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:49:50.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:50 smithi078 ceph-mon[29163]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:49:50.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:50 smithi078 ceph-mon[29163]: Deploying cephadm binary to smithi078 2024-04-25T13:49:53.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Added host 'smithi078' with addr '172.21.15.78' 2024-04-25T13:49:53.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.742+0000 7f80ce55b700 1 Processor -- start 2024-04-25T13:49:53.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.742+0000 7f80ce55b700 1 -- start start 2024-04-25T13:49:53.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.742+0000 7f80ce55b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c810c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:53.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.742+0000 7f80ce55b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80c810cb30 con 0x7f80c807d690 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.742+0000 7f80c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c810c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.743+0000 7f80c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c810c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55560/0 (socket says 172.21.15.78:55560) 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.743+0000 7f80c7fff700 1 -- 172.21.15.78:0/1150494365 learned_addr learned my addr 172.21.15.78:0/1150494365 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.743+0000 7f80c7fff700 1 -- 172.21.15.78:0/1150494365 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c810ccb0 con 0x7f80c807d690 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.743+0000 7f80c7fff700 1 --2- 172.21.15.78:0/1150494365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c810c520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098a0 tx=0x7f80b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3686320f6e692235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.743+0000 7f80c6ffd700 1 -- 172.21.15.78:0/1150494365 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b8004160 con 0x7f80c807d690 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.743+0000 7f80c6ffd700 1 -- 172.21.15.78:0/1150494365 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f80b8004300 con 0x7f80c807d690 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.743+0000 7f80c6ffd700 1 -- 172.21.15.78:0/1150494365 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b8011af0 con 0x7f80c807d690 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.744+0000 7f80ce55b700 1 -- 172.21.15.78:0/1150494365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 msgr2=0x7f80c810c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.744+0000 7f80ce55b700 1 --2- 172.21.15.78:0/1150494365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c810c520 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098a0 tx=0x7f80b8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.744+0000 7f80ce55b700 1 -- 172.21.15.78:0/1150494365 shutdown_connections 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.744+0000 7f80ce55b700 1 --2- 172.21.15.78:0/1150494365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c810c520 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:53.544 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.744+0000 7f80ce55b700 1 -- 172.21.15.78:0/1150494365 >> 172.21.15.78:0/1150494365 conn(0x7f80c8103f50 msgr2=0x7f80c81063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.744+0000 7f80ce55b700 1 -- 172.21.15.78:0/1150494365 shutdown_connections 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.744+0000 7f80ce55b700 1 -- 172.21.15.78:0/1150494365 wait complete. 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80ce55b700 1 Processor -- start 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80ce55b700 1 -- start start 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80ce55b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80ce55b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80c810dd30 con 0x7f80c807d690 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55566/0 (socket says 172.21.15.78:55566) 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80c7fff700 1 -- 172.21.15.78:0/3422637623 learned_addr learned my addr 172.21.15.78:0/3422637623 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.745+0000 7f80c7fff700 1 -- 172.21.15.78:0/3422637623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c819e270 con 0x7f80c807d690 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.746+0000 7f80c7fff700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c819dd30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f80b80117b0 tx=0x7f80b8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.746+0000 7f80c57fa700 1 -- 172.21.15.78:0/3422637623 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b8003af0 con 0x7f80c807d690 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.746+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c819e500 con 0x7f80c807d690 2024-04-25T13:49:53.545 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.746+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c81a1100 con 0x7f80c807d690 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.746+0000 7f80c57fa700 1 -- 172.21.15.78:0/3422637623 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f80b8020070 con 0x7f80c807d690 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.746+0000 7f80c57fa700 1 -- 172.21.15.78:0/3422637623 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b802a060 con 0x7f80c807d690 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.747+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b4005360 con 0x7f80c807d690 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.747+0000 7f80c57fa700 1 -- 172.21.15.78:0/3422637623 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f80b8019450 con 0x7f80c807d690 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.747+0000 7f80c57fa700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f80b003d5c0 0x7f80b003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.747+0000 7f80c57fa700 1 -- 172.21.15.78:0/3422637623 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f80b8052900 con 0x7f80c807d690 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.747+0000 7f80c77fe700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f80b003d5c0 0x7f80b003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.748+0000 7f80c77fe700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f80b003d5c0 0x7f80b003fad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f80bc006fd0 tx=0x7f80bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.752+0000 7f80c57fa700 1 -- 172.21.15.78:0/3422637623 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80b8016260 con 0x7f80c807d690 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:48.958+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}) -- 0x7f80b4000bf0 con 0x7f80b003d5c0 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.966+0000 7f80c57fa700 1 -- 172.21.15.78:0/3422637623 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f80b4000bf0 con 0x7f80b003d5c0 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f80b003d5c0 msgr2=0x7f80b003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:53.546 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f80b003d5c0 0x7f80b003fad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f80bc006fd0 tx=0x7f80bc006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 msgr2=0x7f80c819dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c819dd30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f80b80117b0 tx=0x7f80b8011b70 comp rx=0 tx=0).stop 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 shutdown_connections 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f80b003d5c0 0x7f80b003fad0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 --2- 172.21.15.78:0/3422637623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80c807d690 0x7f80c819dd30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 >> 172.21.15.78:0/3422637623 conn(0x7f80c8103f50 msgr2=0x7f80c8104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 shutdown_connections 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:52.969+0000 7f80ce55b700 1 -- 172.21.15.78:0/3422637623 wait complete. 2024-04-25T13:49:53.547 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mon service... 2024-04-25T13:49:54.094 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:53 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:54.094 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:53 smithi078 ceph-mon[29163]: Added host smithi078 2024-04-25T13:49:54.095 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:53 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.084+0000 7f2330547700 1 Processor -- start 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.084+0000 7f2330547700 1 -- start start 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.084+0000 7f2330547700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232810c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.084+0000 7f2330547700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f232807cd70 con 0x7f232810a4b0 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.085+0000 7f232e2e3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232810c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.085+0000 7f232e2e3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232810c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55576/0 (socket says 172.21.15.78:55576) 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.085+0000 7f232e2e3700 1 -- 172.21.15.78:0/3858975321 learned_addr learned my addr 172.21.15.78:0/3858975321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.085+0000 7f232e2e3700 1 -- 172.21.15.78:0/3858975321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232810ce10 con 0x7f232810a4b0 2024-04-25T13:49:54.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.085+0000 7f232e2e3700 1 --2- 172.21.15.78:0/3858975321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232810c8d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2318009e80 tx=0x7f231800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee824b6f9e505b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.085+0000 7f232d2e1700 1 -- 172.21.15.78:0/3858975321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2318017070 con 0x7f232810a4b0 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.085+0000 7f232d2e1700 1 -- 172.21.15.78:0/3858975321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f231800b8c0 con 0x7f232810a4b0 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f232d2e1700 1 -- 172.21.15.78:0/3858975321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2318004490 con 0x7f232810a4b0 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f2330547700 1 -- 172.21.15.78:0/3858975321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 msgr2=0x7f232810c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f2330547700 1 --2- 172.21.15.78:0/3858975321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232810c8d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2318009e80 tx=0x7f231800b330 comp rx=0 tx=0).stop 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f2330547700 1 -- 172.21.15.78:0/3858975321 shutdown_connections 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f2330547700 1 --2- 172.21.15.78:0/3858975321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232810c8d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f2330547700 1 -- 172.21.15.78:0/3858975321 >> 172.21.15.78:0/3858975321 conn(0x7f2328103fb0 msgr2=0x7f2328106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f2330547700 1 -- 172.21.15.78:0/3858975321 shutdown_connections 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.086+0000 7f2330547700 1 -- 172.21.15.78:0/3858975321 wait complete. 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.087+0000 7f2330547700 1 Processor -- start 2024-04-25T13:49:54.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.087+0000 7f2330547700 1 -- start start 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.087+0000 7f2330547700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.087+0000 7f2330547700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f232810dd10 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.087+0000 7f232e2e3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.087+0000 7f232e2e3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55586/0 (socket says 172.21.15.78:55586) 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.087+0000 7f232e2e3700 1 -- 172.21.15.78:0/500910962 learned_addr learned my addr 172.21.15.78:0/500910962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.088+0000 7f232e2e3700 1 -- 172.21.15.78:0/500910962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232819e210 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.088+0000 7f232e2e3700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232819dcd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2318004220 tx=0x7f231800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.088+0000 7f23277fe700 1 -- 172.21.15.78:0/500910962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2318020020 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.088+0000 7f23277fe700 1 -- 172.21.15.78:0/500910962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2318012430 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.088+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232819e4a0 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.088+0000 7f23277fe700 1 -- 172.21.15.78:0/500910962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2318017040 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.088+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23281a10b0 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.089+0000 7f23277fe700 1 -- 172.21.15.78:0/500910962 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f2318023020 con 0x7f232810a4b0 2024-04-25T13:49:54.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.089+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2328064cf0 con 0x7f232810a4b0 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.089+0000 7f23277fe700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f231003d520 0x7f231003fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.089+0000 7f23277fe700 1 -- 172.21.15.78:0/500910962 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f231800f050 con 0x7f232810a4b0 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.089+0000 7f232dae2700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f231003d520 0x7f231003fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.090+0000 7f232dae2700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f231003d520 0x7f231003fa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f231c006fd0 tx=0x7f231c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.094+0000 7f23277fe700 1 -- 172.21.15.78:0/500910962 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f231802a020 con 0x7f232810a4b0 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.337+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2328105fa0 con 0x7f231003d520 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.343+0000 7f23277fe700 1 -- 172.21.15.78:0/500910962 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2328105fa0 con 0x7f231003d520 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f231003d520 msgr2=0x7f231003fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f231003d520 0x7f231003fa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f231c006fd0 tx=0x7f231c006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 msgr2=0x7f232819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232819dcd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2318004220 tx=0x7f231800bd50 comp rx=0 tx=0).stop 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 shutdown_connections 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f231003d520 0x7f231003fa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 --2- 172.21.15.78:0/500910962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f232810a4b0 0x7f232819dcd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:54.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 >> 172.21.15.78:0/500910962 conn(0x7f2328103fb0 msgr2=0x7f2328105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:54.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 shutdown_connections 2024-04-25T13:49:54.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:54.345+0000 7f2330547700 1 -- 172.21.15.78:0/500910962 wait complete. 2024-04-25T13:49:54.925 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mgr service... 2024-04-25T13:49:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:55 smithi078 ceph-mon[29163]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:49:55.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:55 smithi078 ceph-mon[29163]: Saving service mon spec with placement count:5 2024-04-25T13:49:55.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:55 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:56.915 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:56 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:56.915 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:56 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:56.915 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:56 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:56.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T13:49:56.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.066+0000 7f48912a4700 1 Processor -- start 2024-04-25T13:49:56.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.066+0000 7f48912a4700 1 -- start start 2024-04-25T13:49:56.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.067+0000 7f48912a4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c10e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:56.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.067+0000 7f48912a4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f488c10eea0 con 0x7f488c07d4b0 2024-04-25T13:49:56.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.067+0000 7f488affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c10e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:56.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.067+0000 7f488affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c10e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55600/0 (socket says 172.21.15.78:55600) 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.067+0000 7f488affd700 1 -- 172.21.15.78:0/2703358838 learned_addr learned my addr 172.21.15.78:0/2703358838 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.067+0000 7f488affd700 1 -- 172.21.15.78:0/2703358838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c10f020 con 0x7f488c07d4b0 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.067+0000 7f488affd700 1 --2- 172.21.15.78:0/2703358838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c10e960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f487c0098a0 tx=0x7f487c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1961cb286175ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.068+0000 7f4889ffb700 1 -- 172.21.15.78:0/2703358838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c004160 con 0x7f488c07d4b0 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.068+0000 7f4889ffb700 1 -- 172.21.15.78:0/2703358838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f487c004300 con 0x7f488c07d4b0 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.068+0000 7f4889ffb700 1 -- 172.21.15.78:0/2703358838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c011a20 con 0x7f488c07d4b0 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.068+0000 7f48912a4700 1 -- 172.21.15.78:0/2703358838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 msgr2=0x7f488c10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.068+0000 7f48912a4700 1 --2- 172.21.15.78:0/2703358838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c10e960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f487c0098a0 tx=0x7f487c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.069+0000 7f48912a4700 1 -- 172.21.15.78:0/2703358838 shutdown_connections 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.069+0000 7f48912a4700 1 --2- 172.21.15.78:0/2703358838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c10e960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.069+0000 7f48912a4700 1 -- 172.21.15.78:0/2703358838 >> 172.21.15.78:0/2703358838 conn(0x7f488c103f50 msgr2=0x7f488c1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.069+0000 7f48912a4700 1 -- 172.21.15.78:0/2703358838 shutdown_connections 2024-04-25T13:49:56.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.069+0000 7f48912a4700 1 -- 172.21.15.78:0/2703358838 wait complete. 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.069+0000 7f48912a4700 1 Processor -- start 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.069+0000 7f48912a4700 1 -- start start 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f48912a4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f48912a4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f488c10ff70 con 0x7f488c07d4b0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f488affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f488affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c19fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55602/0 (socket says 172.21.15.78:55602) 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f488affd700 1 -- 172.21.15.78:0/3187127925 learned_addr learned my addr 172.21.15.78:0/3187127925 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f488affd700 1 -- 172.21.15.78:0/3187127925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c1a0400 con 0x7f488c07d4b0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f488affd700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c19fec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f487c006af0 tx=0x7f487c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f4873fff700 1 -- 172.21.15.78:0/3187127925 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c003b20 con 0x7f488c07d4b0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f4873fff700 1 -- 172.21.15.78:0/3187127925 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f487c01b8f0 con 0x7f488c07d4b0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c1a0690 con 0x7f488c07d4b0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.070+0000 7f4873fff700 1 -- 172.21.15.78:0/3187127925 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c0238c0 con 0x7f488c07d4b0 2024-04-25T13:49:56.925 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.071+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c1a32a0 con 0x7f488c07d4b0 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.071+0000 7f4873fff700 1 -- 172.21.15.78:0/3187127925 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f487c023b70 con 0x7f488c07d4b0 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.072+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c064cf0 con 0x7f488c07d4b0 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.072+0000 7f4873fff700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f487403d520 0x7f487403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.072+0000 7f4873fff700 1 -- 172.21.15.78:0/3187127925 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f487c053d00 con 0x7f488c07d4b0 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.072+0000 7f488a7fc700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f487403d520 0x7f487403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.072+0000 7f488a7fc700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f487403d520 0x7f487403fa30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4880006fd0 tx=0x7f4880006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.077+0000 7f4873fff700 1 -- 172.21.15.78:0/3187127925 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f487c01f030 con 0x7f488c07d4b0 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.299+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f488c104ee0 con 0x7f487403d520 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.307+0000 7f4873fff700 1 -- 172.21.15.78:0/3187127925 <== mgr.14120 v2:172.21.15.78:6800/1759244565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f488c104ee0 con 0x7f487403d520 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f487403d520 msgr2=0x7f487403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f487403d520 0x7f487403fa30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4880006fd0 tx=0x7f4880006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 msgr2=0x7f488c19fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:56.926 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c19fec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f487c006af0 tx=0x7f487c011aa0 comp rx=0 tx=0).stop 2024-04-25T13:49:56.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 shutdown_connections 2024-04-25T13:49:56.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f487403d520 0x7f487403fa30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:56.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 --2- 172.21.15.78:0/3187127925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f488c07d4b0 0x7f488c19fec0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:56.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 >> 172.21.15.78:0/3187127925 conn(0x7f488c103f50 msgr2=0x7f488c1047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:56.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 shutdown_connections 2024-04-25T13:49:56.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:56.310+0000 7f48912a4700 1 -- 172.21.15.78:0/3187127925 wait complete. 2024-04-25T13:49:57.890 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:57 smithi078 ceph-mon[29163]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:49:57.890 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:57 smithi078 ceph-mon[29163]: Saving service mgr spec with placement count:2 2024-04-25T13:49:57.890 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:57 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/357159063' entity='client.admin' 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.286+0000 7fdfc16d1700 1 Processor -- start 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.286+0000 7fdfc16d1700 1 -- start start 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.286+0000 7fdfc16d1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.286+0000 7fdfc16d1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1092d0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.287+0000 7fdfbaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.287+0000 7fdfbaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc108ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44044/0 (socket says 172.21.15.78:44044) 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.287+0000 7fdfbaffd700 1 -- 172.21.15.78:0/452523354 learned_addr learned my addr 172.21.15.78:0/452523354 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.287+0000 7fdfbaffd700 1 -- 172.21.15.78:0/452523354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc109a00 con 0x7fdfbc1088d0 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.287+0000 7fdfbaffd700 1 --2- 172.21.15.78:0/452523354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc108ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdfac010b60 tx=0x7fdfac010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792496c23dbae03b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.287+0000 7fdfb9ffb700 1 -- 172.21.15.78:0/452523354 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfac00bdd0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.239 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.287+0000 7fdfb9ffb700 1 -- 172.21.15.78:0/452523354 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac0044e0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.288+0000 7fdfb9ffb700 1 -- 172.21.15.78:0/452523354 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfac0185b0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.288+0000 7fdfc16d1700 1 -- 172.21.15.78:0/452523354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 msgr2=0x7fdfbc108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.288+0000 7fdfc16d1700 1 --2- 172.21.15.78:0/452523354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc108ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdfac010b60 tx=0x7fdfac010e70 comp rx=0 tx=0).stop 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.289+0000 7fdfc16d1700 1 -- 172.21.15.78:0/452523354 shutdown_connections 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.289+0000 7fdfc16d1700 1 --2- 172.21.15.78:0/452523354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc108ce0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.289+0000 7fdfc16d1700 1 -- 172.21.15.78:0/452523354 >> 172.21.15.78:0/452523354 conn(0x7fdfbc103f10 msgr2=0x7fdfbc1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.289+0000 7fdfc16d1700 1 -- 172.21.15.78:0/452523354 shutdown_connections 2024-04-25T13:49:58.240 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.289+0000 7fdfc16d1700 1 -- 172.21.15.78:0/452523354 wait complete. 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.289+0000 7fdfc16d1700 1 Processor -- start 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.289+0000 7fdfc16d1700 1 -- start start 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfc16d1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfc16d1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc07cb10 con 0x7fdfbc1088d0 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfbaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfbaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc19bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44054/0 (socket says 172.21.15.78:44054) 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfbaffd700 1 -- 172.21.15.78:0/357159063 learned_addr learned my addr 172.21.15.78:0/357159063 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfbaffd700 1 -- 172.21.15.78:0/357159063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc19c040 con 0x7fdfbc1088d0 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfbaffd700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc19bb00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006af0 tx=0x7fdfac0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfa3fff700 1 -- 172.21.15.78:0/357159063 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfac004330 con 0x7fdfbc1088d0 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc19c2d0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.241 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfa3fff700 1 -- 172.21.15.78:0/357159063 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac018db0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.290+0000 7fdfa3fff700 1 -- 172.21.15.78:0/357159063 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfac021d80 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.291+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc19eee0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.292+0000 7fdfa3fff700 1 -- 172.21.15.78:0/357159063 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fdfac028030 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.292+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc064cf0 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.292+0000 7fdfa3fff700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fdfa403d710 0x7fdfa403fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.292+0000 7fdfa3fff700 1 -- 172.21.15.78:0/357159063 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdfac026070 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.292+0000 7fdfba7fc700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fdfa403d710 0x7fdfa403fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.293+0000 7fdfba7fc700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fdfa403d710 0x7fdfa403fc20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0006fd0 tx=0x7fdfb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.298+0000 7fdfa3fff700 1 -- 172.21.15.78:0/357159063 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfac02f020 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.505+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fdfbc194d60 con 0x7fdfbc1088d0 2024-04-25T13:49:58.242 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.508+0000 7fdfa3fff700 1 -- 172.21.15.78:0/357159063 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fdfac015290 con 0x7fdfbc1088d0 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fdfa403d710 msgr2=0x7fdfa403fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fdfa403d710 0x7fdfa403fc20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0006fd0 tx=0x7fdfb0006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 msgr2=0x7fdfbc19bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc19bb00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006af0 tx=0x7fdfac0039f0 comp rx=0 tx=0).stop 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 shutdown_connections 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fdfa403d710 0x7fdfa403fc20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 --2- 172.21.15.78:0/357159063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdfbc1088d0 0x7fdfbc19bb00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 >> 172.21.15.78:0/357159063 conn(0x7fdfbc103f10 msgr2=0x7fdfbc191e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 shutdown_connections 2024-04-25T13:49:58.243 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:57.511+0000 7fdfc16d1700 1 -- 172.21.15.78:0/357159063 wait complete. 2024-04-25T13:49:59.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:59 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:49:59.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:49:59 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1571647631' entity='client.admin' 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.132+0000 7fa1a1015700 1 Processor -- start 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.132+0000 7fa1a1015700 1 -- start start 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa1a1015700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa1a1015700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c108a20 con 0x7fa19c108020 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44064/0 (socket says 172.21.15.78:44064) 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa19ad9d700 1 -- 172.21.15.78:0/1104048354 learned_addr learned my addr 172.21.15.78:0/1104048354 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa19ad9d700 1 -- 172.21.15.78:0/1104048354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c109150 con 0x7fa19c108020 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa19ad9d700 1 --2- 172.21.15.78:0/1104048354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c108430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00aa40 tx=0x7fa18c00ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feda876bfe7b40c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa199d9b700 1 -- 172.21.15.78:0/1104048354 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa18c00b040 con 0x7fa19c108020 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa199d9b700 1 -- 172.21.15.78:0/1104048354 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18c00c8f0 con 0x7fa19c108020 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.133+0000 7fa199d9b700 1 -- 172.21.15.78:0/1104048354 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa18c003ae0 con 0x7fa19c108020 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.134+0000 7fa1a1015700 1 -- 172.21.15.78:0/1104048354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 msgr2=0x7fa19c108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.134+0000 7fa1a1015700 1 --2- 172.21.15.78:0/1104048354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c108430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00aa40 tx=0x7fa18c00ad50 comp rx=0 tx=0).stop 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.134+0000 7fa1a1015700 1 -- 172.21.15.78:0/1104048354 shutdown_connections 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.134+0000 7fa1a1015700 1 --2- 172.21.15.78:0/1104048354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c108430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.134+0000 7fa1a1015700 1 -- 172.21.15.78:0/1104048354 >> 172.21.15.78:0/1104048354 conn(0x7fa19c1036e0 msgr2=0x7fa19c105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.134+0000 7fa1a1015700 1 -- 172.21.15.78:0/1104048354 shutdown_connections 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.134+0000 7fa1a1015700 1 -- 172.21.15.78:0/1104048354 wait complete. 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.135+0000 7fa1a1015700 1 Processor -- start 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.135+0000 7fa1a1015700 1 -- start start 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa1a1015700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa1a1015700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c109920 con 0x7fa19c108020 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44078/0 (socket says 172.21.15.78:44078) 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa19ad9d700 1 -- 172.21.15.78:0/1571647631 learned_addr learned my addr 172.21.15.78:0/1571647631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa19ad9d700 1 -- 172.21.15.78:0/1571647631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c19e280 con 0x7fa19c108020 2024-04-25T13:49:59.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa19ad9d700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c19dd40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa18c003780 tx=0x7fa18c00c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa183fff700 1 -- 172.21.15.78:0/1571647631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa18c025070 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa183fff700 1 -- 172.21.15.78:0/1571647631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18c004290 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19c19e510 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa183fff700 1 -- 172.21.15.78:0/1571647631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa18c010580 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.136+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19c1a1110 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.137+0000 7fa183fff700 1 -- 172.21.15.78:0/1571647631 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fa18c028070 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.137+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa19c064cf0 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.138+0000 7fa183fff700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fa18403d570 0x7fa18403fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.138+0000 7fa183fff700 1 -- 172.21.15.78:0/1571647631 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa18c053070 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.138+0000 7fa19a59c700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fa18403d570 0x7fa18403fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.138+0000 7fa19a59c700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fa18403d570 0x7fa18403fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa190006fd0 tx=0x7fa190006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.143+0000 7fa183fff700 1 -- 172.21.15.78:0/1571647631 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa18c014070 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.348+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa19c02e840 con 0x7fa19c108020 2024-04-25T13:49:59.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.351+0000 7fa183fff700 1 -- 172.21.15.78:0/1571647631 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa18c010870 con 0x7fa19c108020 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fa18403d570 msgr2=0x7fa18403fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fa18403d570 0x7fa18403fa80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa190006fd0 tx=0x7fa190006e40 comp rx=0 tx=0).stop 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 msgr2=0x7fa19c19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c19dd40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa18c003780 tx=0x7fa18c00c0b0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 shutdown_connections 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fa18403d570 0x7fa18403fa80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 --2- 172.21.15.78:0/1571647631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa19c108020 0x7fa19c19dd40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 >> 172.21.15.78:0/1571647631 conn(0x7fa19c1036e0 msgr2=0x7fa19c192110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 shutdown_connections 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:49:59.354+0000 7fa1a1015700 1 -- 172.21.15.78:0/1571647631 wait complete. 2024-04-25T13:49:59.969 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling the dashboard module... 2024-04-25T13:50:01.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-mon[29163]: from='mgr.14120 172.21.15.78:0/307253016' entity='mgr.x' 2024-04-25T13:50:01.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3575075142' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T13:50:01.256 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: ignoring --setuser ceph since I am not root 2024-04-25T13:50:01.256 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: ignoring --setgroup ceph since I am not root 2024-04-25T13:50:01.256 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:01.255+0000 7f2823f2b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:50:01.551 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:01.446+0000 7f2823f2b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:50:01.551 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:01.551+0000 7f2823f2b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.369+0000 7f3802fc5700 1 Processor -- start 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.369+0000 7f3802fc5700 1 -- start start 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.369+0000 7f3802fc5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc10c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.369+0000 7f3802fc5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc10cae0 con 0x7f37fc07cc40 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.369+0000 7f3800d61700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc10c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.369+0000 7f3800d61700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc10c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44086/0 (socket says 172.21.15.78:44086) 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.369+0000 7f3800d61700 1 -- 172.21.15.78:0/1563740543 learned_addr learned my addr 172.21.15.78:0/1563740543 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.370+0000 7f3800d61700 1 -- 172.21.15.78:0/1563740543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc10cc60 con 0x7f37fc07cc40 2024-04-25T13:50:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.370+0000 7f3800d61700 1 --2- 172.21.15.78:0/1563740543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc10c4f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37f00098a0 tx=0x7f37f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf8f510a561ab02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.370+0000 7f37fb7fe700 1 -- 172.21.15.78:0/1563740543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f0004160 con 0x7f37fc07cc40 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.370+0000 7f37fb7fe700 1 -- 172.21.15.78:0/1563740543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f0004300 con 0x7f37fc07cc40 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.370+0000 7f37fb7fe700 1 -- 172.21.15.78:0/1563740543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f0011a20 con 0x7f37fc07cc40 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.371+0000 7f3802fc5700 1 -- 172.21.15.78:0/1563740543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 msgr2=0x7f37fc10c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.371+0000 7f3802fc5700 1 --2- 172.21.15.78:0/1563740543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc10c4f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37f00098a0 tx=0x7f37f0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.371+0000 7f3802fc5700 1 -- 172.21.15.78:0/1563740543 shutdown_connections 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.371+0000 7f3802fc5700 1 --2- 172.21.15.78:0/1563740543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc10c4f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.371+0000 7f3802fc5700 1 -- 172.21.15.78:0/1563740543 >> 172.21.15.78:0/1563740543 conn(0x7f37fc103f80 msgr2=0x7f37fc1063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.371+0000 7f3802fc5700 1 -- 172.21.15.78:0/1563740543 shutdown_connections 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.371+0000 7f3802fc5700 1 -- 172.21.15.78:0/1563740543 wait complete. 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3802fc5700 1 Processor -- start 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3802fc5700 1 -- start start 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3802fc5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.768 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3802fc5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc10dd20 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3800d61700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3800d61700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc19dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44102/0 (socket says 172.21.15.78:44102) 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3800d61700 1 -- 172.21.15.78:0/3575075142 learned_addr learned my addr 172.21.15.78:0/3575075142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3800d61700 1 -- 172.21.15.78:0/3575075142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc19e1e0 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.372+0000 7f3800d61700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc19dca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f37f0000c00 tx=0x7f37f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.373+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f0003aa0 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.373+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f001b8f0 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.373+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f0023890 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.373+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37fc19e470 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.373+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37fc1a1080 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.374+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f37f002c3f0 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.374+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37fc064cf0 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.374+0000 7f37f9ffb700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f37ec03d5c0 0x7f37ec03fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.374+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f37f0052cf0 con 0x7f37fc07cc40 2024-04-25T13:50:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.374+0000 7f37fbfff700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f37ec03d5c0 0x7f37ec03fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.375+0000 7f37fbfff700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f37ec03d5c0 0x7f37ec03fad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f37e8006fd0 tx=0x7f37e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.379+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37f001f030 con 0x7f37fc07cc40 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:00.650+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f37fc1a1250 con 0x7f37fc07cc40 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.049+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f37f001bda0 con 0x7f37fc07cc40 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f37f9ffb700 1 -- 172.21.15.78:0/3575075142 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f37f002bcc0 con 0x7f37fc07cc40 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f37ec03d5c0 msgr2=0x7f37ec03fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f37ec03d5c0 0x7f37ec03fad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f37e8006fd0 tx=0x7f37e8006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 msgr2=0x7f37fc19dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc19dca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f37f0000c00 tx=0x7f37f0011aa0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 shutdown_connections 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f37ec03d5c0 0x7f37ec03fad0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 --2- 172.21.15.78:0/3575075142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37fc07cc40 0x7f37fc19dca0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:01.770 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.051+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 >> 172.21.15.78:0/3575075142 conn(0x7f37fc103f80 msgr2=0x7f37fc1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:01.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.052+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 shutdown_connections 2024-04-25T13:50:01.771 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:01.052+0000 7f3802fc5700 1 -- 172.21.15.78:0/3575075142 wait complete. 2024-04-25T13:50:02.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:02 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3575075142' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T13:50:02.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:02 smithi078 ceph-mon[29163]: mgrmap e9: x(active, since 19s) 2024-04-25T13:50:02.153 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:01.869+0000 7f2823f2b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:50:02.153 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:01.956+0000 7f2823f2b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:50:03.173 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:03 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/734857051' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.632+0000 7fcd8ff0b700 1 Processor -- start 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.632+0000 7fcd8ff0b700 1 -- start start 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.632+0000 7fcd8ff0b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8810c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.632+0000 7fcd8ff0b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8807cd70 con 0x7fcd8810a4b0 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.632+0000 7fcd8dca7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8810c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:03.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.632+0000 7fcd8dca7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8810c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44132/0 (socket says 172.21.15.78:44132) 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.632+0000 7fcd8dca7700 1 -- 172.21.15.78:0/195444846 learned_addr learned my addr 172.21.15.78:0/195444846 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.633+0000 7fcd8dca7700 1 -- 172.21.15.78:0/195444846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8810ce10 con 0x7fcd8810a4b0 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.633+0000 7fcd8dca7700 1 --2- 172.21.15.78:0/195444846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8810c8d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd840098a0 tx=0x7fcd84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fae6f0d6b416b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.633+0000 7fcd8cca5700 1 -- 172.21.15.78:0/195444846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd84004160 con 0x7fcd8810a4b0 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.633+0000 7fcd8cca5700 1 -- 172.21.15.78:0/195444846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd84004300 con 0x7fcd8810a4b0 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.633+0000 7fcd8cca5700 1 -- 172.21.15.78:0/195444846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd84011a20 con 0x7fcd8810a4b0 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.634+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/195444846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 msgr2=0x7fcd8810c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.634+0000 7fcd8ff0b700 1 --2- 172.21.15.78:0/195444846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8810c8d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd840098a0 tx=0x7fcd84009bb0 comp rx=0 tx=0).stop 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.634+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/195444846 shutdown_connections 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.634+0000 7fcd8ff0b700 1 --2- 172.21.15.78:0/195444846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8810c8d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.634+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/195444846 >> 172.21.15.78:0/195444846 conn(0x7fcd88103fb0 msgr2=0x7fcd88106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.634+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/195444846 shutdown_connections 2024-04-25T13:50:03.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.634+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/195444846 wait complete. 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8ff0b700 1 Processor -- start 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8ff0b700 1 -- start start 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8ff0b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8ff0b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8810dd50 con 0x7fcd8810a4b0 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8dca7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8dca7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44140/0 (socket says 172.21.15.78:44140) 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8dca7700 1 -- 172.21.15.78:0/734857051 learned_addr learned my addr 172.21.15.78:0/734857051 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8dca7700 1 -- 172.21.15.78:0/734857051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8819e210 con 0x7fcd8810a4b0 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd8dca7700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8819dcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.635+0000 7fcd7effd700 1 -- 172.21.15.78:0/734857051 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd84003aa0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.636+0000 7fcd7effd700 1 -- 172.21.15.78:0/734857051 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd8401b8f0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.636+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8819e4a0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.636+0000 7fcd7effd700 1 -- 172.21.15.78:0/734857051 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd84023890 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.636+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd881a10b0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.637+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd88064cf0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.637+0000 7fcd7effd700 1 -- 172.21.15.78:0/734857051 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fcd8402c3f0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.637+0000 7fcd7effd700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 0x7fcd7403fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.637+0000 7fcd7effd700 1 -- 172.21.15.78:0/734857051 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd84052e00 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.637+0000 7fcd8d4a6700 1 -- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 msgr2=0x7fcd7403fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1759244565 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.637+0000 7fcd8d4a6700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 0x7fcd7403fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.642+0000 7fcd7effd700 1 -- 172.21.15.78:0/734857051 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd8401f030 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.838+0000 7fcd8d4a6700 1 -- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 msgr2=0x7fcd7403fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1759244565 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.838+0000 7fcd8d4a6700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 0x7fcd7403fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.903+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcd881a13a0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.904+0000 7fcd7effd700 1 -- 172.21.15.78:0/734857051 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fcd8401bda0 con 0x7fcd8810a4b0 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 msgr2=0x7fcd7403fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:50:03.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 0x7fcd7403fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 msgr2=0x7fcd8819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8819dcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd84011aa0 comp rx=0 tx=0).stop 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 shutdown_connections 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7fcd7403d5c0 0x7fcd7403fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 --2- 172.21.15.78:0/734857051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd8810a4b0 0x7fcd8819dcd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 >> 172.21.15.78:0/734857051 conn(0x7fcd88103fb0 msgr2=0x7fcd881058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 shutdown_connections 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:02.906+0000 7fcd8ff0b700 1 -- 172.21.15.78:0/734857051 wait complete. 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-04-25T13:50:03.432 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 9... 2024-04-25T13:50:03.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:03.173+0000 7f2823f2b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:50:03.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:03.351+0000 7f2823f2b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:50:03.833 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:03.527+0000 7f2823f2b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:50:04.135 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:03.833+0000 7f2823f2b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:50:04.136 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:03.930+0000 7f2823f2b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:50:04.136 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:04.136+0000 7f2823f2b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:50:04.455 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:04.332+0000 7f2823f2b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:50:05.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:04.938+0000 7f2823f2b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:50:05.678 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:05.372+0000 7f2823f2b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:50:05.678 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:05.472+0000 7f2823f2b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:50:05.678 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:05.678+0000 7f2823f2b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:50:06.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:06.201+0000 7f2823f2b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:50:06.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:06.295+0000 7f2823f2b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:50:06.857 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:06.544+0000 7f2823f2b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:50:06.858 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:06.680+0000 7f2823f2b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:50:06.858 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:06.779+0000 7f2823f2b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:50:07.208 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:06.897+0000 7f2823f2b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:50:07.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:07 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:07.208+0000 7f2823f2b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:50:08.089 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:07 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:07.812+0000 7f2823f2b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:50:08.089 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:50:08.035+0000 7f2823f2b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:50:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: Active manager daemon x restarted 2024-04-25T13:50:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: Activating manager daemon x 2024-04-25T13:50:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T13:50:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: mgrmap e10: x(active, starting, since 0.00431331s) 2024-04-25T13:50:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:50:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:50:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:50:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:50:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:50:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: Manager daemon x is now available 2024-04-25T13:50:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:50:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:50:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:09 smithi078 ceph-mon[29163]: mgrmap e11: x(active, since 1.00713s) 2024-04-25T13:50:09.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.848+0000 7f366fc8d700 1 Processor -- start 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.848+0000 7f366fc8d700 1 -- start start 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.848+0000 7f366fc8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f3668108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.848+0000 7f366fc8d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36681092d0 con 0x7f36681088d0 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.848+0000 7f366da29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f3668108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.848+0000 7f366da29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f3668108ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44146/0 (socket says 172.21.15.78:44146) 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.848+0000 7f366da29700 1 -- 172.21.15.78:0/4181811412 learned_addr learned my addr 172.21.15.78:0/4181811412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.849+0000 7f366da29700 1 -- 172.21.15.78:0/4181811412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3668109a00 con 0x7f36681088d0 2024-04-25T13:50:09.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.849+0000 7f366da29700 1 --2- 172.21.15.78:0/4181811412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f3668108ce0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36640098a0 tx=0x7f3664009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f005343e0dc612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.849+0000 7f366ca27700 1 -- 172.21.15.78:0/4181811412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3664004160 con 0x7f36681088d0 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.849+0000 7f366ca27700 1 -- 172.21.15.78:0/4181811412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3664004300 con 0x7f36681088d0 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.849+0000 7f366ca27700 1 -- 172.21.15.78:0/4181811412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3664011a20 con 0x7f36681088d0 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.850+0000 7f366fc8d700 1 -- 172.21.15.78:0/4181811412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 msgr2=0x7f3668108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.850+0000 7f366fc8d700 1 --2- 172.21.15.78:0/4181811412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f3668108ce0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36640098a0 tx=0x7f3664009bb0 comp rx=0 tx=0).stop 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.850+0000 7f366fc8d700 1 -- 172.21.15.78:0/4181811412 shutdown_connections 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.850+0000 7f366fc8d700 1 --2- 172.21.15.78:0/4181811412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f3668108ce0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.850+0000 7f366fc8d700 1 -- 172.21.15.78:0/4181811412 >> 172.21.15.78:0/4181811412 conn(0x7f3668103fb0 msgr2=0x7f3668106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.850+0000 7f366fc8d700 1 -- 172.21.15.78:0/4181811412 shutdown_connections 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.850+0000 7f366fc8d700 1 -- 172.21.15.78:0/4181811412 wait complete. 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366fc8d700 1 Processor -- start 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366fc8d700 1 -- start start 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366fc8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f366807c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366fc8d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f366810cdb0 con 0x7f36681088d0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366da29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f366807c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366da29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f366807c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44152/0 (socket says 172.21.15.78:44152) 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366da29700 1 -- 172.21.15.78:0/2648795849 learned_addr learned my addr 172.21.15.78:0/2648795849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366da29700 1 -- 172.21.15.78:0/2648795849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366807ce20 con 0x7f36681088d0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f366da29700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f366807c8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3664003e70 tx=0x7f3664011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.851+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3664003af0 con 0x7f36681088d0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.852+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f366401a8e0 con 0x7f36681088d0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.852+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36681a0ac0 con 0x7f36681088d0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.852+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3664024870 con 0x7f36681088d0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.852+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36681a0f40 con 0x7f36681088d0 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.853+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f3664022020 con 0x7f36681088d0 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.853+0000 7f365effd700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.853+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 --> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f36540401f0 con 0x7f365403d5d0 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.853+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f36640528f0 con 0x7f36681088d0 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.853+0000 7f366d228700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 msgr2=0x7f365403fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1759244565 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:03.853+0000 7f366d228700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:04.054+0000 7f366d228700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 msgr2=0x7f365403fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1759244565 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:04.054+0000 7f366d228700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:04.455+0000 7f366d228700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 msgr2=0x7f365403fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1759244565 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:04.455+0000 7f366d228700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:05.256+0000 7f366d228700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 msgr2=0x7f365403fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1759244565 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:05.256+0000 7f366d228700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:06.857+0000 7f366d228700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 msgr2=0x7f365403fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1759244565 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:06.857+0000 7f366d228700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:08.040+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7f366402d3f0 con 0x7f36681088d0 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:08.040+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 msgr2=0x7f365403fae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:08.040+0000 7f365effd700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.040+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f3664054190 con 0x7f36681088d0 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.041+0000 7f365effd700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3664052480 0x7f3654042fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.041+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f36540401f0 con 0x7f3664052480 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.041+0000 7f366d228700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3664052480 0x7f3654042fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.042+0000 7f366d228700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3664052480 0x7f3654042fc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3658003a10 tx=0x7f3658009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.043+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f36540401f0 con 0x7f3664052480 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3668064cf0 con 0x7f3664052480 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f365effd700 1 -- 172.21.15.78:0/2648795849 <== mgr.14152 v2:172.21.15.78:6800/3547239980 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3668064cf0 con 0x7f3664052480 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3664052480 msgr2=0x7f3654042fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3664052480 0x7f3654042fc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3658003a10 tx=0x7f3658009270 comp rx=0 tx=0).stop 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 msgr2=0x7f366807c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f366807c8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3664003e70 tx=0x7f3664011aa0 comp rx=0 tx=0).stop 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 shutdown_connections 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36681088d0 0x7f366807c8e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3664052480 0x7f3654042fc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 --2- 172.21.15.78:0/2648795849 >> [v2:172.21.15.78:6800/1759244565,v1:172.21.15.78:6801/1759244565] conn(0x7f365403d5d0 0x7f365403fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 >> 172.21.15.78:0/2648795849 conn(0x7f3668103fb0 msgr2=0x7f3668105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 shutdown_connections 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:09.051+0000 7f366fc8d700 1 -- 172.21.15.78:0/2648795849 wait complete. 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 9 is available 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi078.stdout:Generating a dashboard self-signed certificate... 2024-04-25T13:50:10.137 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:10 smithi078 ceph-mon[29163]: [25/Apr/2024:13:50:08] ENGINE Bus STARTING 2024-04-25T13:50:10.137 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:10 smithi078 ceph-mon[29163]: [25/Apr/2024:13:50:08] ENGINE Serving on http://172.21.15.78:8765 2024-04-25T13:50:10.137 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:10 smithi078 ceph-mon[29163]: [25/Apr/2024:13:50:08] ENGINE Serving on https://172.21.15.78:7150 2024-04-25T13:50:10.137 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:10 smithi078 ceph-mon[29163]: [25/Apr/2024:13:50:08] ENGINE Bus STARTED 2024-04-25T13:50:10.137 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:10 smithi078 ceph-mon[29163]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:50:10.138 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:10 smithi078 ceph-mon[29163]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:50:10.138 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:10 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:11.290 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T13:50:11.290 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.375+0000 7f5168513700 1 Processor -- start 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.375+0000 7f5168513700 1 -- start start 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.375+0000 7f5168513700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516010c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.375+0000 7f5168513700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f516010cae0 con 0x7f516007cc60 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51662af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516010c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51662af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516010c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53492/0 (socket says 172.21.15.78:53492) 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51662af700 1 -- 172.21.15.78:0/958944643 learned_addr learned my addr 172.21.15.78:0/958944643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51662af700 1 -- 172.21.15.78:0/958944643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516010cc60 con 0x7f516007cc60 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51662af700 1 --2- 172.21.15.78:0/958944643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516010c4d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51500098a0 tx=0x7f5150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4e6dcacedd5eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51652ad700 1 -- 172.21.15.78:0/958944643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5150004160 con 0x7f516007cc60 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51652ad700 1 -- 172.21.15.78:0/958944643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5150004300 con 0x7f516007cc60 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.376+0000 7f51652ad700 1 -- 172.21.15.78:0/958944643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5150011a20 con 0x7f516007cc60 2024-04-25T13:50:11.291 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.378+0000 7f5168513700 1 -- 172.21.15.78:0/958944643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 msgr2=0x7f516010c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.378+0000 7f5168513700 1 --2- 172.21.15.78:0/958944643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516010c4d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f51500098a0 tx=0x7f5150009bb0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.378+0000 7f5168513700 1 -- 172.21.15.78:0/958944643 shutdown_connections 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.378+0000 7f5168513700 1 --2- 172.21.15.78:0/958944643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516010c4d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.378+0000 7f5168513700 1 -- 172.21.15.78:0/958944643 >> 172.21.15.78:0/958944643 conn(0x7f5160103f80 msgr2=0x7f51601063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.378+0000 7f5168513700 1 -- 172.21.15.78:0/958944643 shutdown_connections 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.378+0000 7f5168513700 1 -- 172.21.15.78:0/958944643 wait complete. 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f5168513700 1 Processor -- start 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f5168513700 1 -- start start 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f5168513700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f5168513700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f516010dd20 con 0x7f516007cc60 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f51662af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f51662af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516019dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53494/0 (socket says 172.21.15.78:53494) 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f51662af700 1 -- 172.21.15.78:0/398596142 learned_addr learned my addr 172.21.15.78:0/398596142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.379+0000 7f51662af700 1 -- 172.21.15.78:0/398596142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516019e200 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.380+0000 7f51662af700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516019dcc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51500119f0 tx=0x7f5150011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.380+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51500039e0 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.380+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516019e490 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.380+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f515001b8f0 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.380+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51500237a0 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.380+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51601a10a0 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.381+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f5150023aa0 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.381+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5160064cf0 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.381+0000 7f51577fe700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f514c03d460 0x7f514c03f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.381+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5150052ba0 con 0x7f516007cc60 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.381+0000 7f5165aae700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f514c03d460 0x7f514c03f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.382+0000 7f5165aae700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f514c03d460 0x7f514c03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f515c006fd0 tx=0x7f515c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.391+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f515001f030 con 0x7f516007cc60 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.397+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f515002c3f0 con 0x7f516007cc60 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.617+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5160105f90 con 0x7f514c03d460 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.738+0000 7f51577fe700 1 -- 172.21.15.78:0/398596142 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5160105f90 con 0x7f514c03d460 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f514c03d460 msgr2=0x7f514c03f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f514c03d460 0x7f514c03f970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f515c006fd0 tx=0x7f515c006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 msgr2=0x7f516019dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516019dcc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51500119f0 tx=0x7f5150011ad0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 shutdown_connections 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f514c03d460 0x7f514c03f970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 --2- 172.21.15.78:0/398596142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516007cc60 0x7f516019dcc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 >> 172.21.15.78:0/398596142 conn(0x7f5160103f80 msgr2=0x7f5160105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 shutdown_connections 2024-04-25T13:50:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:10.741+0000 7f5168513700 1 -- 172.21.15.78:0/398596142 wait complete. 2024-04-25T13:50:11.295 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial admin user... 2024-04-25T13:50:11.669 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:11 smithi078 ceph-mon[29163]: mgrmap e12: x(active, since 2s) 2024-04-25T13:50:11.669 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:11.669 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:11 smithi078 ceph-mon[29163]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:11.669 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:11.669 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:12 smithi078 ceph-mon[29163]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:12.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$VHTm7juOaSGr41wiABLjAuNRj0QykrM4rzW5em3Wx7jk.GwYZx2qm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714053012, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T13:50:12.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.689+0000 7ff917f45700 1 Processor -- start 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.689+0000 7ff917f45700 1 -- start start 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.689+0000 7ff917f45700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91010c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.689+0000 7ff917f45700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010cb00 con 0x7ff91007d4f0 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff915ce1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91010c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff915ce1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91010c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53510/0 (socket says 172.21.15.78:53510) 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff915ce1700 1 -- 172.21.15.78:0/4077498510 learned_addr learned my addr 172.21.15.78:0/4077498510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff915ce1700 1 -- 172.21.15.78:0/4077498510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91010cc80 con 0x7ff91007d4f0 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff915ce1700 1 --2- 172.21.15.78:0/4077498510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91010c4f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e80 tx=0x7ff90800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3543dedb20a9363d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff914cdf700 1 -- 172.21.15.78:0/4077498510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff908017070 con 0x7ff91007d4f0 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff914cdf700 1 -- 172.21.15.78:0/4077498510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90800b8c0 con 0x7ff91007d4f0 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.690+0000 7ff914cdf700 1 -- 172.21.15.78:0/4077498510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff908004490 con 0x7ff91007d4f0 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.691+0000 7ff917f45700 1 -- 172.21.15.78:0/4077498510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 msgr2=0x7ff91010c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.691+0000 7ff917f45700 1 --2- 172.21.15.78:0/4077498510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91010c4f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e80 tx=0x7ff90800b330 comp rx=0 tx=0).stop 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.691+0000 7ff917f45700 1 -- 172.21.15.78:0/4077498510 shutdown_connections 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.691+0000 7ff917f45700 1 --2- 172.21.15.78:0/4077498510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91010c4f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.691+0000 7ff917f45700 1 -- 172.21.15.78:0/4077498510 >> 172.21.15.78:0/4077498510 conn(0x7ff910103f00 msgr2=0x7ff9101063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.691+0000 7ff917f45700 1 -- 172.21.15.78:0/4077498510 shutdown_connections 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.691+0000 7ff917f45700 1 -- 172.21.15.78:0/4077498510 wait complete. 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff917f45700 1 Processor -- start 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff917f45700 1 -- start start 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff917f45700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff917f45700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010dd40 con 0x7ff91007d4f0 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff915ce1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff915ce1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53526/0 (socket says 172.21.15.78:53526) 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff915ce1700 1 -- 172.21.15.78:0/2858416802 learned_addr learned my addr 172.21.15.78:0/2858416802 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:12.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff915ce1700 1 -- 172.21.15.78:0/2858416802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91019e1c0 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.692+0000 7ff915ce1700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91019dc80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e50 tx=0x7ff90800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.693+0000 7ff906ffd700 1 -- 172.21.15.78:0/2858416802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff908020020 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.693+0000 7ff906ffd700 1 -- 172.21.15.78:0/2858416802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff908012430 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.693+0000 7ff906ffd700 1 -- 172.21.15.78:0/2858416802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff908017040 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.693+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91019e450 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.693+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101a1060 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.694+0000 7ff906ffd700 1 -- 172.21.15.78:0/2858416802 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7ff908023020 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.694+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff910064cf0 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.694+0000 7ff906ffd700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff8fc03d630 0x7ff8fc03fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.694+0000 7ff906ffd700 1 -- 172.21.15.78:0/2858416802 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff90800f050 con 0x7ff91007d4f0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.694+0000 7ff9154e0700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff8fc03d630 0x7ff8fc03fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.694+0000 7ff9154e0700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff8fc03d630 0x7ff8fc03fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.699+0000 7ff906ffd700 1 -- 172.21.15.78:0/2858416802 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff90802a020 con 0x7ff91007d4f0 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:11.923+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff9101a12f0 con 0x7ff8fc03d630 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.147+0000 7ff906ffd700 1 -- 172.21.15.78:0/2858416802 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff9101a12f0 con 0x7ff8fc03d630 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff8fc03d630 msgr2=0x7ff8fc03fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff8fc03d630 0x7ff8fc03fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 msgr2=0x7ff91019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91019dc80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff908009e50 tx=0x7ff90800bd50 comp rx=0 tx=0).stop 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 shutdown_connections 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff8fc03d630 0x7ff8fc03fb40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 --2- 172.21.15.78:0/2858416802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff91007d4f0 0x7ff91019dc80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 >> 172.21.15.78:0/2858416802 conn(0x7ff910103f00 msgr2=0x7ff910104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 shutdown_connections 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:12.149+0000 7ff917f45700 1 -- 172.21.15.78:0/2858416802 wait complete. 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:Fetching dashboard port number... 2024-04-25T13:50:13.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/241051752' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T13:50:14.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T13:50:14.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.301+0000 7f4059e8d700 1 Processor -- start 2024-04-25T13:50:14.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.301+0000 7f4059e8d700 1 -- start start 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.301+0000 7f4059e8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405410b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.301+0000 7f4059e8d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f405410b930 con 0x7f405410af50 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.302+0000 7f40537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405410b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.302+0000 7f40537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405410b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53528/0 (socket says 172.21.15.78:53528) 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.302+0000 7f40537fe700 1 -- 172.21.15.78:0/1426558519 learned_addr learned my addr 172.21.15.78:0/1426558519 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.302+0000 7f40537fe700 1 -- 172.21.15.78:0/1426558519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405410bab0 con 0x7f405410af50 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.302+0000 7f40537fe700 1 --2- 172.21.15.78:0/1426558519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405410b360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40480098a0 tx=0x7f4048009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0226a9de6841b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.302+0000 7f40527fc700 1 -- 172.21.15.78:0/1426558519 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4048004160 con 0x7f405410af50 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.303+0000 7f40527fc700 1 -- 172.21.15.78:0/1426558519 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4048004300 con 0x7f405410af50 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.303+0000 7f40527fc700 1 -- 172.21.15.78:0/1426558519 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4048011a20 con 0x7f405410af50 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 -- 172.21.15.78:0/1426558519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 msgr2=0x7f405410b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 --2- 172.21.15.78:0/1426558519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405410b360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40480098a0 tx=0x7f4048009bb0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 -- 172.21.15.78:0/1426558519 shutdown_connections 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 --2- 172.21.15.78:0/1426558519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405410b360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 -- 172.21.15.78:0/1426558519 >> 172.21.15.78:0/1426558519 conn(0x7f4054103710 msgr2=0x7f4054105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 -- 172.21.15.78:0/1426558519 shutdown_connections 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 -- 172.21.15.78:0/1426558519 wait complete. 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.304+0000 7f4059e8d700 1 Processor -- start 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f4059e8d700 1 -- start start 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f4059e8d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f4059e8d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f405410def0 con 0x7f405410af50 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f40537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f40537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405419dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53536/0 (socket says 172.21.15.78:53536) 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f40537fe700 1 -- 172.21.15.78:0/241051752 learned_addr learned my addr 172.21.15.78:0/241051752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f40537fe700 1 -- 172.21.15.78:0/241051752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405419e210 con 0x7f405410af50 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.305+0000 7f40537fe700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405419dcd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40480119f0 tx=0x7f4048011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.202 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.306+0000 7f4050ff9700 1 -- 172.21.15.78:0/241051752 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40480039e0 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.306+0000 7f4050ff9700 1 -- 172.21.15.78:0/241051752 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f404801b8f0 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.306+0000 7f4050ff9700 1 -- 172.21.15.78:0/241051752 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40480237a0 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.306+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405419e4a0 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.306+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40541a10b0 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.307+0000 7f4050ff9700 1 -- 172.21.15.78:0/241051752 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f40480193d0 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.307+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4054064cf0 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.307+0000 7f4050ff9700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f403c03d5d0 0x7f403c03fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.307+0000 7f4050ff9700 1 -- 172.21.15.78:0/241051752 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4048052f40 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.307+0000 7f4052ffd700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f403c03d5d0 0x7f403c03fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.307+0000 7f4052ffd700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f403c03d5d0 0x7f403c03fae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4044006fd0 tx=0x7f4044006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.313+0000 7f4050ff9700 1 -- 172.21.15.78:0/241051752 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f404801f030 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.519+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f40541a1280 con 0x7f405410af50 2024-04-25T13:50:14.203 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.520+0000 7f4050ff9700 1 -- 172.21.15.78:0/241051752 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f40480172b0 con 0x7f405410af50 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f403c03d5d0 msgr2=0x7f403c03fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f403c03d5d0 0x7f403c03fae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4044006fd0 tx=0x7f4044006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 msgr2=0x7f405419dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405419dcd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40480119f0 tx=0x7f4048011ad0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 shutdown_connections 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f403c03d5d0 0x7f403c03fae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 --2- 172.21.15.78:0/241051752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f405410af50 0x7f405419dcd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 >> 172.21.15.78:0/241051752 conn(0x7f4054103710 msgr2=0x7f4054104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 shutdown_connections 2024-04-25T13:50:14.204 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:13.522+0000 7f4059e8d700 1 -- 172.21.15.78:0/241051752 wait complete. 2024-04-25T13:50:14.207 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:50:14.207 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T13:50:14.214 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:50:14.214 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T13:50:14.215 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T13:50:14.215 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T13:50:14.216 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph Dashboard is now available at: 2024-04-25T13:50:14.216 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:14.216 INFO:teuthology.orchestra.run.smithi078.stdout: URL: https://smithi078:8443/ 2024-04-25T13:50:14.216 INFO:teuthology.orchestra.run.smithi078.stdout: User: admin 2024-04-25T13:50:14.216 INFO:teuthology.orchestra.run.smithi078.stdout: Password: w22v9k2nkl 2024-04-25T13:50:14.216 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:14.217 INFO:teuthology.orchestra.run.smithi078.stdout:Saving cluster configuration to /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config directory 2024-04-25T13:50:14.217 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling autotune for osd_memory_target 2024-04-25T13:50:15.389 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.648+0000 7fb2c4cc8700 1 Processor -- start 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.648+0000 7fb2c4cc8700 1 -- start start 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.648+0000 7fb2c4cc8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c0109db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.648+0000 7fb2c4cc8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010a3a0 con 0x7fb2c01099a0 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.648+0000 7fb2be59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c0109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.648+0000 7fb2be59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c0109db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53544/0 (socket says 172.21.15.78:53544) 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.648+0000 7fb2be59c700 1 -- 172.21.15.78:0/4194406843 learned_addr learned my addr 172.21.15.78:0/4194406843 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.649+0000 7fb2be59c700 1 -- 172.21.15.78:0/4194406843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c010aad0 con 0x7fb2c01099a0 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.649+0000 7fb2be59c700 1 --2- 172.21.15.78:0/4194406843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c0109db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00098a0 tx=0x7fb2b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608e95a95fce6923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.649+0000 7fb2bdd9b700 1 -- 172.21.15.78:0/4194406843 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b0004160 con 0x7fb2c01099a0 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.649+0000 7fb2bdd9b700 1 -- 172.21.15.78:0/4194406843 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b0004300 con 0x7fb2c01099a0 2024-04-25T13:50:15.390 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.649+0000 7fb2bdd9b700 1 -- 172.21.15.78:0/4194406843 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b0011a20 con 0x7fb2c01099a0 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.650+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4194406843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 msgr2=0x7fb2c0109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.650+0000 7fb2c4cc8700 1 --2- 172.21.15.78:0/4194406843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c0109db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00098a0 tx=0x7fb2b0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.650+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4194406843 shutdown_connections 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.650+0000 7fb2c4cc8700 1 --2- 172.21.15.78:0/4194406843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c0109db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.650+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4194406843 >> 172.21.15.78:0/4194406843 conn(0x7fb2c0105060 msgr2=0x7fb2c01074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.650+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4194406843 shutdown_connections 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.650+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4194406843 wait complete. 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2c4cc8700 1 Processor -- start 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2c4cc8700 1 -- start start 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2c4cc8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2c4cc8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010de80 con 0x7fb2c01099a0 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2be59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.391 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2be59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53560/0 (socket says 172.21.15.78:53560) 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2be59c700 1 -- 172.21.15.78:0/4246057437 learned_addr learned my addr 172.21.15.78:0/4246057437 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.651+0000 7fb2be59c700 1 -- 172.21.15.78:0/4246057437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c019e170 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.652+0000 7fb2be59c700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c019dc30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0003e70 tx=0x7fb2b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.652+0000 7fb2b77fe700 1 -- 172.21.15.78:0/4246057437 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b0003af0 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.652+0000 7fb2b77fe700 1 -- 172.21.15.78:0/4246057437 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b001a8e0 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.652+0000 7fb2b77fe700 1 -- 172.21.15.78:0/4246057437 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b0024870 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.652+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c019e400 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.652+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c01a1010 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.653+0000 7fb2b77fe700 1 -- 172.21.15.78:0/4246057437 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb2b0022020 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.653+0000 7fb2b77fe700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2ac03d730 0x7fb2ac03fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.653+0000 7fb2b77fe700 1 -- 172.21.15.78:0/4246057437 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb2b0053830 con 0x7fb2c01099a0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.653+0000 7fb2b7fff700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2ac03d730 0x7fb2ac03fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.653+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c0064cf0 con 0x7fb2c01099a0 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.653+0000 7fb2b7fff700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2ac03d730 0x7fb2ac03fc40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8006fd0 tx=0x7fb2a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.658+0000 7fb2b77fe700 1 -- 172.21.15.78:0/4246057437 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2b002b300 con 0x7fb2c01099a0 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.854+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fb2c01a1570 con 0x7fb2c01099a0 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.855+0000 7fb2b77fe700 1 -- 172.21.15.78:0/4246057437 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fb2b000e260 con 0x7fb2c01099a0 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2ac03d730 msgr2=0x7fb2ac03fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2ac03d730 0x7fb2ac03fc40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8006fd0 tx=0x7fb2a8006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 msgr2=0x7fb2c019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c019dc30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0003e70 tx=0x7fb2b0011aa0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 shutdown_connections 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2ac03d730 0x7fb2ac03fc40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 --2- 172.21.15.78:0/4246057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2c01099a0 0x7fb2c019dc30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 >> 172.21.15.78:0/4246057437 conn(0x7fb2c0105060 msgr2=0x7fb2c0106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 shutdown_connections 2024-04-25T13:50:15.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:14.857+0000 7fb2c4cc8700 1 -- 172.21.15.78:0/4246057437 wait complete. 2024-04-25T13:50:15.793 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:15 smithi078 ceph-mon[29163]: mgrmap e13: x(active, since 6s) 2024-04-25T13:50:16.739 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.821+0000 7effe768e700 1 Processor -- start 2024-04-25T13:50:16.739 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.822+0000 7effe768e700 1 -- start start 2024-04-25T13:50:16.739 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.822+0000 7effe768e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe0108cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.739 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.822+0000 7effe768e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7effe01092e0 con 0x7effe01088e0 2024-04-25T13:50:16.739 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.822+0000 7effe542a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe0108cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.739 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.822+0000 7effe542a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe0108cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53576/0 (socket says 172.21.15.78:53576) 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.822+0000 7effe542a700 1 -- 172.21.15.78:0/673089580 learned_addr learned my addr 172.21.15.78:0/673089580 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.823+0000 7effe542a700 1 -- 172.21.15.78:0/673089580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe0109a10 con 0x7effe01088e0 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.823+0000 7effe542a700 1 --2- 172.21.15.78:0/673089580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe0108cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7effd00098a0 tx=0x7effd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637cfffcb921e2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.823+0000 7effd7fff700 1 -- 172.21.15.78:0/673089580 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0004160 con 0x7effe01088e0 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.823+0000 7effd7fff700 1 -- 172.21.15.78:0/673089580 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd0004300 con 0x7effe01088e0 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.823+0000 7effd7fff700 1 -- 172.21.15.78:0/673089580 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0011a20 con 0x7effe01088e0 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 -- 172.21.15.78:0/673089580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 msgr2=0x7effe0108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 --2- 172.21.15.78:0/673089580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe0108cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7effd00098a0 tx=0x7effd0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 -- 172.21.15.78:0/673089580 shutdown_connections 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 --2- 172.21.15.78:0/673089580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe0108cf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 -- 172.21.15.78:0/673089580 >> 172.21.15.78:0/673089580 conn(0x7effe0103fa0 msgr2=0x7effe01063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 -- 172.21.15.78:0/673089580 shutdown_connections 2024-04-25T13:50:16.740 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 -- 172.21.15.78:0/673089580 wait complete. 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 Processor -- start 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.824+0000 7effe768e700 1 -- start start 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe768e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe007d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe768e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7effe010cdc0 con 0x7effe01088e0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe542a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe007d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe542a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe007d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53582/0 (socket says 172.21.15.78:53582) 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe542a700 1 -- 172.21.15.78:0/298823231 learned_addr learned my addr 172.21.15.78:0/298823231 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe542a700 1 -- 172.21.15.78:0/298823231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe007d710 con 0x7effe01088e0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe542a700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe007d1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effd0003e70 tx=0x7effd0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effd67fc700 1 -- 172.21.15.78:0/298823231 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0003aa0 con 0x7effe01088e0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effd67fc700 1 -- 172.21.15.78:0/298823231 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd001a8e0 con 0x7effe01088e0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe01a0a30 con 0x7effe01088e0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effd67fc700 1 -- 172.21.15.78:0/298823231 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0024870 con 0x7effe01088e0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.825+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe01a0f10 con 0x7effe01088e0 2024-04-25T13:50:16.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.826+0000 7effd67fc700 1 -- 172.21.15.78:0/298823231 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7effd0022020 con 0x7effe01088e0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.826+0000 7effd67fc700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7effcc03d6c0 0x7effcc03fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.826+0000 7effd67fc700 1 -- 172.21.15.78:0/298823231 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7effd00537b0 con 0x7effe01088e0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.827+0000 7effe4c29700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7effcc03d6c0 0x7effcc03fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.827+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe0064cf0 con 0x7effe01088e0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.827+0000 7effe4c29700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7effcc03d6c0 0x7effcc03fbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effdc006fd0 tx=0x7effdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:15.832+0000 7effd67fc700 1 -- 172.21.15.78:0/298823231 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effd0029df0 con 0x7effe01088e0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.142+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7effe01a12b0 con 0x7effe01088e0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.145+0000 7effd67fc700 1 -- 172.21.15.78:0/298823231 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7effd001aa90 con 0x7effe01088e0 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7effcc03d6c0 msgr2=0x7effcc03fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7effcc03d6c0 0x7effcc03fbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effdc006fd0 tx=0x7effdc006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 msgr2=0x7effe007d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe007d1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effd0003e70 tx=0x7effd0011aa0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 shutdown_connections 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7effcc03d6c0 0x7effcc03fbd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 --2- 172.21.15.78:0/298823231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effe01088e0 0x7effe007d1d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 >> 172.21.15.78:0/298823231 conn(0x7effe0103fa0 msgr2=0x7effe0105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 shutdown_connections 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T13:50:16.147+0000 7effe768e700 1 -- 172.21.15.78:0/298823231 wait complete. 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.743 INFO:teuthology.orchestra.run.smithi078.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout: ceph telemetry on 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout:For more information see: 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:16.744 INFO:teuthology.orchestra.run.smithi078.stdout:Bootstrap complete. 2024-04-25T13:50:16.775 INFO:tasks.cephadm:Fetching config... 2024-04-25T13:50:16.775 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:50:16.775 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T13:50:16.790 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T13:50:16.790 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:50:16.790 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T13:50:16.847 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T13:50:16.847 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:50:16.847 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T13:50:16.910 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T13:50:16.910 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:50:16.910 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T13:50:16.965 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T13:50:16.965 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd3Yzu82ZzBYEC1PE6cL9OWdwfgg+3B2TDbvElxpOot/xYXueUys3GSwO2zVMt7LbbC4vHUH0Vk8WEj7aLjoL9HnT2Ej0LBl6zrwAh4fq38EGzsFomo3U22AA1aMh82p6Sf0W8T2VMYVP4f0r7l/J357ydN7gtTzcHy0ODBzHYDRXsu3ftU1Whzn05XHFZWn8br/fZsWlUXQ6etrGZFd5dMf38YMLWtAVoxAa+K3851kMLuWV34whw8UiQ9nHLwXaFGK4hl4X8RsCtrgBqijjdKn6Vccpc6mbCupT36sxjwlJpVHN+qRKsBdhlAAdFbzjVs91UUw3ot4TeRlEZscxxteOyZzY4xLNbzV2amf+DDLycs3sj8JchGHGuVvvzlYO/7jKJDFuuSQ+HkMHsq++NQLKBnAGSWEaF31wd79pEeoXG+ni3JyfPM7d2+UAa6sbjGs0Q/hEL92saNbASRplUxAQX0+D/FCmWJGmYov5uEexgzexoNZgxITW2T2tdt5M= ceph-5cff6f4a-030a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:50:17.036 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd3Yzu82ZzBYEC1PE6cL9OWdwfgg+3B2TDbvElxpOot/xYXueUys3GSwO2zVMt7LbbC4vHUH0Vk8WEj7aLjoL9HnT2Ej0LBl6zrwAh4fq38EGzsFomo3U22AA1aMh82p6Sf0W8T2VMYVP4f0r7l/J357ydN7gtTzcHy0ODBzHYDRXsu3ftU1Whzn05XHFZWn8br/fZsWlUXQ6etrGZFd5dMf38YMLWtAVoxAa+K3851kMLuWV34whw8UiQ9nHLwXaFGK4hl4X8RsCtrgBqijjdKn6Vccpc6mbCupT36sxjwlJpVHN+qRKsBdhlAAdFbzjVs91UUw3ot4TeRlEZscxxteOyZzY4xLNbzV2amf+DDLycs3sj8JchGHGuVvvzlYO/7jKJDFuuSQ+HkMHsq++NQLKBnAGSWEaF31wd79pEeoXG+ni3JyfPM7d2+UAa6sbjGs0Q/hEL92saNbASRplUxAQX0+D/FCmWJGmYov5uEexgzexoNZgxITW2T2tdt5M= ceph-5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:50:17.047 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd3Yzu82ZzBYEC1PE6cL9OWdwfgg+3B2TDbvElxpOot/xYXueUys3GSwO2zVMt7LbbC4vHUH0Vk8WEj7aLjoL9HnT2Ej0LBl6zrwAh4fq38EGzsFomo3U22AA1aMh82p6Sf0W8T2VMYVP4f0r7l/J357ydN7gtTzcHy0ODBzHYDRXsu3ftU1Whzn05XHFZWn8br/fZsWlUXQ6etrGZFd5dMf38YMLWtAVoxAa+K3851kMLuWV34whw8UiQ9nHLwXaFGK4hl4X8RsCtrgBqijjdKn6Vccpc6mbCupT36sxjwlJpVHN+qRKsBdhlAAdFbzjVs91UUw3ot4TeRlEZscxxteOyZzY4xLNbzV2amf+DDLycs3sj8JchGHGuVvvzlYO/7jKJDFuuSQ+HkMHsq++NQLKBnAGSWEaF31wd79pEeoXG+ni3JyfPM7d2+UAa6sbjGs0Q/hEL92saNbASRplUxAQX0+D/FCmWJGmYov5uEexgzexoNZgxITW2T2tdt5M= ceph-5cff6f4a-030a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:50:17.091 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd3Yzu82ZzBYEC1PE6cL9OWdwfgg+3B2TDbvElxpOot/xYXueUys3GSwO2zVMt7LbbC4vHUH0Vk8WEj7aLjoL9HnT2Ej0LBl6zrwAh4fq38EGzsFomo3U22AA1aMh82p6Sf0W8T2VMYVP4f0r7l/J357ydN7gtTzcHy0ODBzHYDRXsu3ftU1Whzn05XHFZWn8br/fZsWlUXQ6etrGZFd5dMf38YMLWtAVoxAa+K3851kMLuWV34whw8UiQ9nHLwXaFGK4hl4X8RsCtrgBqijjdKn6Vccpc6mbCupT36sxjwlJpVHN+qRKsBdhlAAdFbzjVs91UUw3ot4TeRlEZscxxteOyZzY4xLNbzV2amf+DDLycs3sj8JchGHGuVvvzlYO/7jKJDFuuSQ+HkMHsq++NQLKBnAGSWEaF31wd79pEeoXG+ni3JyfPM7d2+UAa6sbjGs0Q/hEL92saNbASRplUxAQX0+D/FCmWJGmYov5uEexgzexoNZgxITW2T2tdt5M= ceph-5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:50:17.107 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd3Yzu82ZzBYEC1PE6cL9OWdwfgg+3B2TDbvElxpOot/xYXueUys3GSwO2zVMt7LbbC4vHUH0Vk8WEj7aLjoL9HnT2Ej0LBl6zrwAh4fq38EGzsFomo3U22AA1aMh82p6Sf0W8T2VMYVP4f0r7l/J357ydN7gtTzcHy0ODBzHYDRXsu3ftU1Whzn05XHFZWn8br/fZsWlUXQ6etrGZFd5dMf38YMLWtAVoxAa+K3851kMLuWV34whw8UiQ9nHLwXaFGK4hl4X8RsCtrgBqijjdKn6Vccpc6mbCupT36sxjwlJpVHN+qRKsBdhlAAdFbzjVs91UUw3ot4TeRlEZscxxteOyZzY4xLNbzV2amf+DDLycs3sj8JchGHGuVvvzlYO/7jKJDFuuSQ+HkMHsq++NQLKBnAGSWEaF31wd79pEeoXG+ni3JyfPM7d2+UAa6sbjGs0Q/hEL92saNbASRplUxAQX0+D/FCmWJGmYov5uEexgzexoNZgxITW2T2tdt5M= ceph-5cff6f4a-030a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:50:17.148 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd3Yzu82ZzBYEC1PE6cL9OWdwfgg+3B2TDbvElxpOot/xYXueUys3GSwO2zVMt7LbbC4vHUH0Vk8WEj7aLjoL9HnT2Ej0LBl6zrwAh4fq38EGzsFomo3U22AA1aMh82p6Sf0W8T2VMYVP4f0r7l/J357ydN7gtTzcHy0ODBzHYDRXsu3ftU1Whzn05XHFZWn8br/fZsWlUXQ6etrGZFd5dMf38YMLWtAVoxAa+K3851kMLuWV34whw8UiQ9nHLwXaFGK4hl4X8RsCtrgBqijjdKn6Vccpc6mbCupT36sxjwlJpVHN+qRKsBdhlAAdFbzjVs91UUw3ot4TeRlEZscxxteOyZzY4xLNbzV2amf+DDLycs3sj8JchGHGuVvvzlYO/7jKJDFuuSQ+HkMHsq++NQLKBnAGSWEaF31wd79pEeoXG+ni3JyfPM7d2+UAa6sbjGs0Q/hEL92saNbASRplUxAQX0+D/FCmWJGmYov5uEexgzexoNZgxITW2T2tdt5M= ceph-5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:50:17.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T13:50:17.394 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:50:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:17 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/298823231' entity='client.admin' 2024-04-25T13:50:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.027+0000 7f9d4acf6700 1 -- 172.21.15.78:0/2419893855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 msgr2=0x7f9d44102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.027+0000 7f9d4acf6700 1 --2- 172.21.15.78:0/2419893855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44102df0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009910 tx=0x7f9d38009c20 comp rx=0 tx=0).stop 2024-04-25T13:50:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.027+0000 7f9d4acf6700 1 -- 172.21.15.78:0/2419893855 shutdown_connections 2024-04-25T13:50:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.027+0000 7f9d4acf6700 1 --2- 172.21.15.78:0/2419893855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44102df0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.027+0000 7f9d4acf6700 1 -- 172.21.15.78:0/2419893855 >> 172.21.15.78:0/2419893855 conn(0x7f9d440fe000 msgr2=0x7f9d441004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.027+0000 7f9d4acf6700 1 -- 172.21.15.78:0/2419893855 shutdown_connections 2024-04-25T13:50:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.027+0000 7f9d4acf6700 1 -- 172.21.15.78:0/2419893855 wait complete. 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.028+0000 7f9d4acf6700 1 Processor -- start 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.028+0000 7f9d4acf6700 1 -- start start 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d4acf6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d4acf6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441089a0 con 0x7f9d441029e0 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d48a92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d48a92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53626/0 (socket says 172.21.15.78:53626) 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d48a92700 1 -- 172.21.15.78:0/770852199 learned_addr learned my addr 172.21.15.78:0/770852199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d48a92700 1 -- 172.21.15.78:0/770852199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d44198a60 con 0x7f9d441029e0 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d48a92700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44198520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9d380057d0 tx=0x7f9d38005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d41ffb700 1 -- 172.21.15.78:0/770852199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d38027070 con 0x7f9d441029e0 2024-04-25T13:50:19.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d44198cf0 con 0x7f9d441029e0 2024-04-25T13:50:19.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d41ffb700 1 -- 172.21.15.78:0/770852199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d3800be10 con 0x7f9d441029e0 2024-04-25T13:50:19.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.029+0000 7f9d41ffb700 1 -- 172.21.15.78:0/770852199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d38022dd0 con 0x7f9d441029e0 2024-04-25T13:50:19.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.030+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4419b900 con 0x7f9d441029e0 2024-04-25T13:50:19.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.030+0000 7f9d41ffb700 1 -- 172.21.15.78:0/770852199 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f9d3802a070 con 0x7f9d441029e0 2024-04-25T13:50:19.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.031+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d44109120 con 0x7f9d441029e0 2024-04-25T13:50:19.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.031+0000 7f9d41ffb700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9d3403d5c0 0x7f9d3403fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.031+0000 7f9d41ffb700 1 -- 172.21.15.78:0/770852199 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9d3801e030 con 0x7f9d441029e0 2024-04-25T13:50:19.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.031+0000 7f9d43fff700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9d3403d5c0 0x7f9d3403fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.031+0000 7f9d43fff700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9d3403d5c0 0x7f9d3403fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d30006fd0 tx=0x7f9d30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:19.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.036+0000 7f9d41ffb700 1 -- 172.21.15.78:0/770852199 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d38013070 con 0x7f9d441029e0 2024-04-25T13:50:19.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.264+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9d4402e620 con 0x7f9d441029e0 2024-04-25T13:50:19.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.268+0000 7f9d41ffb700 1 -- 172.21.15.78:0/770852199 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9d380292c0 con 0x7f9d441029e0 2024-04-25T13:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9d3403d5c0 msgr2=0x7f9d3403fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9d3403d5c0 0x7f9d3403fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d30006fd0 tx=0x7f9d30006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 msgr2=0x7f9d44198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44198520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9d380057d0 tx=0x7f9d38005800 comp rx=0 tx=0).stop 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 shutdown_connections 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9d3403d5c0 0x7f9d3403fad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 --2- 172.21.15.78:0/770852199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d441029e0 0x7f9d44198520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.270+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 >> 172.21.15.78:0/770852199 conn(0x7f9d440fe000 msgr2=0x7f9d440fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.271+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 shutdown_connections 2024-04-25T13:50:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:19.271+0000 7f9d4acf6700 1 -- 172.21.15.78:0/770852199 wait complete. 2024-04-25T13:50:20.012 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T13:50:20.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T13:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:50:20.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:20 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/770852199' entity='client.admin' 2024-04-25T13:50:22.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 -- 172.21.15.78:0/3276220473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 msgr2=0x7fb33c102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 --2- 172.21.15.78:0/3276220473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c102530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009910 tx=0x7fb32c009c20 comp rx=0 tx=0).stop 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 -- 172.21.15.78:0/3276220473 shutdown_connections 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 --2- 172.21.15.78:0/3276220473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c102530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 -- 172.21.15.78:0/3276220473 >> 172.21.15.78:0/3276220473 conn(0x7fb33c0fd780 msgr2=0x7fb33c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 -- 172.21.15.78:0/3276220473 shutdown_connections 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 -- 172.21.15.78:0/3276220473 wait complete. 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.584+0000 7fb3410c4700 1 Processor -- start 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb3410c4700 1 -- start start 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb3410c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb3410c4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1080e0 con 0x7fb33c102120 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb33ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb33ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53642/0 (socket says 172.21.15.78:53642) 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb33ad9d700 1 -- 172.21.15.78:0/1704869884 learned_addr learned my addr 172.21.15.78:0/1704869884 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb33ad9d700 1 -- 172.21.15.78:0/1704869884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c199b80 con 0x7fb33c102120 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.585+0000 7fb33ad9d700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c199640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0057d0 tx=0x7fb32c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.586+0000 7fb323fff700 1 -- 172.21.15.78:0/1704869884 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb32c027070 con 0x7fb33c102120 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.586+0000 7fb323fff700 1 -- 172.21.15.78:0/1704869884 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c00be10 con 0x7fb33c102120 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.586+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c199e10 con 0x7fb33c102120 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.586+0000 7fb323fff700 1 -- 172.21.15.78:0/1704869884 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb32c022dd0 con 0x7fb33c102120 2024-04-25T13:50:22.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.586+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c19ca20 con 0x7fb33c102120 2024-04-25T13:50:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.587+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33c192850 con 0x7fb33c102120 2024-04-25T13:50:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.587+0000 7fb323fff700 1 -- 172.21.15.78:0/1704869884 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb32c02a070 con 0x7fb33c102120 2024-04-25T13:50:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.587+0000 7fb323fff700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb32403d610 0x7fb32403fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.587+0000 7fb323fff700 1 -- 172.21.15.78:0/1704869884 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb32c01e030 con 0x7fb33c102120 2024-04-25T13:50:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.587+0000 7fb33a59c700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb32403d610 0x7fb32403fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.587+0000 7fb33a59c700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb32403d610 0x7fb32403fb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb330006fd0 tx=0x7fb330006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.592+0000 7fb323fff700 1 -- 172.21.15.78:0/1704869884 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb32c013070 con 0x7fb33c102120 2024-04-25T13:50:22.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.792+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb33c02e620 con 0x7fb32403d610 2024-04-25T13:50:22.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.798+0000 7fb323fff700 1 -- 172.21.15.78:0/1704869884 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb33c02e620 con 0x7fb32403d610 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb32403d610 msgr2=0x7fb32403fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb32403d610 0x7fb32403fb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb330006fd0 tx=0x7fb330006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 msgr2=0x7fb33c199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c199640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0057d0 tx=0x7fb32c005800 comp rx=0 tx=0).stop 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 shutdown_connections 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb32403d610 0x7fb32403fb20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 --2- 172.21.15.78:0/1704869884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb33c102120 0x7fb33c199640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 >> 172.21.15.78:0/1704869884 conn(0x7fb33c0fd780 msgr2=0x7fb33c18f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 shutdown_connections 2024-04-25T13:50:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:22.800+0000 7fb3410c4700 1 -- 172.21.15.78:0/1704869884 wait complete. 2024-04-25T13:50:23.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:23.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:23.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:50:23.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:23.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:23.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:23.545 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi112 2024-04-25T13:50:23.545 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:50:23.545 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:50:23.564 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:50:23.564 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:23.621 INFO:tasks.cephadm:Adding host smithi112 to orchestrator... 2024-04-25T13:50:23.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch host add smithi112 2024-04-25T13:50:23.857 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:50:24.052 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:24.052 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:24.052 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:24.052 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:24.052 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:24.052 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T13:50:24.053 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: Updating smithi078:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:50:24.053 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:24.053 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:24.053 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:25.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:24 smithi078 ceph-mon[29163]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:25.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:24 smithi078 ceph-mon[29163]: Updating smithi078:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:50:25.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.246+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/148384342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 msgr2=0x7f1bbc075e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:25.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.246+0000 7f1bc3bc5700 1 --2- 172.21.15.78:0/148384342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc075e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8009910 tx=0x7f1bb8009c20 comp rx=0 tx=0).stop 2024-04-25T13:50:25.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.246+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/148384342 shutdown_connections 2024-04-25T13:50:25.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.246+0000 7f1bc3bc5700 1 --2- 172.21.15.78:0/148384342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc075e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:25.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.246+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/148384342 >> 172.21.15.78:0/148384342 conn(0x7f1bbc0ff560 msgr2=0x7f1bbc1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:25.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.246+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/148384342 shutdown_connections 2024-04-25T13:50:25.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.246+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/148384342 wait complete. 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.247+0000 7f1bc3bc5700 1 Processor -- start 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.247+0000 7f1bc3bc5700 1 -- start start 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.247+0000 7f1bc3bc5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.247+0000 7f1bc3bc5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc1145d0 con 0x7f1bbc075a30 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.247+0000 7f1bc1961700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.247+0000 7f1bc1961700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc19fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53662/0 (socket says 172.21.15.78:53662) 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.247+0000 7f1bc1961700 1 -- 172.21.15.78:0/73867767 learned_addr learned my addr 172.21.15.78:0/73867767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.248+0000 7f1bc1961700 1 -- 172.21.15.78:0/73867767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc1a0390 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.248+0000 7f1bc1961700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc19fe50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80056d0 tx=0x7f1bb80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.248+0000 7f1bb2ffd700 1 -- 172.21.15.78:0/73867767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bb8026070 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.248+0000 7f1bb2ffd700 1 -- 172.21.15.78:0/73867767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bb8004370 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.248+0000 7f1bb2ffd700 1 -- 172.21.15.78:0/73867767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bb80039b0 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.248+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc1a0620 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.248+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc1a0a80 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.249+0000 7f1bb2ffd700 1 -- 172.21.15.78:0/73867767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f1bb8029070 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.249+0000 7f1bb2ffd700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ba803d610 0x7f1ba803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.249+0000 7f1bb2ffd700 1 -- 172.21.15.78:0/73867767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1bb801e070 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.249+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc077170 con 0x7f1bbc075a30 2024-04-25T13:50:25.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.249+0000 7f1bc1160700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ba803d610 0x7f1ba803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.250+0000 7f1bc1160700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ba803d610 0x7f1ba803fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006fd0 tx=0x7f1bac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:25.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.255+0000 7f1bb2ffd700 1 -- 172.21.15.78:0/73867767 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bb8013070 con 0x7f1bbc075a30 2024-04-25T13:50:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:25.455+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}) -- 0x7f1bbc1a11b0 con 0x7f1ba803d610 2024-04-25T13:50:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:26 smithi078 ceph-mon[29163]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:26 smithi078 ceph-mon[29163]: Deploying cephadm binary to smithi112 2024-04-25T13:50:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:29 smithi078 ceph-mon[29163]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:29.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.511+0000 7f1bb2ffd700 1 -- 172.21.15.78:0/73867767 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1bbc1a11b0 con 0x7f1ba803d610 2024-04-25T13:50:29.511 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi112' with addr '172.21.15.112' 2024-04-25T13:50:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ba803d610 msgr2=0x7f1ba803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ba803d610 0x7f1ba803fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1bac006fd0 tx=0x7f1bac006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 msgr2=0x7f1bbc19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc19fe50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80056d0 tx=0x7f1bb80057b0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 shutdown_connections 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ba803d610 0x7f1ba803fb20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 --2- 172.21.15.78:0/73867767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bbc075a30 0x7f1bbc19fe50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.513+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 >> 172.21.15.78:0/73867767 conn(0x7f1bbc0ff560 msgr2=0x7f1bbc101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.514+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 shutdown_connections 2024-04-25T13:50:29.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:29.514+0000 7f1bc3bc5700 1 -- 172.21.15.78:0/73867767 wait complete. 2024-04-25T13:50:30.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:50:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:50:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:30 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:30 smithi078 ceph-mon[29163]: Added host smithi112 2024-04-25T13:50:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:30 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:30 smithi078 ceph-mon[29163]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:30 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.905+0000 7f815da99700 1 -- 172.21.15.78:0/4109213098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 msgr2=0x7f8158108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.905+0000 7f815da99700 1 --2- 172.21.15.78:0/4109213098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f8158108d50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f814800b0a0 tx=0x7f814800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.905+0000 7f815da99700 1 -- 172.21.15.78:0/4109213098 shutdown_connections 2024-04-25T13:50:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.905+0000 7f815da99700 1 --2- 172.21.15.78:0/4109213098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f8158108d50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.905+0000 7f815da99700 1 -- 172.21.15.78:0/4109213098 >> 172.21.15.78:0/4109213098 conn(0x7f8158078460 msgr2=0x7f81580788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.905+0000 7f815da99700 1 -- 172.21.15.78:0/4109213098 shutdown_connections 2024-04-25T13:50:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.905+0000 7f815da99700 1 -- 172.21.15.78:0/4109213098 wait complete. 2024-04-25T13:50:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.906+0000 7f815da99700 1 Processor -- start 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.906+0000 7f815da99700 1 -- start start 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.906+0000 7f815da99700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f815819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.906+0000 7f815da99700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f815810ce90 con 0x7f8158108940 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f81577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f815819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f81577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f815819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43414/0 (socket says 172.21.15.78:43414) 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f81577fe700 1 -- 172.21.15.78:0/3421163485 learned_addr learned my addr 172.21.15.78:0/3421163485 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f81577fe700 1 -- 172.21.15.78:0/3421163485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815806bbd0 con 0x7f8158108940 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f81577fe700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f815819a5c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8148015040 tx=0x7f8148009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f8154ff9700 1 -- 172.21.15.78:0/3421163485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8148021020 con 0x7f8158108940 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815819abc0 con 0x7f8158108940 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f8154ff9700 1 -- 172.21.15.78:0/3421163485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8148007970 con 0x7f8158108940 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f8154ff9700 1 -- 172.21.15.78:0/3421163485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8148017040 con 0x7f8158108940 2024-04-25T13:50:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.907+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815819b080 con 0x7f8158108940 2024-04-25T13:50:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.908+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8158104050 con 0x7f8158108940 2024-04-25T13:50:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.908+0000 7f8154ff9700 1 -- 172.21.15.78:0/3421163485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f8148024020 con 0x7f8158108940 2024-04-25T13:50:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.909+0000 7f8154ff9700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f814003d610 0x7f814003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.909+0000 7f8154ff9700 1 -- 172.21.15.78:0/3421163485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f814800f050 con 0x7f8158108940 2024-04-25T13:50:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.909+0000 7f8156ffd700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f814003d610 0x7f814003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.909+0000 7f8156ffd700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f814003d610 0x7f814003fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f814c006fd0 tx=0x7f814c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:31.914+0000 7f8154ff9700 1 -- 172.21.15.78:0/3421163485 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f814802b020 con 0x7f8158108940 2024-04-25T13:50:32.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.116+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f815806b320 con 0x7f814003d610 2024-04-25T13:50:32.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.118+0000 7f8154ff9700 1 -- 172.21.15.78:0/3421163485 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f815806b320 con 0x7f814003d610 2024-04-25T13:50:32.117 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:32.118 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}] 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f814003d610 msgr2=0x7f814003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f814003d610 0x7f814003fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f814c006fd0 tx=0x7f814c006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 msgr2=0x7f815819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f815819a5c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8148015040 tx=0x7f8148009790 comp rx=0 tx=0).stop 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 shutdown_connections 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f814003d610 0x7f814003fb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 --2- 172.21.15.78:0/3421163485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8158108940 0x7f815819a5c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 >> 172.21.15.78:0/3421163485 conn(0x7f8158078460 msgr2=0x7f8158100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 shutdown_connections 2024-04-25T13:50:32.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:32.120+0000 7f815da99700 1 -- 172.21.15.78:0/3421163485 wait complete. 2024-04-25T13:50:32.785 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi129 2024-04-25T13:50:32.786 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:50:32.786 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:50:32.804 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:50:32.804 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:32.861 INFO:tasks.cephadm:Adding host smithi129 to orchestrator... 2024-04-25T13:50:32.862 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch host add smithi129 2024-04-25T13:50:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:50:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:33 smithi078 ceph-mon[29163]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:33 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:33.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:33 smithi078 ceph-mon[29163]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:50:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 -- 172.21.15.78:0/4058966487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 msgr2=0x7fb928109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 --2- 172.21.15.78:0/4058966487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928109e50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009910 tx=0x7fb91c009c20 comp rx=0 tx=0).stop 2024-04-25T13:50:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 -- 172.21.15.78:0/4058966487 shutdown_connections 2024-04-25T13:50:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 --2- 172.21.15.78:0/4058966487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928109e50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 -- 172.21.15.78:0/4058966487 >> 172.21.15.78:0/4058966487 conn(0x7fb9280ff530 msgr2=0x7fb928101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 -- 172.21.15.78:0/4058966487 shutdown_connections 2024-04-25T13:50:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 -- 172.21.15.78:0/4058966487 wait complete. 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 Processor -- start 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.495+0000 7fb92ed86700 1 -- start start 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92ed86700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92ed86700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb92810df90 con 0x7fb928109a40 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92cb22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92cb22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43442/0 (socket says 172.21.15.78:43442) 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92cb22700 1 -- 172.21.15.78:0/4030437265 learned_addr learned my addr 172.21.15.78:0/4030437265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92cb22700 1 -- 172.21.15.78:0/4030437265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9280777e0 con 0x7fb928109a40 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92cb22700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928075990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0056d0 tx=0x7fb91c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb925ffb700 1 -- 172.21.15.78:0/4030437265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c027070 con 0x7fb928109a40 2024-04-25T13:50:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb925ffb700 1 -- 172.21.15.78:0/4030437265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91c00be10 con 0x7fb928109a40 2024-04-25T13:50:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb928075ed0 con 0x7fb928109a40 2024-04-25T13:50:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb925ffb700 1 -- 172.21.15.78:0/4030437265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c022d20 con 0x7fb928109a40 2024-04-25T13:50:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.496+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb928076390 con 0x7fb928109a40 2024-04-25T13:50:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.497+0000 7fb925ffb700 1 -- 172.21.15.78:0/4030437265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb91c02a070 con 0x7fb928109a40 2024-04-25T13:50:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.498+0000 7fb925ffb700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb91803d610 0x7fb91803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.498+0000 7fb925ffb700 1 -- 172.21.15.78:0/4030437265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb91c01e030 con 0x7fb928109a40 2024-04-25T13:50:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.498+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb928105150 con 0x7fb928109a40 2024-04-25T13:50:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.498+0000 7fb927fff700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb91803d610 0x7fb91803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.498+0000 7fb927fff700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb91803d610 0x7fb91803fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb914006fd0 tx=0x7fb914006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.503+0000 7fb925ffb700 1 -- 172.21.15.78:0/4030437265 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb91c031020 con 0x7fb928109a40 2024-04-25T13:50:34.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:34.702+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}) -- 0x7fb928076ec0 con 0x7fb91803d610 2024-04-25T13:50:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:35 smithi078 ceph-mon[29163]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:37 smithi078 ceph-mon[29163]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T13:50:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:37 smithi078 ceph-mon[29163]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:50:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:37 smithi078 ceph-mon[29163]: Deploying cephadm binary to smithi129 2024-04-25T13:50:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:37 smithi078 ceph-mon[29163]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:37 smithi078 ceph-mon[29163]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:50:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:37 smithi078 ceph-mon[29163]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:38.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.922+0000 7fb925ffb700 1 -- 172.21.15.78:0/4030437265 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb928076ec0 con 0x7fb91803d610 2024-04-25T13:50:38.923 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi129' with addr '172.21.15.129' 2024-04-25T13:50:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb91803d610 msgr2=0x7fb91803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb91803d610 0x7fb91803fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb914006fd0 tx=0x7fb914006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 msgr2=0x7fb928075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928075990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0056d0 tx=0x7fb91c0057b0 comp rx=0 tx=0).stop 2024-04-25T13:50:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 shutdown_connections 2024-04-25T13:50:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb91803d610 0x7fb91803fb20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 --2- 172.21.15.78:0/4030437265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb928109a40 0x7fb928075990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 >> 172.21.15.78:0/4030437265 conn(0x7fb9280ff530 msgr2=0x7fb928100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 shutdown_connections 2024-04-25T13:50:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:38.925+0000 7fb92ed86700 1 -- 172.21.15.78:0/4030437265 wait complete. 2024-04-25T13:50:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:39 smithi078 ceph-mon[29163]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:39.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:50:39.843 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:50:40.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:40 smithi078 ceph-mon[29163]: Added host smithi129 2024-04-25T13:50:40.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:40.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:41.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:41 smithi078 ceph-mon[29163]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.432+0000 7f59c8fdc700 1 -- 172.21.15.78:0/3350143349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 msgr2=0x7f59c4103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.432+0000 7f59c8fdc700 1 --2- 172.21.15.78:0/3350143349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c4103730 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009910 tx=0x7f59b4009c20 comp rx=0 tx=0).stop 2024-04-25T13:50:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.432+0000 7f59c8fdc700 1 -- 172.21.15.78:0/3350143349 shutdown_connections 2024-04-25T13:50:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.432+0000 7f59c8fdc700 1 --2- 172.21.15.78:0/3350143349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c4103730 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.432+0000 7f59c8fdc700 1 -- 172.21.15.78:0/3350143349 >> 172.21.15.78:0/3350143349 conn(0x7f59c40fdd00 msgr2=0x7f59c4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.432+0000 7f59c8fdc700 1 -- 172.21.15.78:0/3350143349 shutdown_connections 2024-04-25T13:50:41.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.432+0000 7f59c8fdc700 1 -- 172.21.15.78:0/3350143349 wait complete. 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c8fdc700 1 Processor -- start 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c8fdc700 1 -- start start 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c8fdc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c8fdc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59c410cf20 con 0x7f59c4103320 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40308/0 (socket says 172.21.15.78:40308) 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c259c700 1 -- 172.21.15.78:0/138812121 learned_addr learned my addr 172.21.15.78:0/138812121 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c259c700 1 -- 172.21.15.78:0/138812121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c4102e90 con 0x7f59c4103320 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.433+0000 7f59c259c700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c419a6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59b40056d0 tx=0x7f59b40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:41.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.434+0000 7f59b37fe700 1 -- 172.21.15.78:0/138812121 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b4026070 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.434+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c419acc0 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.434+0000 7f59b37fe700 1 -- 172.21.15.78:0/138812121 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b400bc00 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.434+0000 7f59b37fe700 1 -- 172.21.15.78:0/138812121 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b4003920 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.434+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c419b180 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.435+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c419b4c0 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.435+0000 7f59b37fe700 1 -- 172.21.15.78:0/138812121 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f59b4029070 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.435+0000 7f59b37fe700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f59ac03d610 0x7f59ac03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.435+0000 7f59b37fe700 1 -- 172.21.15.78:0/138812121 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59b401e070 con 0x7f59c4103320 2024-04-25T13:50:41.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.435+0000 7f59c1d9b700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f59ac03d610 0x7f59ac03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:41.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.435+0000 7f59c1d9b700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f59ac03d610 0x7f59ac03fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.440+0000 7f59b37fe700 1 -- 172.21.15.78:0/138812121 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59b4013070 con 0x7f59c4103320 2024-04-25T13:50:41.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.646+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59c4064cf0 con 0x7f59ac03d610 2024-04-25T13:50:41.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.648+0000 7f59b37fe700 1 -- 172.21.15.78:0/138812121 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f59c4064cf0 con 0x7f59ac03d610 2024-04-25T13:50:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:50:41.647 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}] 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f59ac03d610 msgr2=0x7f59ac03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f59ac03d610 0x7f59ac03fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 msgr2=0x7f59c419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c419a6c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59b40056d0 tx=0x7f59b40057b0 comp rx=0 tx=0).stop 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 shutdown_connections 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f59ac03d610 0x7f59ac03fb20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 --2- 172.21.15.78:0/138812121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59c4103320 0x7f59c419a6c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 >> 172.21.15.78:0/138812121 conn(0x7f59c40fdd00 msgr2=0x7f59c4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 shutdown_connections 2024-04-25T13:50:41.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:41.650+0000 7f59c8fdc700 1 -- 172.21.15.78:0/138812121 wait complete. 2024-04-25T13:50:42.276 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T13:50:42.277 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T13:50:42.509 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:50:42.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:42 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:42.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:42 smithi078 ceph-mon[29163]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:50:42.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:42 smithi078 ceph-mon[29163]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.944+0000 7f8b8750e700 1 -- 172.21.15.78:0/1187344758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 msgr2=0x7f8b80102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.944+0000 7f8b8750e700 1 --2- 172.21.15.78:0/1187344758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b80102710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009910 tx=0x7f8b7c009c20 comp rx=0 tx=0).stop 2024-04-25T13:50:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.944+0000 7f8b8750e700 1 -- 172.21.15.78:0/1187344758 shutdown_connections 2024-04-25T13:50:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.944+0000 7f8b8750e700 1 --2- 172.21.15.78:0/1187344758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b80102710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.944+0000 7f8b8750e700 1 -- 172.21.15.78:0/1187344758 >> 172.21.15.78:0/1187344758 conn(0x7f8b80077c10 msgr2=0x7f8b80078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.944+0000 7f8b8750e700 1 -- 172.21.15.78:0/1187344758 shutdown_connections 2024-04-25T13:50:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.944+0000 7f8b8750e700 1 -- 172.21.15.78:0/1187344758 wait complete. 2024-04-25T13:50:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b8750e700 1 Processor -- start 2024-04-25T13:50:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b8750e700 1 -- start start 2024-04-25T13:50:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b8750e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b8019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b8750e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8010c5f0 con 0x7f8b80102300 2024-04-25T13:50:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b852aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b8019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b852aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b8019f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40328/0 (socket says 172.21.15.78:40328) 2024-04-25T13:50:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b852aa700 1 -- 172.21.15.78:0/3242584639 learned_addr learned my addr 172.21.15.78:0/3242584639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:50:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b852aa700 1 -- 172.21.15.78:0/3242584639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8019fa00 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b852aa700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b8019f4c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006010 tx=0x7f8b7c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.945+0000 7f8b767fc700 1 -- 172.21.15.78:0/3242584639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b7c026070 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.946+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8019fcf0 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.946+0000 7f8b767fc700 1 -- 172.21.15.78:0/3242584639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b7c00bba0 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.946+0000 7f8b767fc700 1 -- 172.21.15.78:0/3242584639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b7c003910 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.946+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b80199820 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.946+0000 7f8b767fc700 1 -- 172.21.15.78:0/3242584639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f8b7c029070 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.946+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b801037b0 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.947+0000 7f8b767fc700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b6c03d610 0x7f8b6c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.947+0000 7f8b767fc700 1 -- 172.21.15.78:0/3242584639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8b7c01e070 con 0x7f8b80102300 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.947+0000 7f8b84aa9700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b6c03d610 0x7f8b6c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.947+0000 7f8b84aa9700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b6c03d610 0x7f8b6c03fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006fd0 tx=0x7f8b70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:43.952+0000 7f8b767fc700 1 -- 172.21.15.78:0/3242584639 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b7c013070 con 0x7f8b80102300 2024-04-25T13:50:44.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:44.157+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8b8019ff20 con 0x7f8b80102300 2024-04-25T13:50:45.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.099+0000 7f8b767fc700 1 -- 172.21.15.78:0/3242584639 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8b7c015350 con 0x7f8b80102300 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b6c03d610 msgr2=0x7f8b6c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b6c03d610 0x7f8b6c03fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006fd0 tx=0x7f8b70006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 msgr2=0x7f8b8019f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b8019f4c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006010 tx=0x7f8b7c005780 comp rx=0 tx=0).stop 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 shutdown_connections 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b6c03d610 0x7f8b6c03fb20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 --2- 172.21.15.78:0/3242584639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b80102300 0x7f8b8019f4c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:45.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 >> 172.21.15.78:0/3242584639 conn(0x7f8b80077c10 msgr2=0x7f8b8010a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:45.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 shutdown_connections 2024-04-25T13:50:45.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:50:45.101+0000 7f8b8750e700 1 -- 172.21.15.78:0/3242584639 wait complete. 2024-04-25T13:50:45.102 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3242584639' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:45.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:45.762 INFO:tasks.cephadm:Adding mon.a on smithi078 2024-04-25T13:50:45.762 INFO:tasks.cephadm:Adding mon.b on smithi112 2024-04-25T13:50:45.762 INFO:tasks.cephadm:Adding mon.c on smithi129 2024-04-25T13:50:45.762 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c' 2024-04-25T13:50:45.994 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:50:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:46 smithi078 ceph-mon[29163]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-25T13:50:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:46 smithi078 ceph-mon[29163]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:50:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:46 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3242584639' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:50:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:46 smithi078 ceph-mon[29163]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:50:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:47.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.342+0000 7f1a7acc8700 1 -- 172.21.15.129:0/2578132317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 msgr2=0x7f1a74102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:47.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.342+0000 7f1a7acc8700 1 --2- 172.21.15.129:0/2578132317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74102730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000b0a0 tx=0x7f1a7000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:50:47.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 -- 172.21.15.129:0/2578132317 shutdown_connections 2024-04-25T13:50:47.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 --2- 172.21.15.129:0/2578132317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74102730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:47.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 -- 172.21.15.129:0/2578132317 >> 172.21.15.129:0/2578132317 conn(0x7f1a740fdd00 msgr2=0x7f1a74100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:47.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 -- 172.21.15.129:0/2578132317 shutdown_connections 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 -- 172.21.15.129:0/2578132317 wait complete. 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 Processor -- start 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 -- start start 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74107f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.343+0000 7f1a7acc8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7410c610 con 0x7f1a74102320 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a78a64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a78a64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.129:44400/0 (socket says 172.21.15.129:44400) 2024-04-25T13:50:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a78a64700 1 -- 172.21.15.129:0/3729221311 learned_addr learned my addr 172.21.15.129:0/3729221311 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a78a64700 1 -- 172.21.15.129:0/3729221311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a74109d70 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a78a64700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74107f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1a70006130 tx=0x7f1a700097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a69ffb700 1 -- 172.21.15.129:0/3729221311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a70021020 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a69ffb700 1 -- 172.21.15.129:0/3729221311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a700079b0 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.344+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a74108460 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.345+0000 7f1a69ffb700 1 -- 172.21.15.129:0/3729221311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a70017040 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.345+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a741088c0 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.346+0000 7f1a69ffb700 1 -- 172.21.15.129:0/3729221311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f1a70024020 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.346+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a741037d0 con 0x7f1a74102320 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.346+0000 7f1a69ffb700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a5c03d610 0x7f1a5c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:47.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.346+0000 7f1a69ffb700 1 -- 172.21.15.129:0/3729221311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1a7000f050 con 0x7f1a74102320 2024-04-25T13:50:47.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.346+0000 7f1a6bfff700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a5c03d610 0x7f1a5c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:47.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.347+0000 7f1a6bfff700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a5c03d610 0x7f1a5c03fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006fd0 tx=0x7f1a64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.352+0000 7f1a69ffb700 1 -- 172.21.15.129:0/3729221311 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a7002b020 con 0x7f1a74102320 2024-04-25T13:50:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:47 smithi078 ceph-mon[29163]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:47 smithi078 ceph-mon[29163]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:50:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:47 smithi078 ceph-mon[29163]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:47.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.560+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}) -- 0x7f1a741094f0 con 0x7f1a5c03d610 2024-04-25T13:50:47.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.568+0000 7f1a69ffb700 1 -- 172.21.15.129:0/3729221311 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1a741094f0 con 0x7f1a5c03d610 2024-04-25T13:50:47.568 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mon update... 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a5c03d610 msgr2=0x7f1a5c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a5c03d610 0x7f1a5c03fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006fd0 tx=0x7f1a64006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 msgr2=0x7f1a74107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74107f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1a70006130 tx=0x7f1a700097d0 comp rx=0 tx=0).stop 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 shutdown_connections 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a5c03d610 0x7f1a5c03fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 --2- 172.21.15.129:0/3729221311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a74102320 0x7f1a74107f20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 >> 172.21.15.129:0/3729221311 conn(0x7f1a740fdd00 msgr2=0x7f1a7410a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 shutdown_connections 2024-04-25T13:50:47.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:47.569+0000 7f1a7acc8700 1 -- 172.21.15.129:0/3729221311 wait complete. 2024-04-25T13:50:48.147 DEBUG:teuthology.orchestra.run.smithi112:mon.b> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T13:50:48.150 DEBUG:teuthology.orchestra.run.smithi129:mon.c> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T13:50:48.152 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:50:48.152 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:50:48.875 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:50:48.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:48.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c;count:3 2024-04-25T13:50:48.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:48.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:48.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:48.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:48.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:48.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:50:48.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:48.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: Deploying daemon mon.c on smithi129 2024-04-25T13:50:48.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:48 smithi078 ceph-mon[29163]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:51.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.304+0000 7f54ac3e4700 1 -- 172.21.15.129:0/2685003763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 msgr2=0x7f54a4108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:51.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.304+0000 7f54ac3e4700 1 --2- 172.21.15.129:0/2685003763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a4108d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009910 tx=0x7f54a0009c20 comp rx=0 tx=0).stop 2024-04-25T13:50:51.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.304+0000 7f54ac3e4700 1 -- 172.21.15.129:0/2685003763 shutdown_connections 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.304+0000 7f54ac3e4700 1 --2- 172.21.15.129:0/2685003763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a4108d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.304+0000 7f54ac3e4700 1 -- 172.21.15.129:0/2685003763 >> 172.21.15.129:0/2685003763 conn(0x7f54a40fe440 msgr2=0x7f54a4100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.304+0000 7f54ac3e4700 1 -- 172.21.15.129:0/2685003763 shutdown_connections 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.304+0000 7f54ac3e4700 1 -- 172.21.15.129:0/2685003763 wait complete. 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.305+0000 7f54ac3e4700 1 Processor -- start 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.305+0000 7f54ac3e4700 1 -- start start 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.305+0000 7f54ac3e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:51.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.305+0000 7f54ac3e4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54a410ce80 con 0x7f54a4108930 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.305+0000 7f54aa180700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.305+0000 7f54aa180700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.129:42652/0 (socket says 172.21.15.129:42652) 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.305+0000 7f54aa180700 1 -- 172.21.15.129:0/4038210756 learned_addr learned my addr 172.21.15.129:0/4038210756 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.306+0000 7f54aa180700 1 -- 172.21.15.129:0/4038210756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a406c940 con 0x7f54a4108930 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.306+0000 7f54aa180700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a419a660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54a00056d0 tx=0x7f54a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.306+0000 7f549b7fe700 1 -- 172.21.15.129:0/4038210756 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54a0027070 con 0x7f54a4108930 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.306+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a419ac60 con 0x7f54a4108930 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.306+0000 7f549b7fe700 1 -- 172.21.15.129:0/4038210756 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a000be10 con 0x7f54a4108930 2024-04-25T13:50:51.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.306+0000 7f549b7fe700 1 -- 172.21.15.129:0/4038210756 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54a0022d20 con 0x7f54a4108930 2024-04-25T13:50:51.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.306+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a419b120 con 0x7f54a4108930 2024-04-25T13:50:51.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.307+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a4104040 con 0x7f54a4108930 2024-04-25T13:50:51.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.307+0000 7f549b7fe700 1 -- 172.21.15.129:0/4038210756 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f54a002a070 con 0x7f54a4108930 2024-04-25T13:50:51.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.307+0000 7f549b7fe700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f549003d610 0x7f549003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:51.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.308+0000 7f549b7fe700 1 -- 172.21.15.129:0/4038210756 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f54a001e030 con 0x7f54a4108930 2024-04-25T13:50:51.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.308+0000 7f54a997f700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f549003d610 0x7f549003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:51.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.308+0000 7f54a997f700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f549003d610 0x7f549003fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5494006fd0 tx=0x7f5494006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:51.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.313+0000 7f549b7fe700 1 -- 172.21.15.129:0/4038210756 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54a406c940 con 0x7f54a4108930 2024-04-25T13:50:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:51 smithi078 ceph-mon[29163]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:51.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.599+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f54a419b400 con 0x7f54a4108930 2024-04-25T13:50:51.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.600+0000 7f549b7fe700 1 -- 172.21.15.129:0/4038210756 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f54a0031090 con 0x7f54a4108930 2024-04-25T13:50:51.600 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:50:51.600 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":1,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","modified":"2024-04-25T13:49:01.492952Z","created":"2024-04-25T13:49:01.492952Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T13:50:51.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f549003d610 msgr2=0x7f549003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:51.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f549003d610 0x7f549003fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5494006fd0 tx=0x7f5494006e40 comp rx=0 tx=0).stop 2024-04-25T13:50:51.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 msgr2=0x7f54a419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:51.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a419a660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54a00056d0 tx=0x7f54a00057b0 comp rx=0 tx=0).stop 2024-04-25T13:50:51.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 shutdown_connections 2024-04-25T13:50:51.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f549003d610 0x7f549003fb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:51.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 --2- 172.21.15.129:0/4038210756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54a4108930 0x7f54a419a660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:51.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 >> 172.21.15.129:0/4038210756 conn(0x7f54a40fe440 msgr2=0x7f54a40ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:51.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 shutdown_connections 2024-04-25T13:50:51.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:51.603+0000 7f54ac3e4700 1 -- 172.21.15.129:0/4038210756 wait complete. 2024-04-25T13:50:51.604 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 1 2024-04-25T13:50:52.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:52 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/4038210756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:50:53.315 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: load: jerasure load: lrc 2024-04-25T13:50:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:53 smithi078 ceph-mon[29163]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.533 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:50:53.586 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:50:53.610 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Git sha 0 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: DB SUMMARY 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: DB Session ID: GOUP5QOV07UO5WE9BLXR 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:50:53.611 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.env: 0x5622767c2500 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.info_log: 0x56227825ffc0 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.statistics: (nil) 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.use_fsync: 0 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:50:53.612 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.db_log_dir: 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.wal_dir: 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:50:53.613 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.write_buffer_manager: 0x5622782308c0 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:50:53.614 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.unordered_write: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.row_cache: None 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.wal_filter: None 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.wal_compression: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:50:53.615 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:50:53.616 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:50:53.618 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:50:53.618 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:50:53.618 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:50:53.618 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:50:53.618 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_open_files: -1 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Compression algorithms supported: 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kZSTD supported: 0 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:50:53.619 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.merge_operator: 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_filter: None 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:50:53.620 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5622783661a0) 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: index_type: 0 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: data_block_index_type: 0 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: index_shortening: 1 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: checksum: 4 2024-04-25T13:50:53.621 INFO:journalctl@ceph.mon.c.smithi129.stdout: no_block_cache: 0 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache: 0x5622782631f0 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache_options: 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: capacity : 536870912 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: num_shard_bits : 4 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: strict_capacity_limit : 0 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache_compressed: (nil) 2024-04-25T13:50:53.622 INFO:journalctl@ceph.mon.c.smithi129.stdout: persistent_cache: (nil) 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_size: 4096 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_size_deviation: 10 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_restart_interval: 16 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: index_block_restart_interval: 1 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: metadata_block_size: 4096 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: partition_filters: 0 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: use_delta_encoding: 1 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: filter_policy: bloomfilter 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: whole_key_filtering: 1 2024-04-25T13:50:53.623 INFO:journalctl@ceph.mon.c.smithi129.stdout: verify_compression: 0 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: format_version: 5 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: enable_index_compression: 1 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_align: 0 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: max_auto_readahead_size: 262144 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: prepopulate_block_cache: 0 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:50:53.624 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression: NoCompression 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.num_levels: 7 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:50:53.625 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:50:53.626 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:50:53.627 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:50:53.628 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:50:53.629 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:50:53.630 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:50:53.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.ttl: 2592000 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:50:53.632 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 60867384-6c59-4fdb-adbf-658eb3a67dc9 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053053318905, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053053319320, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714053053, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "60867384-6c59-4fdb-adbf-658eb3a67dc9", "db_session_id": "GOUP5QOV07UO5WE9BLXR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:50:53.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053053319399, "job": 1, "event": "recovery_finished"} 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562278247c00 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: DB pointer 0x56227d03a000 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** DB Stats ** 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:50:53.634 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** Compaction Stats [default] ** 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:50:53.635 INFO:journalctl@ceph.mon.c.smithi129.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:50:53.636 INFO:journalctl@ceph.mon.c.smithi129.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:50:53.636 INFO:journalctl@ceph.mon.c.smithi129.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:50:53.636 INFO:journalctl@ceph.mon.c.smithi129.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:50:53.636 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-04-25T13:50:53.636 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** Compaction Stats [default] ** 2024-04-25T13:50:53.636 INFO:journalctl@ceph.mon.c.smithi129.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:50:53.637 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: Block cache BinnedLRUCache@0x5622782631f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-04-25T13:50:53.638 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: using public_addr v2:172.21.15.129:0/0 -> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: starting mon.c rank -1 at public addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] at bind addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(???) e0 preinit fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout: e1 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout: legacy client fscid: -1 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-04-25T13:50:53.639 INFO:journalctl@ceph.mon.c.smithi129.stdout: No filesystems configured 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:50:53.640 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:50:53.641 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Deploying cephadm binary to smithi129 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:53.642 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Added host smithi129 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3242584639' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3242584639' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.644 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c;count:3 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:53.645 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: Deploying daemon mon.c on smithi129 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/4038210756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 ceph-mon[32569]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 bash[32550]: 004acab6f6315a39960c33efe84b15d4eaffa935ebef7f5910b43048115aa415 2024-04-25T13:50:53.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:53 smithi129 systemd[1]: Started Ceph mon.c for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:50:53.820 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:50:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.522+0000 7f858aedd700 1 -- 172.21.15.129:0/2841166144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85740035c0 msgr2=0x7f8574005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.522+0000 7f858aedd700 1 --2- 172.21.15.129:0/2841166144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85740035c0 0x7f8574005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8584102940 tx=0x7f8578013e90 comp rx=0 tx=0).stop 2024-04-25T13:50:58.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.522+0000 7f858aedd700 1 -- 172.21.15.129:0/2841166144 shutdown_connections 2024-04-25T13:50:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.522+0000 7f858aedd700 1 --2- 172.21.15.129:0/2841166144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85740035c0 0x7f8574005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.522+0000 7f858aedd700 1 -- 172.21.15.129:0/2841166144 >> 172.21.15.129:0/2841166144 conn(0x7f85840796b0 msgr2=0x7f8584079af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.522+0000 7f858aedd700 1 -- 172.21.15.129:0/2841166144 shutdown_connections 2024-04-25T13:50:58.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.522+0000 7f858aedd700 1 -- 172.21.15.129:0/2841166144 wait complete. 2024-04-25T13:50:58.524 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:50:58.524 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: mon.a calling monitor election 2024-04-25T13:50:58.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:58.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: mon.c calling monitor election 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:50:58.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: monmap epoch 2 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: last_changed 2024-04-25T13:50:53.346329+0000 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: min_mon_release 19 (squid) 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: election_strategy: 1 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: fsmap 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: mgrmap e13: x(active, since 50s) 2024-04-25T13:50:58.527 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T13:50:58.528 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:58.528 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:58.528 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:58.528 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:50:58.528 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:58.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f858aedd700 1 Processor -- start 2024-04-25T13:50:58.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f858aedd700 1 -- start start 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f858aedd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 0x7f858419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f858aedd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858419add0 0x7f858419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f858aedd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f858410a9f0 con 0x7f858419add0 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f858aedd700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f858410a870 con 0x7f85740035c0 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8588c79700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 0x7f858419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8588c79700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 0x7f858419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38758/0 (socket says 172.21.15.129:38758) 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8588c79700 1 -- 172.21.15.129:0/2944762604 learned_addr learned my addr 172.21.15.129:0/2944762604 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8583fff700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858419add0 0x7f858419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:58.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8588c79700 1 -- 172.21.15.129:0/2944762604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 msgr2=0x7f858419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8588c79700 1 -- 172.21.15.129:0/2944762604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 msgr2=0x7f858419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8588c79700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 0x7f858419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.524+0000 7f8588c79700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 0x7f858419a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f8583fff700 1 -- 172.21.15.129:0/2944762604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 msgr2=0x7f858419a890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f8583fff700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 0x7f858419a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f8583fff700 1 -- 172.21.15.129:0/2944762604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85841a3900 con 0x7f858419add0 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f8583fff700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858419add0 0x7f858419b1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f857000b4d0 tx=0x7f857000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:58.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f8581ffb700 1 -- 172.21.15.129:0/2944762604 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8570011630 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f8581ffb700 1 -- 172.21.15.129:0/2944762604 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8570011c80 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f8581ffb700 1 -- 172.21.15.129:0/2944762604 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8570010da0 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.525+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85841a3b30 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.526+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85841a4090 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.527+0000 7f8581ffb700 1 -- 172.21.15.129:0/2944762604 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f8570018020 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.527+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8584104150 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.527+0000 7f8581ffb700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f856803d400 0x7f856803f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.527+0000 7f8581ffb700 1 -- 172.21.15.129:0/2944762604 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8570050f50 con 0x7f858419add0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.527+0000 7f8588c79700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f856803d400 0x7f856803f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:58.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.528+0000 7f8588c79700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f856803d400 0x7f856803f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8578004d60 tx=0x7f8578004c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:58.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.533+0000 7f8581ffb700 1 -- 172.21.15.129:0/2944762604 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f857001f020 con 0x7f858419add0 2024-04-25T13:50:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: mon.a calling monitor election 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: mon.c calling monitor election 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: monmap epoch 2 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: last_changed 2024-04-25T13:50:53.346329+0000 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: min_mon_release 19 (squid) 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: election_strategy: 1 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: fsmap 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:50:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: mgrmap e13: x(active, since 50s) 2024-04-25T13:50:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: overall HEALTH_OK 2024-04-25T13:50:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:50:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:50:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:50:58.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.817+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f858402e510 con 0x7f858419add0 2024-04-25T13:50:58.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.817+0000 7f8581ffb700 1 -- 172.21.15.129:0/2944762604 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8570010410 con 0x7f858419add0 2024-04-25T13:50:58.818 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:50:58.818 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","modified":"2024-04-25T13:50:53.346329Z","created":"2024-04-25T13:49:01.492952Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:50:58.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.819+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f856803d400 msgr2=0x7f856803f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:58.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.819+0000 7f858aedd700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f856803d400 0x7f856803f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8578004d60 tx=0x7f8578004c70 comp rx=0 tx=0).stop 2024-04-25T13:50:58.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.819+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858419add0 msgr2=0x7f858419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.819+0000 7f858aedd700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858419add0 0x7f858419b1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f857000b4d0 tx=0x7f857000b890 comp rx=0 tx=0).stop 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.820+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 shutdown_connections 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.820+0000 7f858aedd700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f856803d400 0x7f856803f910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.820+0000 7f858aedd700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85740035c0 0x7f858419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.820+0000 7f858aedd700 1 --2- 172.21.15.129:0/2944762604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858419add0 0x7f858419b1e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.820+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 >> 172.21.15.129:0/2944762604 conn(0x7f85840796b0 msgr2=0x7f8584105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.820+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 shutdown_connections 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:50:58.820+0000 7f858aedd700 1 -- 172.21.15.129:0/2944762604 wait complete. 2024-04-25T13:50:58.821 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 2 2024-04-25T13:50:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:59 smithi129 ceph-mon[32569]: Deploying daemon mon.b on smithi112 2024-04-25T13:50:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:59 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/2944762604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:50:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:50:59 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:50:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:59 smithi078 ceph-mon[29163]: Deploying daemon mon.b on smithi112 2024-04-25T13:50:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:59 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/2944762604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:50:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:50:59 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:00.394 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:51:00.394 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:51:00.639 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:51:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:00 smithi078 ceph-mon[29163]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:00.751 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:00 smithi129 ceph-mon[32569]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 -- 172.21.15.129:0/27602617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 msgr2=0x7f089c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:02.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 --2- 172.21.15.129:0/27602617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c1037e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f089400b0a0 tx=0x7f089400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:51:02.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 -- 172.21.15.129:0/27602617 shutdown_connections 2024-04-25T13:51:02.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 --2- 172.21.15.129:0/27602617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c1037e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:02.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 -- 172.21.15.129:0/27602617 >> 172.21.15.129:0/27602617 conn(0x7f089c077ce0 msgr2=0x7f089c078120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:02.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 -- 172.21.15.129:0/27602617 shutdown_connections 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 -- 172.21.15.129:0/27602617 wait complete. 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 Processor -- start 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.080+0000 7f08a2ce9700 1 -- start start 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a2ce9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a2ce9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 0x7f089c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a2ce9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f089c10d1a0 con 0x7f089c1033d0 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a2ce9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f089c10d020 con 0x7f089c19fd60 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a0a85700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a0a85700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.129:41740/0 (socket says 172.21.15.129:41740) 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a0a85700 1 -- 172.21.15.129:0/3770246593 learned_addr learned my addr 172.21.15.129:0/3770246593 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f089bfff700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 0x7f089c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:02.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f089bfff700 1 -- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 msgr2=0x7f089c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f089bfff700 1 -- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 msgr2=0x7f089c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f089bfff700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 0x7f089c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f089bfff700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 0x7f089c199a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a0a85700 1 -- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 msgr2=0x7f089c199a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a0a85700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 0x7f089c199a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.081+0000 7f08a0a85700 1 -- 172.21.15.129:0/3770246593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c1a0330 con 0x7f089c1033d0 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.082+0000 7f08a0a85700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c19f820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f089400d010 tx=0x7f0894007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.082+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0894017040 con 0x7f089c1033d0 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.082+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08940129a0 con 0x7f089c1033d0 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.082+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f089401da40 con 0x7f089c1033d0 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.082+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c199fe0 con 0x7f089c1033d0 2024-04-25T13:51:02.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.082+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c19a570 con 0x7f089c1033d0 2024-04-25T13:51:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.083+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089c1043f0 con 0x7f089c1033d0 2024-04-25T13:51:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.083+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f0894024020 con 0x7f089c1033d0 2024-04-25T13:51:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.083+0000 7f0899ffb700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f088803d3d0 0x7f088803f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.083+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f089400f050 con 0x7f089c1033d0 2024-04-25T13:51:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.083+0000 7f089bfff700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f088803d3d0 0x7f088803f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:02.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.084+0000 7f089bfff700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f088803d3d0 0x7f088803f8e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f089c106be0 tx=0x7f088c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:02.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.089+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f089402b020 con 0x7f089c1033d0 2024-04-25T13:51:02.156 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 podman[32598]: 2024-04-25 13:51:02.12301711 +0000 UTC m=+0.343577051 container start 874de52201275d945cb2a13a96b79afdf5d24169a2b4f4c70aaa5c7c3927004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD) 2024-04-25T13:51:02.156 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:51:02.156 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T13:51:02.156 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pidfile_write: ignore empty --pid-file 2024-04-25T13:51:02.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.200+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894011570 con 0x7f089c1033d0 2024-04-25T13:51:02.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:02.372+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f089c064cf0 con 0x7f089c1033d0 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: load: jerasure load: lrc 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Git sha 0 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: DB SUMMARY 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: DB Session ID: 8JVIXBRE8PINYIOMMH6U 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T13:51:02.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.env: 0x562e1c097500 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:51:02.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.info_log: 0x562e1e3c7fc0 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.statistics: (nil) 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.use_fsync: 0 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:51:02.519 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.db_log_dir: 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.wal_dir: 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:51:02.520 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.write_buffer_manager: 0x562e1e3988c0 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.unordered_write: 0 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:51:02.521 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.row_cache: None 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.wal_filter: None 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.wal_compression: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:51:02.522 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:51:02.523 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:51:02.525 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_open_files: -1 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Compression algorithms supported: 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kZSTD supported: 0 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:51:02.526 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.merge_operator: 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_filter: None 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e1e4ce1a0) 2024-04-25T13:51:02.527 INFO:journalctl@ceph.mon.b.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: index_type: 0 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: data_block_index_type: 0 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: index_shortening: 1 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: checksum: 4 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: no_block_cache: 0 2024-04-25T13:51:02.528 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_cache: 0x562e1e3cb1f0 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_cache_options: 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: capacity : 536870912 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: num_shard_bits : 4 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: strict_capacity_limit : 0 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_cache_compressed: (nil) 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: persistent_cache: (nil) 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_size: 4096 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_size_deviation: 10 2024-04-25T13:51:02.529 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_restart_interval: 16 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: index_block_restart_interval: 1 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: metadata_block_size: 4096 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: partition_filters: 0 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: use_delta_encoding: 1 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: filter_policy: bloomfilter 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: whole_key_filtering: 1 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: verify_compression: 0 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: format_version: 5 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: enable_index_compression: 1 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: block_align: 0 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: max_auto_readahead_size: 262144 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: prepopulate_block_cache: 0 2024-04-25T13:51:02.530 INFO:journalctl@ceph.mon.b.smithi112.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression: NoCompression 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.num_levels: 7 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:51:02.531 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:51:02.532 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:51:02.533 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:51:02.534 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:51:02.535 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.ttl: 2592000 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:51:02.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8f1e7cf2-4288-401e-a521-aeafb55e04c6 2024-04-25T13:51:02.537 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053062173961, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053062174405, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714053062, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8f1e7cf2-4288-401e-a521-aeafb55e04c6", "db_session_id": "8JVIXBRE8PINYIOMMH6U", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714053062174501, "job": 1, "event": "recovery_finished"} 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e1e3afc00 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: DB pointer 0x562e231a2000 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:51:02.538 INFO:journalctl@ceph.mon.b.smithi112.stdout: ** DB Stats ** 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:51:02.539 INFO:journalctl@ceph.mon.b.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:51:02.540 INFO:journalctl@ceph.mon.b.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: Cumulative compaction: 0.00 GB write, 0.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: Interval compaction: 0.00 GB write, 0.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: Block cache BinnedLRUCache@0x562e1e3cb1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: using public_addr v2:172.21.15.112:0/0 -> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: starting mon.b rank -1 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(???) e0 preinit fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: e1 2024-04-25T13:51:02.541 INFO:journalctl@ceph.mon.b.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout: legacy client fscid: -1 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout: 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout: No filesystems configured 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:51:02.542 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:02.543 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Deploying cephadm binary to smithi129 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Added host smithi129 2024-04-25T13:51:02.544 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3242584639' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:02.545 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3242584639' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c;count:3 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:02.546 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Deploying daemon mon.c on smithi129 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/4038210756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.a calling monitor election 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.547 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.c calling monitor election 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: monmap epoch 2 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: last_changed 2024-04-25T13:50:53.346329+0000 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: min_mon_release 19 (squid) 2024-04-25T13:51:02.548 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: election_strategy: 1 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: fsmap 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mgrmap e13: x(active, since 50s) 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: overall HEALTH_OK 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: Deploying daemon mon.b on smithi112 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/2944762604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:02.549 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:02.550 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 ceph-mon[32617]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T13:51:02.550 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 bash[32598]: 874de52201275d945cb2a13a96b79afdf5d24169a2b4f4c70aaa5c7c3927004c 2024-04-25T13:51:02.550 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:02 smithi112 systemd[1]: Started Ceph mon.b for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:51:07.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.212+0000 7f0899ffb700 1 -- 172.21.15.129:0/3770246593 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f089401c800 con 0x7f089c1033d0 2024-04-25T13:51:07.213 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T13:51:07.213 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":3,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","modified":"2024-04-25T13:51:02.199427Z","created":"2024-04-25T13:49:01.492952Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T13:51:07.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f088803d3d0 msgr2=0x7f088803f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f088803d3d0 0x7f088803f8e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f089c106be0 tx=0x7f088c008040 comp rx=0 tx=0).stop 2024-04-25T13:51:07.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 msgr2=0x7f089c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:07.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c19f820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f089400d010 tx=0x7f0894007a50 comp rx=0 tx=0).stop 2024-04-25T13:51:07.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 shutdown_connections 2024-04-25T13:51:07.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f088803d3d0 0x7f088803f8e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f089c1033d0 0x7f089c19f820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 --2- 172.21.15.129:0/3770246593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f089c19fd60 0x7f089c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:07.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 >> 172.21.15.129:0/3770246593 conn(0x7f089c077ce0 msgr2=0x7f089c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:07.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 shutdown_connections 2024-04-25T13:51:07.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:07.216+0000 7f08a2ce9700 1 -- 172.21.15.129:0/3770246593 wait complete. 2024-04-25T13:51:07.216 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 3 2024-04-25T13:51:07.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:51:07.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: mon.a calling monitor election 2024-04-25T13:51:07.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: mon.c calling monitor election 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/3770246593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: mon.b calling monitor election 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:51:07.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: monmap epoch 3 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: last_changed 2024-04-25T13:51:02.199427+0000 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: min_mon_release 19 (squid) 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: election_strategy: 1 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: fsmap 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: mgrmap e13: x(active, since 59s) 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: overall HEALTH_OK 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:07.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:07.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:51:07.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: mon.a calling monitor election 2024-04-25T13:51:07.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: mon.c calling monitor election 2024-04-25T13:51:07.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/3770246593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: mon.b calling monitor election 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: monmap epoch 3 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:07.572 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: last_changed 2024-04-25T13:51:02.199427+0000 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: min_mon_release 19 (squid) 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: election_strategy: 1 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: fsmap 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: mgrmap e13: x(active, since 59s) 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:07.573 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:51:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: mon.a calling monitor election 2024-04-25T13:51:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: mon.c calling monitor election 2024-04-25T13:51:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/3770246593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: mon.b calling monitor election 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: monmap epoch 3 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: last_changed 2024-04-25T13:51:02.199427+0000 2024-04-25T13:51:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: created 2024-04-25T13:49:01.492952+0000 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: min_mon_release 19 (squid) 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: election_strategy: 1 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: 2: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: fsmap 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: mgrmap e13: x(active, since 59s) 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: overall HEALTH_OK 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:07.893 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T13:51:07.893 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T13:51:08.146 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:51:08.440 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.441 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.441 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:08.441 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:08 smithi129 ceph-mon[32569]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:08.441 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.441 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:08.448 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:08.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:08 smithi112 ceph-mon[32617]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:08.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:08.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:08.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:08 smithi078 ceph-mon[29163]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:08.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:08.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:51:09.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.826+0000 7f5c14ffc700 1 -- 172.21.15.78:0/2864937595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c10104c00 msgr2=0x7f5c10105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:09.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.826+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/2864937595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c10104c00 0x7f5c10105010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009910 tx=0x7f5c00009c20 comp rx=0 tx=0).stop 2024-04-25T13:51:09.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.826+0000 7f5c14ffc700 1 -- 172.21.15.78:0/2864937595 shutdown_connections 2024-04-25T13:51:09.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.826+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/2864937595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c10104c00 0x7f5c10105010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:09.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.826+0000 7f5c14ffc700 1 -- 172.21.15.78:0/2864937595 >> 172.21.15.78:0/2864937595 conn(0x7f5c10078460 msgr2=0x7f5c100788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:09.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.826+0000 7f5c14ffc700 1 -- 172.21.15.78:0/2864937595 shutdown_connections 2024-04-25T13:51:09.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:09.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:09.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: Updating smithi078:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.828 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.829 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.829 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.829 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:09.829 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:09.829 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.827+0000 7f5c14ffc700 1 -- 172.21.15.78:0/2864937595 wait complete. 2024-04-25T13:51:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 Processor -- start 2024-04-25T13:51:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 -- start start 2024-04-25T13:51:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c10104c00 0x7f5c1019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c1019fc30 0x7f5c10199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c10199e50 0x7f5c1019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010dff0 con 0x7f5c1019fc30 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010de70 con 0x7f5c10199e50 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c14ffc700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1010e170 con 0x7f5c10104c00 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c0dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c1019fc30 0x7f5c10199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c0dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c1019fc30 0x7f5c10199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56138/0 (socket says 172.21.15.78:56138) 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c0dd9b700 1 -- 172.21.15.78:0/535585351 learned_addr learned my addr 172.21.15.78:0/535585351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.828+0000 7f5c0ed9d700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c10199e50 0x7f5c1019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c0e59c700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c10104c00 0x7f5c1019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c0dd9b700 1 -- 172.21.15.78:0/535585351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c10104c00 msgr2=0x7f5c1019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c0dd9b700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c10104c00 0x7f5c1019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c0dd9b700 1 -- 172.21.15.78:0/535585351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c10199e50 msgr2=0x7f5c1019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c0dd9b700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c10199e50 0x7f5c1019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c0dd9b700 1 -- 172.21.15.78:0/535585351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c101a6580 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c0dd9b700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c1019fc30 0x7f5c10199910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b780 tx=0x7f5c0400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5bff7fe700 1 -- 172.21.15.78:0/535585351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04016070 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c101a6810 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5bff7fe700 1 -- 172.21.15.78:0/535585351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c040036a0 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5bff7fe700 1 -- 172.21.15.78:0/535585351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c040119c0 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.829+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101a6d70 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.830+0000 7f5bff7fe700 1 -- 172.21.15.78:0/535585351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f5c04019070 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.831+0000 7f5bff7fe700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5bf803daa0 0x7f5bf803ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.831+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c10064cf0 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.831+0000 7f5c0e59c700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5bf803daa0 0x7f5bf803ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.831+0000 7f5bff7fe700 1 -- 172.21.15.78:0/535585351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5c04051110 con 0x7f5c1019fc30 2024-04-25T13:51:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.831+0000 7f5c0e59c700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5bf803daa0 0x7f5bf803ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009750 tx=0x7f5c0001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:09.836+0000 7f5bff7fe700 1 -- 172.21.15.78:0/535585351 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c0400f2e0 con 0x7f5c1019fc30 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T13:51:09.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: Updating smithi078:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.916 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.916 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:09.916 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:09.916 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:09.916 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-25T13:51:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: Updating smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: Updating smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: Updating smithi078:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:10.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:10.018 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:10.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.056+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5c101a70c0 con 0x7f5c1019fc30 2024-04-25T13:51:10.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.056+0000 7f5bff7fe700 1 -- 172.21.15.78:0/535585351 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f5c04010e60 con 0x7f5c1019fc30 2024-04-25T13:51:10.057 INFO:teuthology.orchestra.run.smithi078.stdout:# minimal ceph.conf for 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:10.057 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-04-25T13:51:10.057 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:51:10.057 INFO:teuthology.orchestra.run.smithi078.stdout: mon_host = [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5bf803daa0 msgr2=0x7f5bf803ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5bf803daa0 0x7f5bf803ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009750 tx=0x7f5c0001a040 comp rx=0 tx=0).stop 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c1019fc30 msgr2=0x7f5c10199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c1019fc30 0x7f5c10199910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b780 tx=0x7f5c0400ba90 comp rx=0 tx=0).stop 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 shutdown_connections 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5bf803daa0 0x7f5bf803ffb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c10199e50 0x7f5c1019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c10104c00 0x7f5c1019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 --2- 172.21.15.78:0/535585351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c1019fc30 0x7f5c10199910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:10.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 >> 172.21.15.78:0/535585351 conn(0x7f5c10078460 msgr2=0x7f5c10109fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:10.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 shutdown_connections 2024-04-25T13:51:10.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:10.059+0000 7f5c14ffc700 1 -- 172.21.15.78:0/535585351 wait complete. 2024-04-25T13:51:10.737 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T13:51:10.737 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:51:10.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:51:10.764 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:51:10.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:51:10.827 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:51:10.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:51:10.858 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:51:10.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:51:10.919 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:10 smithi078 ceph-mon[29163]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:51:10.919 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:10 smithi078 ceph-mon[29163]: Reconfiguring daemon mon.a on smithi078 2024-04-25T13:51:10.919 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:10 smithi078 ceph-mon[29163]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:10.919 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:10 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/535585351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:10.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:10 smithi112 ceph-mon[32617]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:51:10.922 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:10 smithi112 ceph-mon[32617]: Reconfiguring daemon mon.a on smithi078 2024-04-25T13:51:10.922 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:10 smithi112 ceph-mon[32617]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:10.922 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:10 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/535585351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:10.925 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:51:10.925 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:51:10.948 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:10 smithi129 ceph-mon[32569]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:51:10.948 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:10 smithi129 ceph-mon[32569]: Reconfiguring daemon mon.a on smithi078 2024-04-25T13:51:10.948 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:10 smithi129 ceph-mon[32569]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:10.948 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:10 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/535585351' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:10.953 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:51:10.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:51:11.017 INFO:tasks.cephadm:Adding mgr.x on smithi078 2024-04-25T13:51:11.017 INFO:tasks.cephadm:Adding mgr.y on smithi112 2024-04-25T13:51:11.017 INFO:tasks.cephadm:Adding mgr.z on smithi129 2024-04-25T13:51:11.017 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi078=x;smithi112=y;smithi129=z' 2024-04-25T13:51:11.300 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:51:12.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.881+0000 7fcfba288700 1 -- 172.21.15.129:0/2350898437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb41030b0 msgr2=0x7fcfb41034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.881+0000 7fcfba288700 1 --2- 172.21.15.129:0/2350898437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb41030b0 0x7fcfb41034c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009910 tx=0x7fcfa4009c20 comp rx=0 tx=0).stop 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.881+0000 7fcfba288700 1 -- 172.21.15.129:0/2350898437 shutdown_connections 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.881+0000 7fcfba288700 1 --2- 172.21.15.129:0/2350898437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb41030b0 0x7fcfb41034c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.881+0000 7fcfba288700 1 -- 172.21.15.129:0/2350898437 >> 172.21.15.129:0/2350898437 conn(0x7fcfb40fd8a0 msgr2=0x7fcfb40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.882+0000 7fcfba288700 1 -- 172.21.15.129:0/2350898437 shutdown_connections 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.882+0000 7fcfba288700 1 -- 172.21.15.129:0/2350898437 wait complete. 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.882+0000 7fcfba288700 1 Processor -- start 2024-04-25T13:51:12.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.882+0000 7fcfba288700 1 -- start start 2024-04-25T13:51:12.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfba288700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb41030b0 0x7fcfb40ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:12.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfba288700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb40fde60 0x7fcfb40fe270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:12.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfba288700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb40fe7b0 0x7fcfb41a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:12.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfba288700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4108aa0 con 0x7fcfb40fe7b0 2024-04-25T13:51:12.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfba288700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4108920 con 0x7fcfb40fde60 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfba288700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4108c20 con 0x7fcfb41030b0 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb41030b0 0x7fcfb40ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb41030b0 0x7fcfb40ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47854/0 (socket says 172.21.15.129:47854) 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 -- 172.21.15.129:0/1295158544 learned_addr learned my addr 172.21.15.129:0/1295158544 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb8825700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb40fe7b0 0x7fcfb41a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 -- 172.21.15.129:0/1295158544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb40fde60 msgr2=0x7fcfb40fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb40fde60 0x7fcfb40fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 -- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb40fe7b0 msgr2=0x7fcfb41a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb40fe7b0 0x7fcfb41a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 -- 172.21.15.129:0/1295158544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb41a3e50 con 0x7fcfb41030b0 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb3fff700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb41030b0 0x7fcfb40ff7b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4005850 tx=0x7fcfa4004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb17fa700 1 -- 172.21.15.129:0/1295158544 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004300 con 0x7fcfb41030b0 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb17fa700 1 -- 172.21.15.129:0/1295158544 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa40044a0 con 0x7fcfb41030b0 2024-04-25T13:51:12.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfb17fa700 1 -- 172.21.15.129:0/1295158544 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4003b90 con 0x7fcfb41030b0 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.883+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb41a4080 con 0x7fcfb41030b0 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.884+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a45b0 con 0x7fcfb41030b0 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.884+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb4064cf0 con 0x7fcfb41030b0 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.885+0000 7fcfb17fa700 1 -- 172.21.15.129:0/1295158544 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fcfa400f460 con 0x7fcfb41030b0 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.885+0000 7fcfb17fa700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf9c03daa0 0x7fcf9c03ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.885+0000 7fcfb17fa700 1 -- 172.21.15.129:0/1295158544 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcfa4052840 con 0x7fcfb41030b0 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.885+0000 7fcfb37fe700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf9c03daa0 0x7fcf9c03ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:12.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.886+0000 7fcfb37fe700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf9c03daa0 0x7fcf9c03ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40ff160 tx=0x7fcfa8006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:12.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:12.890+0000 7fcfb17fa700 1 -- 172.21.15.129:0/1295158544 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfa4013070 con 0x7fcfb41030b0 2024-04-25T13:51:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:51:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:12.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:12.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: Reconfiguring daemon mon.b on smithi112 2024-04-25T13:51:12.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:12 smithi129 ceph-mon[32569]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: Reconfiguring daemon mon.b on smithi112 2024-04-25T13:51:12.921 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:12 smithi112 ceph-mon[32617]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:12.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:12.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:12.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:51:12.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:12.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:12.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:12.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: Reconfiguring daemon mon.b on smithi112 2024-04-25T13:51:12.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:12 smithi078 ceph-mon[29163]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:13.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.100+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi112=y;smithi129=z", "target": ["mon-mgr", ""]}) -- 0x7fcfb4063ad0 con 0x7fcf9c03daa0 2024-04-25T13:51:13.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.109+0000 7fcfb17fa700 1 -- 172.21.15.129:0/1295158544 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcfb4063ad0 con 0x7fcf9c03daa0 2024-04-25T13:51:13.108 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mgr update... 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf9c03daa0 msgr2=0x7fcf9c03ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf9c03daa0 0x7fcf9c03ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40ff160 tx=0x7fcfa8006dd0 comp rx=0 tx=0).stop 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb41030b0 msgr2=0x7fcfb40ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb41030b0 0x7fcfb40ff7b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4005850 tx=0x7fcfa4004f80 comp rx=0 tx=0).stop 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 shutdown_connections 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf9c03daa0 0x7fcf9c03ffb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb40fe7b0 0x7fcfb41a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfb41030b0 0x7fcfb40ff7b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 --2- 172.21.15.129:0/1295158544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfb40fde60 0x7fcfb40fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 >> 172.21.15.129:0/1295158544 conn(0x7fcfb40fd8a0 msgr2=0x7fcfb4102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:13.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 shutdown_connections 2024-04-25T13:51:13.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:51:13.111+0000 7fcfba288700 1 -- 172.21.15.129:0/1295158544 wait complete. 2024-04-25T13:51:13.745 DEBUG:teuthology.orchestra.run.smithi112:mgr.y> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T13:51:13.748 DEBUG:teuthology.orchestra.run.smithi129:mgr.z> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T13:51:13.750 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T13:51:13.751 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:51:13.751 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:51:13.769 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:51:13.770 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 13:50:20.866560060 +0000 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 13:44:48.642308242 +0000 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 13:44:48.642308242 +0000 2024-04-25T13:51:13.827 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 13:44:48.642308242 +0000 2024-04-25T13:51:13.827 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:51:13.893 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T13:51:13.893 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T13:51:13.893 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000232491 s, 2.2 MB/s 2024-04-25T13:51:13.894 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:51:13.952 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-04-25T13:51:14.009 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:51:14.009 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:14.010 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T13:51:14.010 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:14.010 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:14.010 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 13:50:20.904559512 +0000 2024-04-25T13:51:14.010 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 13:44:48.989302940 +0000 2024-04-25T13:51:14.010 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 13:44:48.989302940 +0000 2024-04-25T13:51:14.010 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 13:44:48.989302940 +0000 2024-04-25T13:51:14.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:51:14.074 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T13:51:14.074 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T13:51:14.074 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000213863 s, 2.4 MB/s 2024-04-25T13:51:14.076 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:51:14.132 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-04-25T13:51:14.188 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:51:14.188 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:14.188 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T13:51:14.188 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:14.188 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:14.189 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 13:50:20.936559051 +0000 2024-04-25T13:51:14.189 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 13:44:49.353297378 +0000 2024-04-25T13:51:14.189 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 13:44:49.353297378 +0000 2024-04-25T13:51:14.189 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 13:44:49.353297378 +0000 2024-04-25T13:51:14.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi112=y;smithi129=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: Saving service mgr spec with placement smithi078=x;smithi112=y;smithi129=z;count:3 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:14.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:14.241 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T13:51:14.242 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T13:51:14.242 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000207782 s, 2.5 MB/s 2024-04-25T13:51:14.243 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:51:14.300 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 13:50:20.968558590 +0000 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 13:44:49.677292427 +0000 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 13:44:49.677292427 +0000 2024-04-25T13:51:14.358 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 13:44:49.677292427 +0000 2024-04-25T13:51:14.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:51:14.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi112=y;smithi129=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: Saving service mgr spec with placement smithi078=x;smithi112=y;smithi129=z;count:3 2024-04-25T13:51:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:14.421 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T13:51:14.421 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T13:51:14.421 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000211833 s, 2.4 MB/s 2024-04-25T13:51:14.422 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:51:14.479 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T13:51:14.479 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:51:14.498 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:51:14.498 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-25T13:51:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi112=y;smithi129=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: Saving service mgr spec with placement smithi078=x;smithi112=y;smithi129=z;count:3 2024-04-25T13:51:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:14.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:51:14.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:51:14.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:14.533 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 13:51:01.563802553 +0000 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 13:45:13.432804870 +0000 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 13:45:13.432804870 +0000 2024-04-25T13:51:14.534 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 13:45:13.432804870 +0000 2024-04-25T13:51:14.534 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:51:14.599 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T13:51:14.599 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T13:51:14.599 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000283133 s, 1.8 MB/s 2024-04-25T13:51:14.600 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:51:14.658 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 13:51:01.564802525 +0000 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 13:45:13.757795584 +0000 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 13:45:13.757795584 +0000 2024-04-25T13:51:14.715 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 13:45:13.757795584 +0000 2024-04-25T13:51:14.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:51:14.779 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T13:51:14.780 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T13:51:14.780 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000267423 s, 1.9 MB/s 2024-04-25T13:51:14.781 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:51:14.838 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-25T13:51:14.894 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:51:14.894 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:14.894 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T13:51:14.894 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:14.894 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:14.895 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 13:51:01.564802525 +0000 2024-04-25T13:51:14.895 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 13:45:14.108785554 +0000 2024-04-25T13:51:14.895 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 13:45:14.108785554 +0000 2024-04-25T13:51:14.895 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 13:45:14.108785554 +0000 2024-04-25T13:51:14.895 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:51:14.957 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T13:51:14.957 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T13:51:14.957 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000214923 s, 2.4 MB/s 2024-04-25T13:51:14.958 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:51:15.014 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 13:51:01.564802525 +0000 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 13:45:14.441776040 +0000 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 13:45:14.441776040 +0000 2024-04-25T13:51:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 13:45:14.441776040 +0000 2024-04-25T13:51:15.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:51:15.131 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T13:51:15.131 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T13:51:15.131 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000194649 s, 2.6 MB/s 2024-04-25T13:51:15.133 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:51:15.188 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T13:51:15.188 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:51:15.206 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:51:15.206 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-04-25T13:51:15.262 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:51:15.262 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:15.262 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T13:51:15.262 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:15.262 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:15.262 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 13:50:52.759716070 +0000 2024-04-25T13:51:15.263 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 13:45:03.870012697 +0000 2024-04-25T13:51:15.263 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 13:45:03.870012697 +0000 2024-04-25T13:51:15.263 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 13:45:03.870012697 +0000 2024-04-25T13:51:15.263 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:51:15.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:15 smithi112 ceph-mon[32617]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:51:15.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:15 smithi112 ceph-mon[32617]: Reconfiguring daemon mon.c on smithi129 2024-04-25T13:51:15.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:15 smithi112 ceph-mon[32617]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:15.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:15.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:15.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:15.327 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T13:51:15.327 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T13:51:15.327 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000347242 s, 1.5 MB/s 2024-04-25T13:51:15.328 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:51:15.383 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-04-25T13:51:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:15 smithi129 ceph-mon[32569]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:51:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:15 smithi129 ceph-mon[32569]: Reconfiguring daemon mon.c on smithi129 2024-04-25T13:51:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:15 smithi129 ceph-mon[32569]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:15.427 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:51:15.427 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:15.428 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T13:51:15.428 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:15.428 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:15.428 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 13:50:52.759716070 +0000 2024-04-25T13:51:15.428 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 13:45:04.213004179 +0000 2024-04-25T13:51:15.428 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 13:45:04.213004179 +0000 2024-04-25T13:51:15.428 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 13:45:04.213004179 +0000 2024-04-25T13:51:15.428 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:51:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:15 smithi078 ceph-mon[29163]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:51:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:15 smithi078 ceph-mon[29163]: Reconfiguring daemon mon.c on smithi129 2024-04-25T13:51:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:15 smithi078 ceph-mon[29163]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:15.492 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T13:51:15.492 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T13:51:15.492 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000177863 s, 2.9 MB/s 2024-04-25T13:51:15.493 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:51:15.551 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-04-25T13:51:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:51:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:15.608 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T13:51:15.608 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:15.609 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:15.609 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 13:50:52.760716048 +0000 2024-04-25T13:51:15.609 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 13:45:04.553995711 +0000 2024-04-25T13:51:15.609 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 13:45:04.553995711 +0000 2024-04-25T13:51:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 13:45:04.553995711 +0000 2024-04-25T13:51:15.609 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:51:15.671 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T13:51:15.671 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T13:51:15.671 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000195387 s, 2.6 MB/s 2024-04-25T13:51:15.672 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:51:15.727 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-04-25T13:51:15.782 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:51:15.782 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:51:15.782 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T13:51:15.783 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:51:15.783 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:51:15.783 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 13:50:52.760716048 +0000 2024-04-25T13:51:15.783 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 13:45:04.876987690 +0000 2024-04-25T13:51:15.783 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 13:45:04.876987690 +0000 2024-04-25T13:51:15.783 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 13:45:04.876987690 +0000 2024-04-25T13:51:15.783 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:51:15.848 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T13:51:15.848 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T13:51:15.848 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000180681 s, 2.8 MB/s 2024-04-25T13:51:15.850 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:51:15.905 INFO:tasks.cephadm:Deploying osd.0 on smithi078 with /dev/vg_nvme/lv_4... 2024-04-25T13:51:15.905 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:51:16.139 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:51:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:51:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:16.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:16.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:51:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:51:16.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:16.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:17.161 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:17 smithi129 ceph-mon[32569]: Deploying daemon mgr.z on smithi129 2024-04-25T13:51:17.161 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:17 smithi129 ceph-mon[32569]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:17.414 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 systemd[1]: Starting Ceph mgr.z for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:51:17.448 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:17 smithi078 ceph-mon[29163]: Deploying daemon mgr.z on smithi129 2024-04-25T13:51:17.448 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:17 smithi078 ceph-mon[29163]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:17.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:17 smithi112 ceph-mon[32617]: Deploying daemon mgr.z on smithi129 2024-04-25T13:51:17.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:17 smithi112 ceph-mon[32617]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:17.831 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 podman[34413]: 2024-04-25 13:51:17.44862835 +0000 UTC m=+0.018062555 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:51:17.832 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 podman[34413]: 2024-04-25 13:51:17.555269366 +0000 UTC m=+0.124703560 container create 5a240cc8953ee241528b7e036c3f69fb65eb2ff48a506f770ac50272319d2ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T13:51:17.832 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 podman[34413]: 2024-04-25 13:51:17.749352771 +0000 UTC m=+0.318786965 container init 5a240cc8953ee241528b7e036c3f69fb65eb2ff48a506f770ac50272319d2ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:51:17.832 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 podman[34413]: 2024-04-25 13:51:17.751786533 +0000 UTC m=+0.321220729 container start 5a240cc8953ee241528b7e036c3f69fb65eb2ff48a506f770ac50272319d2ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T13:51:17.832 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 ceph-mgr[34432]: -- 172.21.15.129:0/4158418251 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55718cac05a0 con 0x55719184a000 2024-04-25T13:51:17.986 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:51:17.986 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:51:17.986 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T13:51:17.986 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T13:51:17.987 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124956 s, 839 MB/s 2024-04-25T13:51:17.987 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T13:51:18.108 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 bash[34413]: 5a240cc8953ee241528b7e036c3f69fb65eb2ff48a506f770ac50272319d2ab1 2024-04-25T13:51:18.108 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 systemd[1]: Started Ceph mgr.z for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:51:18.108 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:17 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:17.923+0000 7f041e123200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:51:18.108 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:18.108+0000 7f041e123200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:51:18.413 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:18.210+0000 7f041e123200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:51:18.587 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-04-25T13:51:18.820 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:51:18.907 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:18.518+0000 7f041e123200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:51:18.907 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:18.601+0000 7f041e123200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:51:19.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:19.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:51:19.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:19.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:19.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: Deploying daemon mgr.y on smithi112 2024-04-25T13:51:19.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:18 smithi129 ceph-mon[32569]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:19.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:19.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:51:19.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:19.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:19.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: Deploying daemon mgr.y on smithi112 2024-04-25T13:51:19.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:18 smithi078 ceph-mon[29163]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:51:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:19.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: Deploying daemon mgr.y on smithi112 2024-04-25T13:51:19.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:18 smithi112 ceph-mon[32617]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:19.947 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:19 smithi112 systemd[1]: Starting Ceph mgr.y for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:51:19.947 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:19 smithi112 podman[33984]: 2024-04-25 13:51:19.947353141 +0000 UTC m=+0.091831512 container create 93c7a9abd610d6956018b9e7730b8f537cc265f5633d3e0a21f944ade0ba2232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T13:51:20.138 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:19.799+0000 7f041e123200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:51:20.138 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:19.968+0000 7f041e123200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:51:20.138 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:20.138+0000 7f041e123200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 -- 172.21.15.78:0/4210111833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 msgr2=0x7fe234103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 --2- 172.21.15.78:0/4210111833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe234103df0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe228009990 tx=0x7fe228009ca0 comp rx=0 tx=0).stop 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 -- 172.21.15.78:0/4210111833 shutdown_connections 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 --2- 172.21.15.78:0/4210111833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe234104380 0x7fe23410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 --2- 172.21.15.78:0/4210111833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2341028b0 0x7fe234102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 --2- 172.21.15.78:0/4210111833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe234103df0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 -- 172.21.15.78:0/4210111833 >> 172.21.15.78:0/4210111833 conn(0x7fe2340fdef0 msgr2=0x7fe2341003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:20.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 -- 172.21.15.78:0/4210111833 shutdown_connections 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.209+0000 7fe23893a700 1 -- 172.21.15.78:0/4210111833 wait complete. 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 Processor -- start 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 -- start start 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2341028b0 0x7fe23419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe23419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe234104380 0x7fe2341a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe23410daf0 con 0x7fe2341039e0 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe23410d970 con 0x7fe234104380 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23893a700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe23410dc70 con 0x7fe2341028b0 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe23419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe23419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40040/0 (socket says 172.21.15.78:40040) 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 -- 172.21.15.78:0/3735394007 learned_addr learned my addr 172.21.15.78:0/3735394007 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe23259c700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2341028b0 0x7fe23419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:20.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe232d9d700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe234104380 0x7fe2341a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 -- 172.21.15.78:0/3735394007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2341028b0 msgr2=0x7fe23419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2341028b0 0x7fe23419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 -- 172.21.15.78:0/3735394007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe234104380 msgr2=0x7fe2341a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe234104380 0x7fe2341a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 -- 172.21.15.78:0/3735394007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2341a2410 con 0x7fe2341039e0 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.210+0000 7fe231d9b700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe23419adf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe22800b760 tx=0x7fe228004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.211+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22800fad0 con 0x7fe2341039e0 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.211+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe228004cb0 con 0x7fe2341039e0 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.211+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2341a26a0 con 0x7fe2341039e0 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.211+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2280173f0 con 0x7fe2341039e0 2024-04-25T13:51:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.211+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2341a2a90 con 0x7fe2341039e0 2024-04-25T13:51:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.212+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fe228028060 con 0x7fe2341039e0 2024-04-25T13:51:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.212+0000 7fe2237fe700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe21c03d850 0x7fe21c03fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.212+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe2280535b0 con 0x7fe2341039e0 2024-04-25T13:51:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.212+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe234051190 con 0x7fe2341039e0 2024-04-25T13:51:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.212+0000 7fe23259c700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe21c03d850 0x7fe21c03fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.212+0000 7fe23259c700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe21c03d850 0x7fe21c03fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2341038e0 tx=0x7fe224010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:20.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.217+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe228013070 con 0x7fe2341039e0 2024-04-25T13:51:20.257 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:19 smithi112 podman[33984]: 2024-04-25 13:51:19.873057599 +0000 UTC m=+0.017535981 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:51:20.257 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 podman[33984]: 2024-04-25 13:51:20.133101862 +0000 UTC m=+0.277580233 container init 93c7a9abd610d6956018b9e7730b8f537cc265f5633d3e0a21f944ade0ba2232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T13:51:20.257 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 podman[33984]: 2024-04-25 13:51:20.135947187 +0000 UTC m=+0.280425565 container start 93c7a9abd610d6956018b9e7730b8f537cc265f5633d3e0a21f944ade0ba2232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T13:51:20.257 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 ceph-mgr[34004]: -- 172.21.15.112:0/1875072275 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55850c6ba5a0 con 0x558511444900 2024-04-25T13:51:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:20.436+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe23410a540 con 0x7fe21c03d850 2024-04-25T13:51:20.516 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 bash[33984]: 93c7a9abd610d6956018b9e7730b8f537cc265f5633d3e0a21f944ade0ba2232 2024-04-25T13:51:20.516 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 systemd[1]: Started Ceph mgr.y for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:51:20.516 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:20.300+0000 7fb08c9d6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:51:20.516 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:20.497+0000 7fb08c9d6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:51:20.734 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:20.426+0000 7f041e123200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:51:20.734 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:20.521+0000 7f041e123200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:51:20.734 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:20.734+0000 7f041e123200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:51:20.937 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:20.609+0000 7fb08c9d6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:51:20.937 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:20.938+0000 7fb08c9d6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:51:21.106 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:20.933+0000 7f041e123200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:21.043+0000 7fb08c9d6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:21.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:21.410 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:21.410 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.411 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.411 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.411 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:21.411 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:51:21.411 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:51:21.411 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:21.411 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:21.663 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:21.568+0000 7f041e123200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:51:22.292 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:21.996+0000 7f041e123200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:51:22.292 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:22.093+0000 7f041e123200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:22.293+0000 7f041e123200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:22.293 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:22.369 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:22.277+0000 7fb08c9d6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:22.444+0000 7fb08c9d6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:51:22.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:51:22.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:22.766 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:22.611+0000 7fb08c9d6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:51:23.108 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:22.817+0000 7f041e123200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:51:23.109 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:22.910+0000 7f041e123200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: Reconfiguring daemon mgr.x on smithi078 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1030460945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa5f025-762a-4cf2-a61e-9be6a4dd8013"}]: dispatch 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1030460945' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa5f025-762a-4cf2-a61e-9be6a4dd8013"}]': finished 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:23.182 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:23.183 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:22.892+0000 7fb08c9d6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:51:23.183 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:22.981+0000 7fb08c9d6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:51:23.183 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:23.182+0000 7fb08c9d6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:51:23.376 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:23.152+0000 7f041e123200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:23.283+0000 7f041e123200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:23.377+0000 7f041e123200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: Reconfiguring daemon mgr.x on smithi078 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1030460945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa5f025-762a-4cf2-a61e-9be6a4dd8013"}]: dispatch 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1030460945' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa5f025-762a-4cf2-a61e-9be6a4dd8013"}]': finished 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:51:23.377 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:23.378 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:23.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:51:23.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: Reconfiguring daemon mgr.x on smithi078 2024-04-25T13:51:23.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:23.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1030460945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caa5f025-762a-4cf2-a61e-9be6a4dd8013"}]: dispatch 2024-04-25T13:51:23.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1030460945' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caa5f025-762a-4cf2-a61e-9be6a4dd8013"}]': finished 2024-04-25T13:51:23.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:51:23.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:23.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:23.516 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:23.371+0000 7fb08c9d6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:51:23.663 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:23.491+0000 7f041e123200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:51:24.115 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:23.804+0000 7f041e123200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:51:24.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:24 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2579350113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:51:24.266 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:23.960+0000 7fb08c9d6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:51:24.398 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:24 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2579350113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:51:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:24 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2579350113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:51:24.663 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:24.399+0000 7f041e123200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:51:24.663 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 13:51:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-25T13:51:24.612+0000 7f041e123200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:51:24.664 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:24.374+0000 7fb08c9d6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:51:24.664 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:24.468+0000 7fb08c9d6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:51:24.665 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:24.665+0000 7fb08c9d6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:51:25.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:25.122+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 14) ==== 98518+0+0 (secure 0 0 0) 0x7fe228009d10 con 0x7fe2341039e0 2024-04-25T13:51:25.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:25 smithi129 ceph-mon[32569]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:25.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:25 smithi129 ceph-mon[32569]: Standby manager daemon z started 2024-04-25T13:51:25.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:25 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:51:25.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:25 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:51:25.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:25 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:51:25.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:25 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:51:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:25 smithi078 ceph-mon[29163]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:25 smithi078 ceph-mon[29163]: Standby manager daemon z started 2024-04-25T13:51:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:25 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:51:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:25 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:51:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:25 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:51:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:25 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:51:25.507 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:25.180+0000 7fb08c9d6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:51:25.507 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:25.270+0000 7fb08c9d6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:51:25.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-mon[32617]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:25.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-mon[32617]: Standby manager daemon z started 2024-04-25T13:51:25.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:51:25.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:51:25.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:51:25.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.129:0/2890814215' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:51:25.766 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:25.508+0000 7fb08c9d6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:51:25.766 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:25.637+0000 7fb08c9d6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:51:25.766 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:25.730+0000 7fb08c9d6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:51:26.126 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:25 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:25.842+0000 7fb08c9d6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:51:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:26 smithi129 ceph-mon[32569]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T13:51:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:51:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:26 smithi078 ceph-mon[29163]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T13:51:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:51:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:26 smithi112 ceph-mon[32617]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T13:51:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:51:26.516 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:26 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:26.137+0000 7fb08c9d6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:51:27.016 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:26 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:26.723+0000 7fb08c9d6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:51:27.016 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 13:51:26 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y[33999]: 2024-04-25T13:51:26.936+0000 7fb08c9d6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:51:27.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:27.141+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fe22802a450 con 0x7fe2341039e0 2024-04-25T13:51:27.407 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:27 smithi078 ceph-mon[29163]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:27.407 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:27 smithi078 ceph-mon[29163]: Standby manager daemon y started 2024-04-25T13:51:27.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:27 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:51:27.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:27 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:51:27.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:27 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:51:27.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:27 smithi078 ceph-mon[29163]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:51:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:27 smithi129 ceph-mon[32569]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:27 smithi129 ceph-mon[32569]: Standby manager daemon y started 2024-04-25T13:51:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:27 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:51:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:27 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:51:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:27 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:51:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:27 smithi129 ceph-mon[32569]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:51:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:27 smithi112 ceph-mon[32617]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:27 smithi112 ceph-mon[32617]: Standby manager daemon y started 2024-04-25T13:51:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:27 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:51:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:27 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:51:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:27 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:51:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:27 smithi112 ceph-mon[32617]: from='mgr.? 172.21.15.112:0/2102683495' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:51:28.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:28 smithi129 ceph-mon[32569]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T13:51:28.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:28 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:51:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:28 smithi078 ceph-mon[29163]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T13:51:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:28 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:51:28.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:28 smithi112 ceph-mon[32617]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T13:51:28.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:28 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:51:29.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:29 smithi129 ceph-mon[32569]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:29 smithi078 ceph-mon[29163]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:29.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:29 smithi112 ceph-mon[32617]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:31.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:31 smithi129 ceph-mon[32569]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:31.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:51:31.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:31 smithi078 ceph-mon[29163]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:51:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:31 smithi112 ceph-mon[32617]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:51:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:32.406 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:32 smithi078 ceph-mon[29163]: Deploying daemon osd.0 on smithi078 2024-04-25T13:51:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:32 smithi129 ceph-mon[32569]: Deploying daemon osd.0 on smithi078 2024-04-25T13:51:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:32 smithi112 ceph-mon[32617]: Deploying daemon osd.0 on smithi078 2024-04-25T13:51:33.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:33 smithi129 ceph-mon[32569]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:33 smithi078 ceph-mon[29163]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:33.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:33 smithi112 ceph-mon[32617]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:34.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:34 smithi129 ceph-mon[32569]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:34 smithi078 ceph-mon[29163]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:34 smithi112 ceph-mon[32617]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:37.409 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:37 smithi078 ceph-mon[29163]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:37.409 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:37 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:37.409 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:37 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:37.409 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:37 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:37 smithi129 ceph-mon[32569]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:37 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:37 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:37 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:37 smithi112 ceph-mon[32617]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:37 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:37 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:37 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:39 smithi078 ceph-mon[29163]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:39 smithi129 ceph-mon[32569]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:39.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:39 smithi112 ceph-mon[32617]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:40.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.301+0000 7fe2237fe700 1 -- 172.21.15.78:0/3735394007 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe23410a540 con 0x7fe21c03d850 2024-04-25T13:51:40.302 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 0 on host 'smithi078' 2024-04-25T13:51:40.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe21c03d850 msgr2=0x7fe21c03fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe21c03d850 0x7fe21c03fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe2341038e0 tx=0x7fe224010040 comp rx=0 tx=0).stop 2024-04-25T13:51:40.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 msgr2=0x7fe23419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe23419adf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe22800b760 tx=0x7fe228004790 comp rx=0 tx=0).stop 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 shutdown_connections 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe21c03d850 0x7fe21c03fd60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe234104380 0x7fe2341a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2341028b0 0x7fe23419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 --2- 172.21.15.78:0/3735394007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2341039e0 0x7fe23419adf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.303+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 >> 172.21.15.78:0/3735394007 conn(0x7fe2340fdef0 msgr2=0x7fe234108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.304+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 shutdown_connections 2024-04-25T13:51:40.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:51:40.304+0000 7fe23893a700 1 -- 172.21.15.78:0/3735394007 wait complete. 2024-04-25T13:51:40.965 DEBUG:teuthology.orchestra.run.smithi078:osd.0> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T13:51:40.967 INFO:tasks.cephadm:Deploying osd.1 on smithi112 with /dev/vg_nvme/lv_4... 2024-04-25T13:51:40.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:51:41.211 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:51:41.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:41 smithi078 ceph-mon[29163]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:41.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:41.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:41.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:41.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:41 smithi078 ceph-mon[29163]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:51:41.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:41 smithi112 ceph-mon[32617]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:41.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:41.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:41.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:41.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:41 smithi112 ceph-mon[32617]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:51:41.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:41 smithi129 ceph-mon[32569]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:41.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:41.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:41.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:51:41.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:41 smithi129 ceph-mon[32569]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:51:42.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:42 smithi078 ceph-mon[29163]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:51:42.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:42 smithi078 ceph-mon[29163]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:51:42.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:42 smithi078 ceph-mon[29163]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:51:42.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:42 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:42.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:42 smithi078 ceph-mon[29163]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:42.376 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:51:42 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0[43933]: 2024-04-25T13:51:42.272+0000 7f0e54f9b700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T13:51:42.376 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:51:42 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0[43933]: 2024-04-25T13:51:42.274+0000 7f0e50d50700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:51:42.443 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:42 smithi112 ceph-mon[32617]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:51:42.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:42 smithi112 ceph-mon[32617]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:51:42.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:42 smithi112 ceph-mon[32617]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:51:42.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:42 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:42.444 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:42 smithi112 ceph-mon[32617]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:42.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:42 smithi129 ceph-mon[32569]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:51:42.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:42 smithi129 ceph-mon[32569]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:51:42.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:42 smithi129 ceph-mon[32569]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:51:42.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:42 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:42.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:42 smithi129 ceph-mon[32569]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:43.001 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:51:43.001 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:51:43.002 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T13:51:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T13:51:43.002 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0160169 s, 655 MB/s 2024-04-25T13:51:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T13:51:43.297 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:43 smithi112 ceph-mon[32617]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:51:43.297 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:43 smithi112 ceph-mon[32617]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:51:43.297 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:43.297 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:43.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:43 smithi078 ceph-mon[29163]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:51:43.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:43 smithi078 ceph-mon[29163]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:51:43.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:43.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:43 smithi129 ceph-mon[32569]: from='osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:51:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:43 smithi129 ceph-mon[32569]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:51:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:43.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:43.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-04-25T13:51:43.913 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:51:44.304 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] boot 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi078 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.305 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:44 smithi112 ceph-mon[32617]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] boot 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi078 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:44.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:44 smithi129 ceph-mon[32569]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:51:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:51:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: osd.0 [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] boot 2024-04-25T13:51:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:51:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:51:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi078 2024-04-25T13:51:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:51:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:44 smithi078 ceph-mon[29163]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:51:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 -- 172.21.15.112:0/3929650470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23980771a0 msgr2=0x7f239810c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 --2- 172.21.15.112:0/3929650470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23980771a0 0x7f239810c070 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2388009990 tx=0x7f2388009ca0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 -- 172.21.15.112:0/3929650470 shutdown_connections 2024-04-25T13:51:45.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 --2- 172.21.15.112:0/3929650470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f239810c5b0 0x7f239810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 --2- 172.21.15.112:0/3929650470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980787c0 0x7f2398076b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 --2- 172.21.15.112:0/3929650470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23980771a0 0x7f239810c070 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 -- 172.21.15.112:0/3929650470 >> 172.21.15.112:0/3929650470 conn(0x7f23980ff870 msgr2=0x7f2398101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 -- 172.21.15.112:0/3929650470 shutdown_connections 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.365+0000 7f239f91c700 1 -- 172.21.15.112:0/3929650470 wait complete. 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 Processor -- start 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 -- start start 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980771a0 0x7f239819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f239810c5b0 0x7f239819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23981a1b90 0x7f23981a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2398111ff0 con 0x7f23980771a0 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2398111e70 con 0x7f239810c5b0 2024-04-25T13:51:45.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239f91c700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2398112170 con 0x7f23981a1b90 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239ceb7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f239810c5b0 0x7f239819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239ceb7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f239810c5b0 0x7f239819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59600/0 (socket says 172.21.15.112:59600) 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239ceb7700 1 -- 172.21.15.112:0/3230084945 learned_addr learned my addr 172.21.15.112:0/3230084945 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239deb9700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23981a1b90 0x7f23981a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239d6b8700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980771a0 0x7f239819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239ceb7700 1 -- 172.21.15.112:0/3230084945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23981a1b90 msgr2=0x7f23981a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.366+0000 7f239ceb7700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23981a1b90 0x7f23981a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f239ceb7700 1 -- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980771a0 msgr2=0x7f239819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f239ceb7700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980771a0 0x7f239819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f239ceb7700 1 -- 172.21.15.112:0/3230084945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23981a4630 con 0x7f239810c5b0 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f239ceb7700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f239810c5b0 0x7f239819cc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2388006010 tx=0x7f2388009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f238e7fc700 1 -- 172.21.15.112:0/3230084945 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388027070 con 0x7f239810c5b0 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f238e7fc700 1 -- 172.21.15.112:0/3230084945 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2388004140 con 0x7f239810c5b0 2024-04-25T13:51:45.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23981a48c0 con 0x7f239810c5b0 2024-04-25T13:51:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f238e7fc700 1 -- 172.21.15.112:0/3230084945 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388022dd0 con 0x7f239810c5b0 2024-04-25T13:51:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.367+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23981a4df0 con 0x7f239810c5b0 2024-04-25T13:51:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.368+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2398051190 con 0x7f239810c5b0 2024-04-25T13:51:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.369+0000 7f238e7fc700 1 -- 172.21.15.112:0/3230084945 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f238802a070 con 0x7f239810c5b0 2024-04-25T13:51:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.370+0000 7f238e7fc700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23840afb70 0x7f23840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.370+0000 7f238e7fc700 1 -- 172.21.15.112:0/3230084945 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f23880de7b0 con 0x7f239810c5b0 2024-04-25T13:51:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.370+0000 7f239d6b8700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23840afb70 0x7f23840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.371+0000 7f239d6b8700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23840afb70 0x7f23840b2080 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23980784c0 tx=0x7f2394009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.374+0000 7f238e7fc700 1 -- 172.21.15.112:0/3230084945 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2388031080 con 0x7f239810c5b0 2024-04-25T13:51:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:45 smithi112 ceph-mon[32617]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:51:45.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:51:45.586+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f239802e930 con 0x7f23840afb70 2024-04-25T13:51:45.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:45 smithi129 ceph-mon[32569]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:51:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:45 smithi078 ceph-mon[29163]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:51:46.438 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:46 smithi112 ceph-mon[32617]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:46.438 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:51:46.438 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:51:46.438 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:46.438 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:46 smithi112 ceph-mon[32617]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:46 smithi129 ceph-mon[32569]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:51:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:51:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:46 smithi129 ceph-mon[32569]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:46 smithi078 ceph-mon[29163]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:51:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:51:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:51:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:46 smithi078 ceph-mon[29163]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:49.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:49 smithi129 ceph-mon[32569]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:49 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/4148721359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]: dispatch 2024-04-25T13:51:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:49 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]: dispatch 2024-04-25T13:51:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:49 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]': finished 2024-04-25T13:51:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:49 smithi129 ceph-mon[32569]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:51:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:51:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:49 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/948105253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:51:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:49 smithi078 ceph-mon[29163]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:49 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/4148721359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]: dispatch 2024-04-25T13:51:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:49 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]: dispatch 2024-04-25T13:51:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:49 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]': finished 2024-04-25T13:51:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:49 smithi078 ceph-mon[29163]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:51:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:51:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:49 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/948105253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:51:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:49 smithi112 ceph-mon[32617]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:49 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/4148721359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]: dispatch 2024-04-25T13:51:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:49 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]: dispatch 2024-04-25T13:51:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:49 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a258ec36-259b-448b-8103-de5ddfbbcfdc"}]': finished 2024-04-25T13:51:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:49 smithi112 ceph-mon[32617]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:51:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:51:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:49 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/948105253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:51:51.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:51 smithi129 ceph-mon[32569]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:51 smithi078 ceph-mon[29163]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:51.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:51 smithi112 ceph-mon[32617]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:53.310 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:53 smithi112 ceph-mon[32617]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:53.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:53 smithi129 ceph-mon[32569]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:53 smithi078 ceph-mon[29163]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:55.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:55 smithi129 ceph-mon[32569]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:55.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:51:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:55 smithi078 ceph-mon[29163]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:51:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:55 smithi112 ceph-mon[32617]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:51:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:51:56.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:56 smithi112 ceph-mon[32617]: Deploying daemon osd.1 on smithi112 2024-04-25T13:51:56.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:56 smithi129 ceph-mon[32569]: Deploying daemon osd.1 on smithi112 2024-04-25T13:51:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:56 smithi078 ceph-mon[29163]: Deploying daemon osd.1 on smithi112 2024-04-25T13:51:57.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:57 smithi129 ceph-mon[32569]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:57.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:57 smithi078 ceph-mon[29163]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:57.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:57 smithi112 ceph-mon[32617]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:59.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:51:59 smithi129 ceph-mon[32569]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:51:59 smithi078 ceph-mon[29163]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:51:59.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:51:59 smithi112 ceph-mon[32617]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:01 smithi112 ceph-mon[32617]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:01 smithi129 ceph-mon[32569]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:01 smithi078 ceph-mon[29163]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:03 smithi129 ceph-mon[32569]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:03 smithi078 ceph-mon[29163]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:03 smithi112 ceph-mon[32617]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:03.899 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 1 on host 'smithi112' 2024-04-25T13:52:03.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.899+0000 7f238e7fc700 1 -- 172.21.15.112:0/3230084945 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f239802e930 con 0x7f23840afb70 2024-04-25T13:52:03.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.901+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23840afb70 msgr2=0x7f23840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.901+0000 7f239f91c700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23840afb70 0x7f23840b2080 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23980784c0 tx=0x7f2394009040 comp rx=0 tx=0).stop 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f239810c5b0 msgr2=0x7f239819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f239810c5b0 0x7f239819cc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2388006010 tx=0x7f2388009510 comp rx=0 tx=0).stop 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 shutdown_connections 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23840afb70 0x7f23840b2080 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23981a1b90 0x7f23981a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980771a0 0x7f239819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 --2- 172.21.15.112:0/3230084945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f239810c5b0 0x7f239819cc00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 >> 172.21.15.112:0/3230084945 conn(0x7f23980ff870 msgr2=0x7f2398108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 shutdown_connections 2024-04-25T13:52:03.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:03.902+0000 7f239f91c700 1 -- 172.21.15.112:0/3230084945 wait complete. 2024-04-25T13:52:04.661 DEBUG:teuthology.orchestra.run.smithi112:osd.1> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T13:52:04.663 INFO:tasks.cephadm:Deploying osd.2 on smithi129 with /dev/vg_nvme/lv_4... 2024-04-25T13:52:04.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:52:04.921 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:52:04.967 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:04.967 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:04.967 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:04.967 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:04 smithi112 ceph-mon[32617]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:04.967 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:04 smithi112 ceph-mon[32617]: from='osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:52:04.967 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:04 smithi112 ceph-mon[32617]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:52:05.033 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:05.033 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:05.033 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:05.033 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:04 smithi129 ceph-mon[32569]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:05.034 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:04 smithi129 ceph-mon[32569]: from='osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:52:05.034 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:04 smithi129 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:52:05.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:05.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:05.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:05.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:04 smithi078 ceph-mon[29163]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:05.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:04 smithi078 ceph-mon[29163]: from='osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:52:05.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:04 smithi078 ceph-mon[29163]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:52:06.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:05 smithi112 ceph-mon[32617]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:52:06.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:05 smithi112 ceph-mon[32617]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:52:06.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:05 smithi112 ceph-mon[32617]: from='osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:52:06.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:05 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:06.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:05 smithi112 ceph-mon[32617]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:52:06.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:05 smithi129 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:52:06.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:05 smithi129 ceph-mon[32569]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:52:06.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:05 smithi129 ceph-mon[32569]: from='osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:52:06.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:05 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:06.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:05 smithi129 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:52:06.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:05 smithi078 ceph-mon[29163]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:52:06.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:05 smithi078 ceph-mon[29163]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:52:06.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:05 smithi078 ceph-mon[29163]: from='osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:52:06.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:05 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:06.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:05 smithi078 ceph-mon[29163]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:52:06.516 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1[37221]: 2024-04-25T13:52:06.309+0000 7fe4739c5700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T13:52:06.516 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1[37221]: 2024-04-25T13:52:06.315+0000 7fe46b772700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:52:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:52:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:52:06.735 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-25T13:52:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-25T13:52:06.735 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123993 s, 846 MB/s 2024-04-25T13:52:06.735 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-25T13:52:07.113 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:06 smithi129 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:52:07.113 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:06 smithi129 ceph-mon[32569]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:52:07.113 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:07.113 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:07.113 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:06 smithi129 ceph-mon[32569]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:07.113 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:06 smithi129 ceph-mon[32569]: from='osd.1 ' entity='osd.1' 2024-04-25T13:52:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:06 smithi078 ceph-mon[29163]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:52:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:06 smithi078 ceph-mon[29163]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:52:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:06 smithi078 ceph-mon[29163]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:06 smithi078 ceph-mon[29163]: from='osd.1 ' entity='osd.1' 2024-04-25T13:52:07.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-mon[32617]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:52:07.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-mon[32617]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:52:07.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:07.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:07.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-mon[32617]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:52:07.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:06 smithi112 ceph-mon[32617]: from='osd.1 ' entity='osd.1' 2024-04-25T13:52:07.401 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-04-25T13:52:07.644 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] boot 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.009 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.010 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:08.010 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.010 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:08.010 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:08.010 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] boot 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:08.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:08.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:08.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:52:08.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:52:08.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:08.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: osd.1 [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] boot 2024-04-25T13:52:08.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:52:08.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:52:08.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:08.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:08.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:08.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:08.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:09.020 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:08 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi112 2024-04-25T13:52:09.020 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:08 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T13:52:09.020 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:08 smithi129 ceph-mon[32569]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:08 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi112 2024-04-25T13:52:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:08 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T13:52:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:08 smithi078 ceph-mon[29163]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 -- 172.21.15.129:0/710519621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 msgr2=0x7fe46810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 --2- 172.21.15.129:0/710519621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe46810b4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe460009990 tx=0x7fe460009ca0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 -- 172.21.15.129:0/710519621 shutdown_connections 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 --2- 172.21.15.129:0/710519621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe46810b4b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 --2- 172.21.15.129:0/710519621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe468103130 0x7fe468103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 --2- 172.21.15.129:0/710519621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe468104260 0x7fe468104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 -- 172.21.15.129:0/710519621 >> 172.21.15.129:0/710519621 conn(0x7fe4680fe790 msgr2=0x7fe468100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 -- 172.21.15.129:0/710519621 shutdown_connections 2024-04-25T13:52:09.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.254+0000 7fe470257700 1 -- 172.21.15.129:0/710519621 wait complete. 2024-04-25T13:52:09.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 Processor -- start 2024-04-25T13:52:09.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 -- start start 2024-04-25T13:52:09.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe468103130 0x7fe46819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe468104260 0x7fe46819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe4681a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe46810eba0 con 0x7fe468103130 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe46810ea20 con 0x7fe468104260 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe470257700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe46810ed20 con 0x7fe468104c00 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe46e7f4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe4681a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe46e7f4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe4681a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45552/0 (socket says 172.21.15.129:45552) 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.255+0000 7fe46e7f4700 1 -- 172.21.15.129:0/2997871235 learned_addr learned my addr 172.21.15.129:0/2997871235 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46d7f2700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe468104260 0x7fe46819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46dff3700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe468103130 0x7fe46819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46e7f4700 1 -- 172.21.15.129:0/2997871235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe468104260 msgr2=0x7fe46819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46e7f4700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe468104260 0x7fe46819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46e7f4700 1 -- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe468103130 msgr2=0x7fe46819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46e7f4700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe468103130 0x7fe46819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46e7f4700 1 -- 172.21.15.129:0/2997871235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4681a2190 con 0x7fe468104c00 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe46e7f4700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe4681a1ba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4600038a0 tx=0x7fe460009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe45effd700 1 -- 172.21.15.129:0/2997871235 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460027070 con 0x7fe468104c00 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe45effd700 1 -- 172.21.15.129:0/2997871235 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe460022470 con 0x7fe468104c00 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4681a2420 con 0x7fe468104c00 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe45effd700 1 -- 172.21.15.129:0/2997871235 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46000f6a0 con 0x7fe468104c00 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.256+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681a2870 con 0x7fe468104c00 2024-04-25T13:52:09.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.257+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe468051190 con 0x7fe468104c00 2024-04-25T13:52:09.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.258+0000 7fe45effd700 1 -- 172.21.15.129:0/2997871235 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fe46002a070 con 0x7fe468104c00 2024-04-25T13:52:09.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.259+0000 7fe45effd700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4540afaf0 0x7fe4540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.259+0000 7fe45effd700 1 -- 172.21.15.129:0/2997871235 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fe4600de4d0 con 0x7fe468104c00 2024-04-25T13:52:09.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.259+0000 7fe46dff3700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4540afaf0 0x7fe4540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.261+0000 7fe46dff3700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4540afaf0 0x7fe4540b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe468104160 tx=0x7fe464009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:09.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.263+0000 7fe45effd700 1 -- 172.21.15.129:0/2997871235 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe460013070 con 0x7fe468104c00 2024-04-25T13:52:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:08 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi112 2024-04-25T13:52:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:08 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T13:52:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:08 smithi112 ceph-mon[32617]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:09.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:09.469+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe4681a2c50 con 0x7fe4540afaf0 2024-04-25T13:52:10.159 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:09 smithi129 ceph-mon[32569]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:52:10.159 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:10.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:10.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:10.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:09 smithi078 ceph-mon[29163]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:52:10.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:10.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:10.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:10.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:09 smithi112 ceph-mon[32617]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:52:10.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:10.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:10.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:11.139 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:10 smithi129 ceph-mon[32569]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:11.139 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:10 smithi129 ceph-mon[32569]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:10 smithi078 ceph-mon[29163]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:10 smithi078 ceph-mon[29163]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:11.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:10 smithi112 ceph-mon[32617]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:11.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:10 smithi112 ceph-mon[32617]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:13.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:13 smithi129 ceph-mon[32569]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/1838394175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]: dispatch 2024-04-25T13:52:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:13 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]: dispatch 2024-04-25T13:52:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:13 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]': finished 2024-04-25T13:52:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:13 smithi129 ceph-mon[32569]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:52:13.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:13 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:13.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/279847995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:52:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:13 smithi078 ceph-mon[29163]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/1838394175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]: dispatch 2024-04-25T13:52:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:13 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]: dispatch 2024-04-25T13:52:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:13 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]': finished 2024-04-25T13:52:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:13 smithi078 ceph-mon[29163]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:52:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:13 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/279847995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:52:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:13 smithi112 ceph-mon[32617]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/1838394175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]: dispatch 2024-04-25T13:52:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:13 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]: dispatch 2024-04-25T13:52:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:13 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25bc85c2-81d0-479f-8b08-e5c37a158b38"}]': finished 2024-04-25T13:52:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:13 smithi112 ceph-mon[32617]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:52:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:13 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/279847995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:52:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:15 smithi129 ceph-mon[32569]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:15 smithi078 ceph-mon[29163]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:15 smithi112 ceph-mon[32617]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:17.400 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:17 smithi129 ceph-mon[32569]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:17 smithi078 ceph-mon[29163]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:17.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:17 smithi112 ceph-mon[32617]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:19.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:19 smithi129 ceph-mon[32569]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:19 smithi078 ceph-mon[29163]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:19 smithi112 ceph-mon[32617]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:20.400 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:52:20.400 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:52:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:20.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:52:20.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:21.334 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:21 smithi129 ceph-mon[32569]: Deploying daemon osd.2 on smithi129 2024-04-25T13:52:21.334 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:21 smithi129 ceph-mon[32569]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:21 smithi078 ceph-mon[29163]: Deploying daemon osd.2 on smithi129 2024-04-25T13:52:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:21 smithi078 ceph-mon[29163]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:21 smithi112 ceph-mon[32617]: Deploying daemon osd.2 on smithi129 2024-04-25T13:52:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:21 smithi112 ceph-mon[32617]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:23.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:23 smithi129 ceph-mon[32569]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:23 smithi078 ceph-mon[29163]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:23.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:23 smithi112 ceph-mon[32617]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:25.354 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:25 smithi129 ceph-mon[32569]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:25.354 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:25 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:25.354 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:25 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:25.355 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:25 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:25 smithi078 ceph-mon[29163]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:25 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:25 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:25 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:25.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:25 smithi112 ceph-mon[32617]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:25.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:25 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:25.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:25 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:25.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:25 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:27.198 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:27 smithi129 ceph-mon[32569]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:27.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:27 smithi078 ceph-mon[29163]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:27 smithi112 ceph-mon[32617]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.037+0000 7fe45effd700 1 -- 172.21.15.129:0/2997871235 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe4681a2c50 con 0x7fe4540afaf0 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 2 on host 'smithi129' 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4540afaf0 msgr2=0x7fe4540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4540afaf0 0x7fe4540b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe468104160 tx=0x7fe464009040 comp rx=0 tx=0).stop 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 msgr2=0x7fe4681a1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe4681a1ba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe4600038a0 tx=0x7fe460009f90 comp rx=0 tx=0).stop 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 shutdown_connections 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4540afaf0 0x7fe4540b2000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe468104c00 0x7fe4681a1ba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe468103130 0x7fe46819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 --2- 172.21.15.129:0/2997871235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe468104260 0x7fe46819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 >> 172.21.15.129:0/2997871235 conn(0x7fe4680fe790 msgr2=0x7fe4680fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 shutdown_connections 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:52:28.039+0000 7fe470257700 1 -- 172.21.15.129:0/2997871235 wait complete. 2024-04-25T13:52:28.674 DEBUG:teuthology.orchestra.run.smithi129:osd.2> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T13:52:28.677 INFO:tasks.cephadm:Deploying osd.3 on smithi078 with /dev/vg_nvme/lv_3... 2024-04-25T13:52:28.677 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:52:28.920 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:52:29.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:28 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:28 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:28 smithi078 ceph-mon[29163]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:29.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:28 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:28 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:28 smithi078 ceph-mon[29163]: from='osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:52:29.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:28 smithi078 ceph-mon[29163]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:52:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:28 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:28 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:28 smithi112 ceph-mon[32617]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:28 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:28 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:28 smithi112 ceph-mon[32617]: from='osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:52:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:28 smithi112 ceph-mon[32617]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:52:29.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:28 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:28 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:28 smithi129 ceph-mon[32569]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:28 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:28 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:28 smithi129 ceph-mon[32569]: from='osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:52:29.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:28 smithi129 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:52:30.153 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:30 smithi129 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:52:30.153 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:30 smithi129 ceph-mon[32569]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:52:30.153 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:30 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:30.153 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:30 smithi129 ceph-mon[32569]: from='osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:52:30.154 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:30 smithi129 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:52:30.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:30 smithi078 ceph-mon[29163]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:52:30.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:30 smithi078 ceph-mon[29163]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:52:30.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:30 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:30.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:30 smithi078 ceph-mon[29163]: from='osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:52:30.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:30 smithi078 ceph-mon[29163]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:52:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:30 smithi112 ceph-mon[32617]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:52:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:30 smithi112 ceph-mon[32617]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:52:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:30 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:30 smithi112 ceph-mon[32617]: from='osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:52:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:30 smithi112 ceph-mon[32617]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:52:30.913 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2[37676]: 2024-04-25T13:52:30.540+0000 7f631b05b700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T13:52:30.914 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2[37676]: 2024-04-25T13:52:30.544+0000 7f6312e08700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:52:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:52:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:52:30.983 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T13:52:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T13:52:30.983 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131054 s, 800 MB/s 2024-04-25T13:52:30.983 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T13:52:31.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:31 smithi078 ceph-mon[29163]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:31.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:31 smithi078 ceph-mon[29163]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:52:31.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:31 smithi078 ceph-mon[29163]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:52:31.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:31.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:31.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:31 smithi078 ceph-mon[29163]: from='osd.2 ' entity='osd.2' 2024-04-25T13:52:31.280 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:31 smithi129 ceph-mon[32569]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:31.280 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:31 smithi129 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:52:31.280 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:31 smithi129 ceph-mon[32569]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:52:31.280 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:31.280 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:31.280 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:31 smithi129 ceph-mon[32569]: from='osd.2 ' entity='osd.2' 2024-04-25T13:52:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:31 smithi112 ceph-mon[32617]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:31 smithi112 ceph-mon[32617]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:52:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:31 smithi112 ceph-mon[32617]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:52:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:31 smithi112 ceph-mon[32617]: from='osd.2 ' entity='osd.2' 2024-04-25T13:52:31.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-04-25T13:52:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:52:32.390 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:52:32.390 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] boot 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:32.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:32.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:52:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:52:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] boot 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:52:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:52:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: osd.2 [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] boot 2024-04-25T13:52:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:52:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:52:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:33.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:33 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi129 2024-04-25T13:52:33.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:33 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-25T13:52:33.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:33 smithi078 ceph-mon[29163]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:33.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:33 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:52:33.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:33 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi129 2024-04-25T13:52:33.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:33 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-25T13:52:33.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:33 smithi129 ceph-mon[32569]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:33.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:33 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:52:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:33 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi129 2024-04-25T13:52:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:33 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-25T13:52:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:33 smithi112 ceph-mon[32617]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:52:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:33 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 -- 172.21.15.78:0/145376178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744103100 msgr2=0x7fa744103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 --2- 172.21.15.78:0/145376178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744103100 0x7fa744103510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa734009990 tx=0x7fa734009ca0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 -- 172.21.15.78:0/145376178 shutdown_connections 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 --2- 172.21.15.78:0/145376178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa744104bd0 0x7fa74410b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 --2- 172.21.15.78:0/145376178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744103100 0x7fa744103510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 --2- 172.21.15.78:0/145376178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa744104230 0x7fa744104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 -- 172.21.15.78:0/145376178 >> 172.21.15.78:0/145376178 conn(0x7fa7440fe760 msgr2=0x7fa744100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.568+0000 7fa74902a700 1 -- 172.21.15.78:0/145376178 shutdown_connections 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.569+0000 7fa74902a700 1 -- 172.21.15.78:0/145376178 wait complete. 2024-04-25T13:52:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.569+0000 7fa74902a700 1 Processor -- start 2024-04-25T13:52:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.569+0000 7fa74902a700 1 -- start start 2024-04-25T13:52:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74902a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa744103100 0x7fa74419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74902a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744104230 0x7fa74419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74902a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa744104bd0 0x7fa7441a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74902a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa74410eb40 con 0x7fa744104230 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74902a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa74410e9c0 con 0x7fa744103100 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74902a700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa74410ecc0 con 0x7fa744104bd0 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744104230 0x7fa74419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744104230 0x7fa74419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32970/0 (socket says 172.21.15.78:32970) 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 -- 172.21.15.78:0/2708233032 learned_addr learned my addr 172.21.15.78:0/2708233032 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa742d9d700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa744103100 0x7fa74419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74359e700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa744104bd0 0x7fa7441a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 -- 172.21.15.78:0/2708233032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa744104bd0 msgr2=0x7fa7441a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa744104bd0 0x7fa7441a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 -- 172.21.15.78:0/2708233032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa744103100 msgr2=0x7fa74419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa744103100 0x7fa74419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 -- 172.21.15.78:0/2708233032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7441a22c0 con 0x7fa744104230 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74359e700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa744104bd0 0x7fa7441a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa74259c700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744104230 0x7fa74419ad30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa73800d980 tx=0x7fa73800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.570+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738016070 con 0x7fa744104230 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.571+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7441a25b0 con 0x7fa744104230 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.571+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa738004830 con 0x7fa744104230 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.571+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7380054c0 con 0x7fa744104230 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.571+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7441a2af0 con 0x7fa744104230 2024-04-25T13:52:33.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.571+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa744051190 con 0x7fa744104230 2024-04-25T13:52:33.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.573+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fa738004220 con 0x7fa744104230 2024-04-25T13:52:33.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.574+0000 7fa72bfff700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa72c0afaf0 0x7fa72c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.574+0000 7fa742d9d700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa72c0afaf0 0x7fa72c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.574+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fa7380ddb10 con 0x7fa744104230 2024-04-25T13:52:33.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.574+0000 7fa742d9d700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa72c0afaf0 0x7fa72c0b2000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa7340038a0 tx=0x7fa73401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.578+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa73801f020 con 0x7fa744104230 2024-04-25T13:52:33.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:33.783+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa7441a2d50 con 0x7fa72c0afaf0 2024-04-25T13:52:34.358 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:52:34.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 ceph-mon[29163]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:52:34.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:52:34.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:34.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:34.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:34.359 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48246]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:52:34.359 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.359 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48246]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.359 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48246]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.359 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48249]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:52:34.360 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48249]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.360 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48249]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.360 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48249]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41285]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41285]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41285]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41285]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41288]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41288]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41288]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.397 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41288]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.398 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:52:34.398 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 ceph-mon[32569]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:52:34.398 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:52:34.398 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:34.398 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:34.398 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:34.434 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:52:34.434 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 ceph-mon[32617]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:52:34.434 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:52:34.434 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:34.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:34.435 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41360]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41360]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41360]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41360]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41363]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41363]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41363]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.435 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41363]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41291]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:52:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41291]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41291]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41291]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41294]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:52:34.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41294]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41294]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:34 smithi129 sudo[41294]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48308]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48308]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:34 smithi078 sudo[48311]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41366]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:52:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41366]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41366]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41366]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41369]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:52:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41369]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:52:34.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41369]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:52:34.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:34 smithi112 sudo[41369]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:52:35.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:35.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:35.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:52:35.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:52:35.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:52:35.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:35 smithi078 ceph-mon[29163]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:35 smithi129 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:52:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:52:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:52:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:52:35.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:52:35.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:52:35.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:35 smithi112 ceph-mon[32617]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:52:36.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:36.120+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa7380a3740 con 0x7fa744104230 2024-04-25T13:52:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:36 smithi129 ceph-mon[32569]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:52:36.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:36 smithi078 ceph-mon[29163]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:52:36.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:36 smithi112 ceph-mon[32617]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: Cluster is now healthy 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1313945179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9b02621-7cdb-480e-863d-09bfc670da4f"}]: dispatch 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1313945179' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9b02621-7cdb-480e-863d-09bfc670da4f"}]': finished 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:52:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:37 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:52:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: Cluster is now healthy 2024-04-25T13:52:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:52:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1313945179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9b02621-7cdb-480e-863d-09bfc670da4f"}]: dispatch 2024-04-25T13:52:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1313945179' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9b02621-7cdb-480e-863d-09bfc670da4f"}]': finished 2024-04-25T13:52:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:52:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:37 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: Cluster is now healthy 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1313945179' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9b02621-7cdb-480e-863d-09bfc670da4f"}]: dispatch 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1313945179' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9b02621-7cdb-480e-863d-09bfc670da4f"}]': finished 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T13:52:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:37 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:38.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:38 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3564662255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:52:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:38 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3564662255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:52:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:38 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3564662255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:52:39.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:39 smithi129 ceph-mon[32569]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:39 smithi078 ceph-mon[29163]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:39 smithi112 ceph-mon[32617]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:41.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:41 smithi129 ceph-mon[32569]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:41 smithi078 ceph-mon[29163]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:41 smithi112 ceph-mon[32617]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:43.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:43 smithi129 ceph-mon[32569]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:43.421 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:43 smithi078 ceph-mon[29163]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:43 smithi112 ceph-mon[32617]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:44.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:52:44.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:52:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:52:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:45.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:45 smithi078 ceph-mon[29163]: Deploying daemon osd.3 on smithi078 2024-04-25T13:52:45.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:45 smithi078 ceph-mon[29163]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:45.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:45 smithi129 ceph-mon[32569]: Deploying daemon osd.3 on smithi078 2024-04-25T13:52:45.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:45 smithi129 ceph-mon[32569]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:45 smithi112 ceph-mon[32617]: Deploying daemon osd.3 on smithi078 2024-04-25T13:52:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:45 smithi112 ceph-mon[32617]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:47.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:47 smithi078 ceph-mon[29163]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:47 smithi129 ceph-mon[32569]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:47 smithi112 ceph-mon[32617]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:49.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:49 smithi078 ceph-mon[29163]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:49.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:49 smithi129 ceph-mon[32569]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:49 smithi112 ceph-mon[32617]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:51.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:50 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:51.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:50 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:51.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:50 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:51.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:50 smithi129 ceph-mon[32569]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:51.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:50 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:51.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:50 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:51.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:50 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:51.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:50 smithi078 ceph-mon[29163]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:51.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:50 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:51.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:50 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:51.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:50 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:51.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:50 smithi112 ceph-mon[32617]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:53 smithi078 ceph-mon[29163]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:53.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.342+0000 7fa72bfff700 1 -- 172.21.15.78:0/2708233032 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7441a2d50 con 0x7fa72c0afaf0 2024-04-25T13:52:53.342 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 3 on host 'smithi078' 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa72c0afaf0 msgr2=0x7fa72c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa72c0afaf0 0x7fa72c0b2000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa7340038a0 tx=0x7fa73401a040 comp rx=0 tx=0).stop 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744104230 msgr2=0x7fa74419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744104230 0x7fa74419ad30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa73800d980 tx=0x7fa73800dc90 comp rx=0 tx=0).stop 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 shutdown_connections 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa72c0afaf0 0x7fa72c0b2000 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa744104bd0 0x7fa7441a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa744103100 0x7fa74419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 --2- 172.21.15.78:0/2708233032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa744104230 0x7fa74419ad30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:53.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 >> 172.21.15.78:0/2708233032 conn(0x7fa7440fe760 msgr2=0x7fa7441000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:53.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 shutdown_connections 2024-04-25T13:52:53.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:52:53.344+0000 7fa74902a700 1 -- 172.21.15.78:0/2708233032 wait complete. 2024-04-25T13:52:53.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:53 smithi129 ceph-mon[32569]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:53.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:53 smithi112 ceph-mon[32617]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:54.048 DEBUG:teuthology.orchestra.run.smithi078:osd.3> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T13:52:54.050 INFO:tasks.cephadm:Deploying osd.4 on smithi112 with /dev/vg_nvme/lv_3... 2024-04-25T13:52:54.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:52:54.295 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:52:54.417 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:54 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.417 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:54 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.417 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:54 smithi112 ceph-mon[32617]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:52:54.417 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:54 smithi112 ceph-mon[32617]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:54.418 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:54 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.418 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:54 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:54 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:54 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:54 smithi129 ceph-mon[32569]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:52:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:54 smithi129 ceph-mon[32569]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:54.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:54 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:54 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:54 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:54 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:54 smithi078 ceph-mon[29163]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:52:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:54 smithi078 ceph-mon[29163]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:54 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:54 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:55.577 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:55 smithi112 ceph-mon[32617]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:52:55.578 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:55 smithi112 ceph-mon[32617]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:52:55.578 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:55 smithi112 ceph-mon[32617]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:52:55.578 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:55.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:55 smithi129 ceph-mon[32569]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:52:55.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:55 smithi129 ceph-mon[32569]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:52:55.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:55 smithi129 ceph-mon[32569]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:52:55.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:55 smithi078 ceph-mon[29163]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:52:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:55 smithi078 ceph-mon[29163]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:52:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:55 smithi078 ceph-mon[29163]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:52:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:56.024 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 13:52:55 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3[50449]: 2024-04-25T13:52:55.775+0000 7f6a57aae700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T13:52:56.024 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 13:52:55 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3[50449]: 2024-04-25T13:52:55.781+0000 7f6a4f85b700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:52:56.148 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:52:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:52:56.149 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T13:52:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T13:52:56.149 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014221 s, 737 MB/s 2024-04-25T13:52:56.149 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T13:52:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:56 smithi129 ceph-mon[32569]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:52:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:56 smithi129 ceph-mon[32569]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:52:56.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:56 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:56.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:56 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:56.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:56 smithi129 ceph-mon[32569]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' 2024-04-25T13:52:56.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:56 smithi129 ceph-mon[32569]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:56 smithi078 ceph-mon[29163]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:52:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:56 smithi078 ceph-mon[29163]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:52:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:56 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:56 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:56 smithi078 ceph-mon[29163]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' 2024-04-25T13:52:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:56 smithi078 ceph-mon[29163]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:56.751 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:56 smithi112 ceph-mon[32617]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:52:56.751 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:56 smithi112 ceph-mon[32617]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:52:56.751 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:56 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:56.751 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:56 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:56.751 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:56 smithi112 ceph-mon[32617]: from='osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666]' entity='osd.3' 2024-04-25T13:52:56.751 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:56 smithi112 ceph-mon[32617]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:52:56.778 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-04-25T13:52:57.013 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:52:57.429 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:57 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:52:57.429 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:57 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:52:57.430 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:57.430 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:57 smithi112 ceph-mon[32617]: osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] boot 2024-04-25T13:52:57.430 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:57 smithi112 ceph-mon[32617]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T13:52:57.430 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:57.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:57 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:52:57.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:57 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:52:57.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:57.629 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:57 smithi078 ceph-mon[29163]: osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] boot 2024-04-25T13:52:57.629 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:57 smithi078 ceph-mon[29163]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T13:52:57.629 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:57.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:57 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:52:57.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:57 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:52:57.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:57.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:57 smithi129 ceph-mon[32569]: osd.3 [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] boot 2024-04-25T13:52:57.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:57 smithi129 ceph-mon[32569]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T13:52:57.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:52:58.341 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:52:58.341 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi078 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:58.342 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.343 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:58 smithi112 ceph-mon[32617]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:52:58.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1897444757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18105340 msgr2=0x7fde18105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1897444757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18105340 0x7fde18105750 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fde04009990 tx=0x7fde04009ca0 comp rx=0 tx=0).stop 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1897444757 shutdown_connections 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1897444757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde18105ce0 0x7fde1810c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1897444757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde18104210 0x7fde18104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1897444757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18105340 0x7fde18105750 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1897444757 >> 172.21.15.112:0/1897444757 conn(0x7fde180ff870 msgr2=0x7fde18101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1897444757 shutdown_connections 2024-04-25T13:52:58.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.590+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1897444757 wait complete. 2024-04-25T13:52:58.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 Processor -- start 2024-04-25T13:52:58.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 -- start start 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18104210 0x7fde1819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde18105340 0x7fde1819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde18105ce0 0x7fde181a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde1810fc70 con 0x7fde18105340 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde1810faf0 con 0x7fde18104210 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1f0ee700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde1810fdf0 con 0x7fde18105ce0 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18104210 0x7fde1819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18104210 0x7fde1819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46896/0 (socket says 172.21.15.112:46896) 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 -- 172.21.15.112:0/1450806989 learned_addr learned my addr 172.21.15.112:0/1450806989 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde17fff700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde18105340 0x7fde1819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1d68b700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde18105ce0 0x7fde181a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 -- 172.21.15.112:0/1450806989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde18105ce0 msgr2=0x7fde181a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:58.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde18105ce0 0x7fde181a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 -- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde18105340 msgr2=0x7fde1819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde18105340 0x7fde1819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.591+0000 7fde1ce8a700 1 -- 172.21.15.112:0/1450806989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde181a22b0 con 0x7fde18104210 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.592+0000 7fde1ce8a700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18104210 0x7fde1819a750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d950 tx=0x7fde0c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.592+0000 7fde15ffb700 1 -- 172.21.15.112:0/1450806989 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c016070 con 0x7fde18104210 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.592+0000 7fde15ffb700 1 -- 172.21.15.112:0/1450806989 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0c00be00 con 0x7fde18104210 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.592+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde181a25a0 con 0x7fde18104210 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.592+0000 7fde15ffb700 1 -- 172.21.15.112:0/1450806989 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00f3c0 con 0x7fde18104210 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.592+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde18076ba0 con 0x7fde18104210 2024-04-25T13:52:58.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.593+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde18051190 con 0x7fde18104210 2024-04-25T13:52:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.595+0000 7fde15ffb700 1 -- 172.21.15.112:0/1450806989 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fde0c019070 con 0x7fde18104210 2024-04-25T13:52:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.596+0000 7fde15ffb700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fde080b3f90 0x7fde080b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.596+0000 7fde15ffb700 1 -- 172.21.15.112:0/1450806989 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fde0c0dd8e0 con 0x7fde18104210 2024-04-25T13:52:58.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.596+0000 7fde17fff700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fde080b3f90 0x7fde080b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:58.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.596+0000 7fde17fff700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fde080b3f90 0x7fde080b64a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde0400b5c0 tx=0x7fde0401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:58.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.601+0000 7fde15ffb700 1 -- 172.21.15.112:0/1450806989 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde0c020020 con 0x7fde18104210 2024-04-25T13:52:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:52:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi078 2024-04-25T13:52:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:58 smithi129 ceph-mon[32569]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:52:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T13:52:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi078 2024-04-25T13:52:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:52:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:58 smithi078 ceph-mon[29163]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:52:58.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:52:58.806+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fde18063ad0 con 0x7fde080b3f90 2024-04-25T13:52:59.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:59 smithi078 ceph-mon[29163]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:52:59.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:59 smithi078 ceph-mon[29163]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:59.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:59 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:59.622 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:59 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:59.622 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:52:59 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:59 smithi129 ceph-mon[32569]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:52:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:59 smithi129 ceph-mon[32569]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:59 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:59 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:52:59 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:52:59.680 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:59 smithi112 ceph-mon[32617]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:52:59.680 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:59 smithi112 ceph-mon[32617]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:52:59.680 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:59 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:52:59.680 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:59 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:52:59.680 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:52:59 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:00.603 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:00 smithi112 ceph-mon[32617]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:00.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:00 smithi078 ceph-mon[29163]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:00 smithi129 ceph-mon[32569]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:01.802 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:01 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/2389797536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]: dispatch 2024-04-25T13:53:01.802 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:01 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]: dispatch 2024-04-25T13:53:01.802 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:01 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]': finished 2024-04-25T13:53:01.802 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:01 smithi078 ceph-mon[29163]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T13:53:01.803 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:01.860 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:01 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/2389797536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]: dispatch 2024-04-25T13:53:01.860 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:01 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]: dispatch 2024-04-25T13:53:01.860 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:01 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]': finished 2024-04-25T13:53:01.860 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:01 smithi112 ceph-mon[32617]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T13:53:01.861 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:01 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/2389797536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]: dispatch 2024-04-25T13:53:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:01 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]: dispatch 2024-04-25T13:53:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:01 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b298aa83-5099-4f31-a030-f3b449241d55"}]': finished 2024-04-25T13:53:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:01 smithi129 ceph-mon[32569]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T13:53:01.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:02.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:02 smithi129 ceph-mon[32569]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:02.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:02 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/1748574843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:02 smithi078 ceph-mon[29163]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:02 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/1748574843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:02 smithi112 ceph-mon[32617]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/1748574843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:05.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:05 smithi129 ceph-mon[32569]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T13:53:05.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:05 smithi078 ceph-mon[29163]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T13:53:05.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:05 smithi112 ceph-mon[32617]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T13:53:07.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:07 smithi129 ceph-mon[32569]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:53:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:07 smithi078 ceph-mon[29163]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:53:07.500 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:07 smithi112 ceph-mon[32617]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:53:09.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:09 smithi129 ceph-mon[32569]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T13:53:09.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:53:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:09.449 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:09 smithi112 ceph-mon[32617]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T13:53:09.450 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:53:09.450 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:09.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:09 smithi078 ceph-mon[29163]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T13:53:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:53:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:10.211 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:10 smithi112 ceph-mon[32617]: Deploying daemon osd.4 on smithi112 2024-04-25T13:53:10.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:10 smithi129 ceph-mon[32569]: Deploying daemon osd.4 on smithi112 2024-04-25T13:53:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:10 smithi078 ceph-mon[29163]: Deploying daemon osd.4 on smithi112 2024-04-25T13:53:11.226 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:11 smithi112 ceph-mon[32617]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:53:11.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:11 smithi129 ceph-mon[32569]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:53:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:11 smithi078 ceph-mon[29163]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:53:13.333 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:13 smithi112 ceph-mon[32617]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T13:53:13.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:13 smithi129 ceph-mon[32569]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T13:53:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:13 smithi078 ceph-mon[29163]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T13:53:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:15 smithi129 ceph-mon[32569]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:53:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:15.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:15 smithi112 ceph-mon[32617]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:53:15.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:15.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:15.460 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:15 smithi078 ceph-mon[29163]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:53:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:17.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:17 smithi129 ceph-mon[32569]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:17.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:17 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:17 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:17 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:17.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:17 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:17.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:17 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:17 smithi078 ceph-mon[29163]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:17 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:17 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:17 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:17 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:17 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.478 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:17 smithi112 ceph-mon[32617]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:17.478 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:17 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.478 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:17 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.478 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:17 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:17.479 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:17 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:17.479 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:17 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:17.875 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 4 on host 'smithi112' 2024-04-25T13:53:17.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.875+0000 7fde15ffb700 1 -- 172.21.15.112:0/1450806989 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fde18063ad0 con 0x7fde080b3f90 2024-04-25T13:53:17.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fde080b3f90 msgr2=0x7fde080b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fde080b3f90 0x7fde080b64a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde0400b5c0 tx=0x7fde0401a040 comp rx=0 tx=0).stop 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18104210 msgr2=0x7fde1819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18104210 0x7fde1819a750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00d950 tx=0x7fde0c00dc60 comp rx=0 tx=0).stop 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 shutdown_connections 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fde080b3f90 0x7fde080b64a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde18105ce0 0x7fde181a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde18104210 0x7fde1819a750 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 --2- 172.21.15.112:0/1450806989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde18105340 0x7fde1819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 >> 172.21.15.112:0/1450806989 conn(0x7fde180ff870 msgr2=0x7fde18108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 shutdown_connections 2024-04-25T13:53:17.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:53:17.878+0000 7fde1f0ee700 1 -- 172.21.15.112:0/1450806989 wait complete. 2024-04-25T13:53:18.484 DEBUG:teuthology.orchestra.run.smithi112:osd.4> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T13:53:18.487 INFO:tasks.cephadm:Deploying osd.5 on smithi129 with /dev/vg_nvme/lv_3... 2024-04-25T13:53:18.487 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:53:18.731 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:53:19.071 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:19.071 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:19.071 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:18 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:19.071 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:18 smithi129 ceph-mon[32569]: from='osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:53:19.071 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:18 smithi129 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:53:19.071 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:18 smithi129 ceph-mon[32569]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:18 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:18 smithi078 ceph-mon[29163]: from='osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:53:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:18 smithi078 ceph-mon[29163]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:53:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:18 smithi078 ceph-mon[29163]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:18 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:18 smithi112 ceph-mon[32617]: from='osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:53:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:18 smithi112 ceph-mon[32617]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:53:19.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:18 smithi112 ceph-mon[32617]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:19.960 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:19 smithi112 ceph-mon[32617]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:53:19.960 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:19 smithi112 ceph-mon[32617]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T13:53:19.960 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:19 smithi112 ceph-mon[32617]: from='osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:53:19.960 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:19 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:19.960 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:19 smithi112 ceph-mon[32617]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:53:20.030 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:19 smithi129 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:53:20.031 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:19 smithi129 ceph-mon[32569]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T13:53:20.031 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:19 smithi129 ceph-mon[32569]: from='osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:53:20.031 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:19 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:20.031 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:19 smithi129 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:53:20.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:19 smithi078 ceph-mon[29163]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:53:20.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:19 smithi078 ceph-mon[29163]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T13:53:20.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:19 smithi078 ceph-mon[29163]: from='osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:53:20.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:19 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:20.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:19 smithi078 ceph-mon[29163]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:53:20.645 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:53:20.645 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:53:20.645 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-25T13:53:20.645 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-25T13:53:20.645 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123835 s, 847 MB/s 2024-04-25T13:53:20.645 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-25T13:53:20.766 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4[44177]: 2024-04-25T13:53:20.308+0000 7f6928671700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T13:53:20.766 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4[44177]: 2024-04-25T13:53:20.317+0000 7f692041e700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:53:20.985 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:20 smithi129 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:53:20.985 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:20 smithi129 ceph-mon[32569]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:53:20.985 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:20.985 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:20.985 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:20 smithi129 ceph-mon[32569]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:20.985 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:20 smithi129 ceph-mon[32569]: from='osd.4 ' entity='osd.4' 2024-04-25T13:53:21.132 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-mon[32617]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:53:21.132 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-mon[32617]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:53:21.132 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:21.132 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:21.132 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-mon[32617]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:21.132 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:20 smithi112 ceph-mon[32617]: from='osd.4 ' entity='osd.4' 2024-04-25T13:53:21.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:20 smithi078 ceph-mon[29163]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:53:21.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:20 smithi078 ceph-mon[29163]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T13:53:21.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:21.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:21.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:20 smithi078 ceph-mon[29163]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:53:21.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:20 smithi078 ceph-mon[29163]: from='osd.4 ' entity='osd.4' 2024-04-25T13:53:21.314 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-04-25T13:53:21.561 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] boot 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:22.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.165 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:22.165 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:22.165 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:53:22.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:53:22.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:22.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] boot 2024-04-25T13:53:22.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T13:53:22.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:22.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:53:22.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:53:22.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:22.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: osd.4 [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] boot 2024-04-25T13:53:22.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T13:53:22.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:22.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:22.891 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:22 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi112 2024-04-25T13:53:22.891 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:22 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T13:53:22.891 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:22 smithi129 ceph-mon[32569]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:23.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 -- 172.21.15.129:0/2063377483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c104210 msgr2=0x7fb23c104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:23.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 --2- 172.21.15.129:0/2063377483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c104210 0x7fb23c104620 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009990 tx=0x7fb22c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:53:23.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 -- 172.21.15.129:0/2063377483 shutdown_connections 2024-04-25T13:53:23.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 --2- 172.21.15.129:0/2063377483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23c105ce0 0x7fb23c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:23.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 --2- 172.21.15.129:0/2063377483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c104210 0x7fb23c104620 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:23.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 --2- 172.21.15.129:0/2063377483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c105340 0x7fb23c105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:23.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 -- 172.21.15.129:0/2063377483 >> 172.21.15.129:0/2063377483 conn(0x7fb23c0ff870 msgr2=0x7fb23c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:23.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 -- 172.21.15.129:0/2063377483 shutdown_connections 2024-04-25T13:53:23.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.100+0000 7fb242976700 1 -- 172.21.15.129:0/2063377483 wait complete. 2024-04-25T13:53:23.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 Processor -- start 2024-04-25T13:53:23.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 -- start start 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c104210 0x7fb23c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c105340 0x7fb23c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23c105ce0 0x7fb23c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10fc60 con 0x7fb23c104210 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10fae0 con 0x7fb23c105ce0 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb242976700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c10fde0 con 0x7fb23c105340 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb23b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c105340 0x7fb23c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb23b7fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c105340 0x7fb23c19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58856/0 (socket says 172.21.15.129:58856) 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.101+0000 7fb23b7fe700 1 -- 172.21.15.129:0/1208935468 learned_addr learned my addr 172.21.15.129:0/1208935468 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:53:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb240f13700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23c105ce0 0x7fb23c1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23bfff700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c104210 0x7fb23c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23b7fe700 1 -- 172.21.15.129:0/1208935468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23c105ce0 msgr2=0x7fb23c1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23b7fe700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23c105ce0 0x7fb23c1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23b7fe700 1 -- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c104210 msgr2=0x7fb23c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23b7fe700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c104210 0x7fb23c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23b7fe700 1 -- 172.21.15.129:0/1208935468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c1a2270 con 0x7fb23c105340 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb240f13700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23c105ce0 0x7fb23c1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23bfff700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c104210 0x7fb23c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb23b7fe700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c105340 0x7fb23c19ace0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c9b0 tx=0x7fb23000cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb2397fa700 1 -- 172.21.15.129:0/1208935468 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230007b00 con 0x7fb23c105340 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c1a2560 con 0x7fb23c105340 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb2397fa700 1 -- 172.21.15.129:0/1208935468 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230007ca0 con 0x7fb23c105340 2024-04-25T13:53:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb2397fa700 1 -- 172.21.15.129:0/1208935468 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230018730 con 0x7fb23c105340 2024-04-25T13:53:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.102+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a2aa0 con 0x7fb23c105340 2024-04-25T13:53:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.103+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c051190 con 0x7fb23c105340 2024-04-25T13:53:23.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.105+0000 7fb2397fa700 1 -- 172.21.15.129:0/1208935468 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb23000f450 con 0x7fb23c105340 2024-04-25T13:53:23.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.106+0000 7fb2397fa700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2240afb80 0x7fb2240b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:23.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.106+0000 7fb2397fa700 1 -- 172.21.15.129:0/1208935468 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fb2300ddef0 con 0x7fb23c105340 2024-04-25T13:53:23.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.106+0000 7fb23bfff700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2240afb80 0x7fb2240b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:23.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.107+0000 7fb23bfff700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2240afb80 0x7fb2240b2090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0038a0 tx=0x7fb22c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:23.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.110+0000 7fb2397fa700 1 -- 172.21.15.129:0/1208935468 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb230014030 con 0x7fb23c105340 2024-04-25T13:53:23.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:22 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi112 2024-04-25T13:53:23.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:22 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T13:53:23.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:22 smithi078 ceph-mon[29163]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:23.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:22 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi112 2024-04-25T13:53:23.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:22 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T13:53:23.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:22 smithi112 ceph-mon[32617]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:23.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:23.315+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb23c063ad0 con 0x7fb2240afb80 2024-04-25T13:53:24.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:23 smithi129 ceph-mon[32569]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T13:53:24.165 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:53:24.165 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:53:24.165 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:24.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:23 smithi078 ceph-mon[29163]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T13:53:24.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:53:24.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:53:24.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:24.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:23 smithi112 ceph-mon[32617]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T13:53:24.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:53:24.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:53:24.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:25.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:24 smithi129 ceph-mon[32569]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:53:25.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:24 smithi129 ceph-mon[32569]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:25.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:24 smithi078 ceph-mon[29163]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:53:25.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:24 smithi078 ceph-mon[29163]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:25.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:24 smithi112 ceph-mon[32617]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:53:25.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:24 smithi112 ceph-mon[32617]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:26 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/1865926918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]: dispatch 2024-04-25T13:53:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:26 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]: dispatch 2024-04-25T13:53:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:26 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]': finished 2024-04-25T13:53:26.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:26 smithi129 ceph-mon[32569]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T13:53:26.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:26 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/1865926918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]: dispatch 2024-04-25T13:53:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:26 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]: dispatch 2024-04-25T13:53:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:26 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]': finished 2024-04-25T13:53:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:26 smithi078 ceph-mon[29163]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T13:53:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:26 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/1865926918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]: dispatch 2024-04-25T13:53:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:26 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]: dispatch 2024-04-25T13:53:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:26 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5283d1e6-8c10-4e92-8c1c-baf015823098"}]': finished 2024-04-25T13:53:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:26 smithi112 ceph-mon[32617]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T13:53:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:27 smithi129 ceph-mon[32569]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:27 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/3886575573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:27.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:27 smithi078 ceph-mon[29163]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:27.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:27 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/3886575573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:27 smithi112 ceph-mon[32617]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:27 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/3886575573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:29.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:29 smithi129 ceph-mon[32569]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:29 smithi078 ceph-mon[29163]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:29 smithi112 ceph-mon[32617]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:31.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:31 smithi129 ceph-mon[32569]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:31 smithi078 ceph-mon[29163]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:31 smithi112 ceph-mon[32617]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:33.312 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:33 smithi129 ceph-mon[32569]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:33 smithi078 ceph-mon[29163]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:33 smithi112 ceph-mon[32617]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:34.428 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:53:34.428 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:53:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:34.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:53:34.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:35.424 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:35 smithi129 ceph-mon[32569]: Deploying daemon osd.5 on smithi129 2024-04-25T13:53:35.424 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:35 smithi129 ceph-mon[32569]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:35 smithi078 ceph-mon[29163]: Deploying daemon osd.5 on smithi129 2024-04-25T13:53:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:35 smithi078 ceph-mon[29163]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:35 smithi112 ceph-mon[32617]: Deploying daemon osd.5 on smithi129 2024-04-25T13:53:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:35 smithi112 ceph-mon[32617]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:36 smithi078 ceph-mon[29163]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:36.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:36 smithi112 ceph-mon[32617]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:36.568 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:36 smithi129 ceph-mon[32569]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:39.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:39 smithi129 ceph-mon[32569]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:39 smithi078 ceph-mon[29163]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:39 smithi112 ceph-mon[32617]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:40.694 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:40.694 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:40.695 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:40.695 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:40 smithi129 ceph-mon[32569]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:40 smithi078 ceph-mon[29163]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:40 smithi112 ceph-mon[32617]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:43.330 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:43 smithi129 ceph-mon[32569]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:43.330 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.330 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.331 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:43.331 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:43.331 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.387 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 5 on host 'smithi129' 2024-04-25T13:53:43.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.386+0000 7fb2397fa700 1 -- 172.21.15.129:0/1208935468 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb23c063ad0 con 0x7fb2240afb80 2024-04-25T13:53:43.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2240afb80 msgr2=0x7fb2240b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:43.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2240afb80 0x7fb2240b2090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0038a0 tx=0x7fb22c01a040 comp rx=0 tx=0).stop 2024-04-25T13:53:43.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c105340 msgr2=0x7fb23c19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:43.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c105340 0x7fb23c19ace0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c9b0 tx=0x7fb23000cd70 comp rx=0 tx=0).stop 2024-04-25T13:53:43.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 shutdown_connections 2024-04-25T13:53:43.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2240afb80 0x7fb2240b2090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:43.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb23c105ce0 0x7fb23c1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:43.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb23c104210 0x7fb23c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:43.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 --2- 172.21.15.129:0/1208935468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c105340 0x7fb23c19ace0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:43.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 >> 172.21.15.129:0/1208935468 conn(0x7fb23c0ff870 msgr2=0x7fb23c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:43.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 shutdown_connections 2024-04-25T13:53:43.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:53:43.389+0000 7fb242976700 1 -- 172.21.15.129:0/1208935468 wait complete. 2024-04-25T13:53:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:43 smithi078 ceph-mon[29163]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:43 smithi112 ceph-mon[32617]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:44.001 DEBUG:teuthology.orchestra.run.smithi129:osd.5> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T13:53:44.004 INFO:tasks.cephadm:Deploying osd.6 on smithi078 with /dev/vg_nvme/lv_2... 2024-04-25T13:53:44.004 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:53:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:53:44.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:44.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:44.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:44 smithi078 ceph-mon[29163]: from='osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:53:44.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:44 smithi078 ceph-mon[29163]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:53:44.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:44.648 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:44 smithi078 ceph-mon[29163]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:44 smithi129 ceph-mon[32569]: from='osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:53:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:44 smithi129 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:53:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:44 smithi129 ceph-mon[32569]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:44 smithi112 ceph-mon[32617]: from='osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:53:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:44 smithi112 ceph-mon[32617]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:53:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:53:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:44 smithi112 ceph-mon[32617]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:45.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:45 smithi129 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:53:45.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:45 smithi129 ceph-mon[32569]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T13:53:45.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:45 smithi129 ceph-mon[32569]: from='osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:53:45.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:45 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:45.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:45 smithi129 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:53:45.700 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:45 smithi078 ceph-mon[29163]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:53:45.700 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:45 smithi078 ceph-mon[29163]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T13:53:45.700 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:45 smithi078 ceph-mon[29163]: from='osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:53:45.700 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:45.701 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:45 smithi078 ceph-mon[29163]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:53:45.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:45 smithi112 ceph-mon[32617]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:53:45.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:45 smithi112 ceph-mon[32617]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T13:53:45.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:45 smithi112 ceph-mon[32617]: from='osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:53:45.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:45 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:45.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:45 smithi112 ceph-mon[32617]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:53:46.164 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 13:53:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T13:53:45.841+0000 7ff9d464d700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T13:53:46.164 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 13:53:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T13:53:45.850+0000 7ff9cc3fa700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:53:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:53:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:53:46.266 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T13:53:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T13:53:46.266 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125912 s, 833 MB/s 2024-04-25T13:53:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T13:53:46.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:46 smithi129 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:53:46.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:46 smithi129 ceph-mon[32569]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:53:46.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:46.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:46.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:46 smithi129 ceph-mon[32569]: from='osd.5 ' entity='osd.5' 2024-04-25T13:53:46.631 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:46 smithi129 ceph-mon[32569]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:46.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:46 smithi078 ceph-mon[29163]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:53:46.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:46 smithi078 ceph-mon[29163]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:53:46.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:46.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:46.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:46 smithi078 ceph-mon[29163]: from='osd.5 ' entity='osd.5' 2024-04-25T13:53:46.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:46 smithi078 ceph-mon[29163]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:46 smithi112 ceph-mon[32617]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:53:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:46 smithi112 ceph-mon[32617]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T13:53:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:46 smithi112 ceph-mon[32617]: from='osd.5 ' entity='osd.5' 2024-04-25T13:53:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:46 smithi112 ceph-mon[32617]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:53:46.980 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-04-25T13:53:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] boot 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi129 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-25T13:53:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] boot 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi129 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:47.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:47.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:53:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:53:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: osd.5 [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] boot 2024-04-25T13:53:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T13:53:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:53:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi129 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:53:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:53:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:48 smithi129 ceph-mon[32569]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T13:53:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:48 smithi129 ceph-mon[32569]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:48.679 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:48 smithi078 ceph-mon[29163]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T13:53:48.680 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:48 smithi078 ceph-mon[29163]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:48 smithi112 ceph-mon[32617]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T13:53:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:48 smithi112 ceph-mon[32617]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.934+0000 7f5a97bd4700 1 -- 172.21.15.78:0/2477466234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a90104bb0 msgr2=0x7f5a9010b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.934+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/2477466234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a90104bb0 0x7f5a9010b460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800b0a0 tx=0x7f5a8800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.934+0000 7f5a97bd4700 1 -- 172.21.15.78:0/2477466234 shutdown_connections 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.934+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/2477466234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a90104bb0 0x7f5a9010b460 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.935+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/2477466234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a901030e0 0x7f5a901034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.935+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/2477466234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a90104210 0x7f5a90104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.935+0000 7f5a97bd4700 1 -- 172.21.15.78:0/2477466234 >> 172.21.15.78:0/2477466234 conn(0x7f5a900fe740 msgr2=0x7f5a90100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.935+0000 7f5a97bd4700 1 -- 172.21.15.78:0/2477466234 shutdown_connections 2024-04-25T13:53:48.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.935+0000 7f5a97bd4700 1 -- 172.21.15.78:0/2477466234 wait complete. 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.935+0000 7f5a97bd4700 1 Processor -- start 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.935+0000 7f5a97bd4700 1 -- start start 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a97bd4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a901030e0 0x7f5a9019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a97bd4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a90104210 0x7f5a9019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a97bd4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a90104bb0 0x7f5a901a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a97bd4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9010fcc0 con 0x7f5a901030e0 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a97bd4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9010fb40 con 0x7f5a90104bb0 2024-04-25T13:53:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a97bd4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9010fe40 con 0x7f5a90104210 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a901030e0 0x7f5a9019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a901030e0 0x7f5a9019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54852/0 (socket says 172.21.15.78:54852) 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 -- 172.21.15.78:0/3607017323 learned_addr learned my addr 172.21.15.78:0/3607017323 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a9516f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a90104210 0x7f5a9019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a96171700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a90104bb0 0x7f5a901a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 -- 172.21.15.78:0/3607017323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a90104210 msgr2=0x7f5a9019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a90104210 0x7f5a9019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 -- 172.21.15.78:0/3607017323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a90104bb0 msgr2=0x7f5a901a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a90104bb0 0x7f5a901a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 -- 172.21.15.78:0/3607017323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a901a2270 con 0x7f5a901030e0 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.936+0000 7f5a95970700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a901030e0 0x7f5a9019a710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00d950 tx=0x7f5a8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.937+0000 7f5a82ffd700 1 -- 172.21.15.78:0/3607017323 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c017070 con 0x7f5a901030e0 2024-04-25T13:53:48.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.937+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a901a2560 con 0x7f5a901030e0 2024-04-25T13:53:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.937+0000 7f5a82ffd700 1 -- 172.21.15.78:0/3607017323 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a8c009730 con 0x7f5a901030e0 2024-04-25T13:53:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.937+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a901a2af0 con 0x7f5a901030e0 2024-04-25T13:53:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.937+0000 7f5a82ffd700 1 -- 172.21.15.78:0/3607017323 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c003a00 con 0x7f5a901030e0 2024-04-25T13:53:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.938+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a74005360 con 0x7f5a901030e0 2024-04-25T13:53:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.939+0000 7f5a82ffd700 1 -- 172.21.15.78:0/3607017323 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f5a8c003ba0 con 0x7f5a901030e0 2024-04-25T13:53:48.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.940+0000 7f5a82ffd700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5a7c0afb80 0x7f5a7c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:48.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.940+0000 7f5a9516f700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5a7c0afb80 0x7f5a7c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:48.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.940+0000 7f5a82ffd700 1 -- 172.21.15.78:0/3607017323 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f5a8c0ddfb0 con 0x7f5a901030e0 2024-04-25T13:53:48.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.940+0000 7f5a9516f700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5a7c0afb80 0x7f5a7c0b2090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a84005950 tx=0x7f5a8400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:48.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:48.944+0000 7f5a82ffd700 1 -- 172.21.15.78:0/3607017323 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a8c020030 con 0x7f5a901030e0 2024-04-25T13:53:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:53:49.151+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5a74000bf0 con 0x7f5a7c0afb80 2024-04-25T13:53:49.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:49 smithi129 ceph-mon[32569]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T13:53:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:49 smithi129 ceph-mon[32569]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:53:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:53:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:53:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:49.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:49 smithi112 ceph-mon[32617]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T13:53:49.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:49 smithi112 ceph-mon[32617]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:53:49.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:53:49.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:53:49.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:49 smithi078 ceph-mon[29163]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T13:53:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:49 smithi078 ceph-mon[29163]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:53:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:53:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:53:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:53:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:50 smithi129 ceph-mon[32569]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:50 smithi078 ceph-mon[29163]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:50.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:50 smithi112 ceph-mon[32617]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:52.361 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:52 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3033336387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bee711c4-03f1-463b-a61a-208445896ac9"}]: dispatch 2024-04-25T13:53:52.361 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:52 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3033336387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bee711c4-03f1-463b-a61a-208445896ac9"}]': finished 2024-04-25T13:53:52.361 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:52 smithi078 ceph-mon[29163]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T13:53:52.361 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:53:52.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:52 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3033336387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bee711c4-03f1-463b-a61a-208445896ac9"}]: dispatch 2024-04-25T13:53:52.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:52 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3033336387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bee711c4-03f1-463b-a61a-208445896ac9"}]': finished 2024-04-25T13:53:52.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:52 smithi129 ceph-mon[32569]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T13:53:52.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:53:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:52 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3033336387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bee711c4-03f1-463b-a61a-208445896ac9"}]: dispatch 2024-04-25T13:53:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:52 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3033336387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bee711c4-03f1-463b-a61a-208445896ac9"}]': finished 2024-04-25T13:53:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:52 smithi112 ceph-mon[32617]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T13:53:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:53:53.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:53 smithi129 ceph-mon[32569]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:53.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:53 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2968973331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:53 smithi078 ceph-mon[29163]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:53 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2968973331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:53.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:53 smithi112 ceph-mon[32617]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:53:53.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:53 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2968973331' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:53:55.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:55 smithi129 ceph-mon[32569]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T13:53:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:55 smithi078 ceph-mon[29163]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T13:53:55.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:55 smithi112 ceph-mon[32617]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T13:53:57.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:57 smithi129 ceph-mon[32569]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:53:57.462 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:57 smithi078 ceph-mon[29163]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:53:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:57 smithi112 ceph-mon[32617]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:53:59.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:53:59 smithi129 ceph-mon[32569]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:53:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:53:59 smithi078 ceph-mon[29163]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:53:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:53:59 smithi112 ceph-mon[32617]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:54:00.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:00 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:54:00.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:00 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:00.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:00 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:54:00.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:00 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:00.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:00 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:54:00.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:00 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:01 smithi112 ceph-mon[32617]: Deploying daemon osd.6 on smithi078 2024-04-25T13:54:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:01 smithi112 ceph-mon[32617]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:54:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:01 smithi129 ceph-mon[32569]: Deploying daemon osd.6 on smithi078 2024-04-25T13:54:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:01 smithi129 ceph-mon[32569]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:54:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:01 smithi078 ceph-mon[29163]: Deploying daemon osd.6 on smithi078 2024-04-25T13:54:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:01 smithi078 ceph-mon[29163]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:54:03.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:03 smithi129 ceph-mon[32569]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:54:03.439 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:03 smithi078 ceph-mon[29163]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:54:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:03 smithi112 ceph-mon[32617]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:54:05.333 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:05 smithi078 ceph-mon[29163]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:54:05.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:05 smithi129 ceph-mon[32569]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:54:05.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:05 smithi112 ceph-mon[32617]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:54:06.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:06.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:06.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:06.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:06 smithi129 ceph-mon[32569]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:06.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:06.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:06.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:06.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:06 smithi078 ceph-mon[29163]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:07.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:07.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:07.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:07.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:06 smithi112 ceph-mon[32617]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:09.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:09 smithi078 ceph-mon[29163]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:09.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:09.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:09.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:09 smithi129 ceph-mon[32569]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:09 smithi112 ceph-mon[32617]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.760+0000 7f5a82ffd700 1 -- 172.21.15.78:0/3607017323 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5a74000bf0 con 0x7f5a7c0afb80 2024-04-25T13:54:09.760 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 6 on host 'smithi078' 2024-04-25T13:54:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5a7c0afb80 msgr2=0x7f5a7c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5a7c0afb80 0x7f5a7c0b2090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a84005950 tx=0x7f5a8400a300 comp rx=0 tx=0).stop 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a901030e0 msgr2=0x7f5a9019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a901030e0 0x7f5a9019a710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00d950 tx=0x7f5a8c00dc60 comp rx=0 tx=0).stop 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 shutdown_connections 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5a7c0afb80 0x7f5a7c0b2090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a90104bb0 0x7f5a901a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a901030e0 0x7f5a9019a710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 --2- 172.21.15.78:0/3607017323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a90104210 0x7f5a9019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 >> 172.21.15.78:0/3607017323 conn(0x7f5a900fe740 msgr2=0x7f5a90107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 shutdown_connections 2024-04-25T13:54:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:54:09.763+0000 7f5a97bd4700 1 -- 172.21.15.78:0/3607017323 wait complete. 2024-04-25T13:54:10.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:10 smithi129 ceph-mon[32569]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:54:10.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:10 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:10.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:10 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:10.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:10 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:10 smithi078 ceph-mon[29163]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:54:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:10 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:10 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:10 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:10 smithi112 ceph-mon[32617]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:54:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:10 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:10 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:10 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:10.540 DEBUG:teuthology.orchestra.run.smithi078:osd.6> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T13:54:10.542 INFO:tasks.cephadm:Deploying osd.7 on smithi112 with /dev/vg_nvme/lv_2... 2024-04-25T13:54:10.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:54:10.785 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:54:11.159 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:11 smithi112 ceph-mon[32617]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:11.160 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:11 smithi112 ceph-mon[32617]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:54:11.160 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:11 smithi112 ceph-mon[32617]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T13:54:11.160 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:11 smithi112 ceph-mon[32617]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:54:11.160 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:11.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:11 smithi129 ceph-mon[32569]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:11.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:11 smithi129 ceph-mon[32569]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:54:11.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:11 smithi129 ceph-mon[32569]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T13:54:11.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:11 smithi129 ceph-mon[32569]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:54:11.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:11 smithi078 ceph-mon[29163]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:11 smithi078 ceph-mon[29163]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:54:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:11 smithi078 ceph-mon[29163]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T13:54:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:11 smithi078 ceph-mon[29163]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:54:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:11.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 13:54:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T13:54:11.155+0000 7f35fc6c0700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T13:54:11.470 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 13:54:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T13:54:11.166+0000 7f35f8475700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:54:12.300 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:12 smithi112 ceph-mon[32617]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:54:12.300 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:12 smithi112 ceph-mon[32617]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T13:54:12.300 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:12 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:12.300 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:12 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:12 smithi129 ceph-mon[32569]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:54:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:12 smithi129 ceph-mon[32569]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T13:54:12.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:12 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:12.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:12 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:12 smithi078 ceph-mon[29163]: from='osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:54:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:12 smithi078 ceph-mon[29163]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T13:54:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:12 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:54:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:54:12.877 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T13:54:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T13:54:12.877 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124023 s, 845 MB/s 2024-04-25T13:54:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T13:54:13.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:13 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:54:13.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:13 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:54:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:13 smithi129 ceph-mon[32569]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:13 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:13 smithi129 ceph-mon[32569]: osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] boot 2024-04-25T13:54:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:13 smithi129 ceph-mon[32569]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T13:54:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:13 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:13.438 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:13 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:54:13.438 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:13 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:54:13.438 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:13 smithi078 ceph-mon[29163]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:13.438 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:13 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:13.438 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:13 smithi078 ceph-mon[29163]: osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] boot 2024-04-25T13:54:13.438 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:13 smithi078 ceph-mon[29163]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T13:54:13.439 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:13 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:13.514 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:13 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:54:13.514 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:13 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:54:13.514 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:13 smithi112 ceph-mon[32617]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:54:13.514 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:13 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:13.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:13 smithi112 ceph-mon[32617]: osd.6 [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] boot 2024-04-25T13:54:13.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:13 smithi112 ceph-mon[32617]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T13:54:13.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:13 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:54:13.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-04-25T13:54:13.781 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:54:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T13:54:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi078 2024-04-25T13:54:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:14 smithi078 ceph-mon[29163]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi078 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:14.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:14 smithi112 ceph-mon[32617]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T13:54:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi078 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:14.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:14 smithi129 ceph-mon[32569]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 -- 172.21.15.112:0/1291829956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010103e00 msgr2=0x7fe010104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 --2- 172.21.15.112:0/1291829956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010103e00 0x7fe010104210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe004009990 tx=0x7fe004009ca0 comp rx=0 tx=0).stop 2024-04-25T13:54:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 -- 172.21.15.112:0/1291829956 shutdown_connections 2024-04-25T13:54:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 --2- 172.21.15.112:0/1291829956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe010104880 0x7fe01010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 --2- 172.21.15.112:0/1291829956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe010105850 0x7fe010105c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:15.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 --2- 172.21.15.112:0/1291829956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010103e00 0x7fe010104210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 -- 172.21.15.112:0/1291829956 >> 172.21.15.112:0/1291829956 conn(0x7fe0100ff870 msgr2=0x7fe010101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 -- 172.21.15.112:0/1291829956 shutdown_connections 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 -- 172.21.15.112:0/1291829956 wait complete. 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 Processor -- start 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.487+0000 7fe016915700 1 -- start start 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe016915700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe010103e00 0x7fe01019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe016915700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe010104880 0x7fe01019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe016915700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010105850 0x7fe0101a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe016915700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010fde0 con 0x7fe010104880 2024-04-25T13:54:15.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe016915700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010fc60 con 0x7fe010105850 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe016915700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010ff60 con 0x7fe010103e00 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010105850 0x7fe0101a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010105850 0x7fe0101a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50458/0 (socket says 172.21.15.112:50458) 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 -- 172.21.15.112:0/3893209736 learned_addr learned my addr 172.21.15.112:0/3893209736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe00f7fe700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe010104880 0x7fe01019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe00ffff700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe010103e00 0x7fe01019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 -- 172.21.15.112:0/3893209736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe010103e00 msgr2=0x7fe01019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe010103e00 0x7fe01019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 -- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe010104880 msgr2=0x7fe01019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe010104880 0x7fe01019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 -- 172.21.15.112:0/3893209736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0101a45a0 con 0x7fe010105850 2024-04-25T13:54:15.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.488+0000 7fe014eb2700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010105850 0x7fe0101a3fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe008007ee0 tx=0x7fe00800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:15.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.489+0000 7fe00d7fa700 1 -- 172.21.15.112:0/3893209736 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008004bb0 con 0x7fe010105850 2024-04-25T13:54:15.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.489+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0101a4890 con 0x7fe010105850 2024-04-25T13:54:15.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.489+0000 7fe00d7fa700 1 -- 172.21.15.112:0/3893209736 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe008004d50 con 0x7fe010105850 2024-04-25T13:54:15.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.489+0000 7fe00d7fa700 1 -- 172.21.15.112:0/3893209736 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008005920 con 0x7fe010105850 2024-04-25T13:54:15.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.489+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0101a4d80 con 0x7fe010105850 2024-04-25T13:54:15.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.490+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe010051190 con 0x7fe010105850 2024-04-25T13:54:15.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.491+0000 7fe00d7fa700 1 -- 172.21.15.112:0/3893209736 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe00801a020 con 0x7fe010105850 2024-04-25T13:54:15.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.491+0000 7fe00d7fa700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdff80afb80 0x7fdff80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:15.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.492+0000 7fe00d7fa700 1 -- 172.21.15.112:0/3893209736 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7fe0080dd5e0 con 0x7fe010105850 2024-04-25T13:54:15.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.492+0000 7fe00ffff700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdff80afb80 0x7fdff80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:15.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.492+0000 7fe00ffff700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdff80afb80 0x7fdff80b2090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe010105240 tx=0x7fe00000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:15.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.497+0000 7fe00d7fa700 1 -- 172.21.15.112:0/3893209736 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe008010070 con 0x7fe010105850 2024-04-25T13:54:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:15.709+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe0100780f0 con 0x7fdff80afb80 2024-04-25T13:54:16.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:54:16.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:54:16.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:15 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:16.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:54:16.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:54:16.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:15 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:54:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:54:16.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:15 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:16.956 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:16 smithi112 ceph-mon[32617]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:54:16.957 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:16 smithi112 ceph-mon[32617]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:17.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:16 smithi129 ceph-mon[32569]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:54:17.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:16 smithi129 ceph-mon[32569]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:17.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:16 smithi078 ceph-mon[29163]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:54:17.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:16 smithi078 ceph-mon[29163]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:19 smithi129 ceph-mon[32569]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:19 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/885385813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]: dispatch 2024-04-25T13:54:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:19 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]: dispatch 2024-04-25T13:54:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:19 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]': finished 2024-04-25T13:54:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:19 smithi129 ceph-mon[32569]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T13:54:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:19 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:19 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/4002153560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:54:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:19 smithi078 ceph-mon[29163]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:19 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/885385813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]: dispatch 2024-04-25T13:54:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:19 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]: dispatch 2024-04-25T13:54:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:19 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]': finished 2024-04-25T13:54:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:19 smithi078 ceph-mon[29163]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T13:54:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:19 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:19 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/4002153560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:54:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:19 smithi112 ceph-mon[32617]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:19 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/885385813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]: dispatch 2024-04-25T13:54:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:19 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]: dispatch 2024-04-25T13:54:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:19 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74bfe646-13d1-471c-acf9-8f84969d2313"}]': finished 2024-04-25T13:54:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:19 smithi112 ceph-mon[32617]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T13:54:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:19 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:19.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:19 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/4002153560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:54:21.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:21 smithi129 ceph-mon[32569]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:21 smithi078 ceph-mon[29163]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:21 smithi112 ceph-mon[32617]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:23.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:23 smithi129 ceph-mon[32569]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:23 smithi078 ceph-mon[29163]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:23.486 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:23 smithi112 ceph-mon[32617]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:25.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:25 smithi112 ceph-mon[32617]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:25.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:25 smithi129 ceph-mon[32569]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:25 smithi078 ceph-mon[29163]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:54:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:54:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:26.476 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:54:26.476 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:27.222 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:27 smithi112 ceph-mon[32617]: Deploying daemon osd.7 on smithi112 2024-04-25T13:54:27.223 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:27 smithi112 ceph-mon[32617]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:27 smithi129 ceph-mon[32569]: Deploying daemon osd.7 on smithi112 2024-04-25T13:54:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:27 smithi129 ceph-mon[32569]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:27.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:27 smithi078 ceph-mon[29163]: Deploying daemon osd.7 on smithi112 2024-04-25T13:54:27.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:27 smithi078 ceph-mon[29163]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:29.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:29 smithi129 ceph-mon[32569]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:29.414 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:29 smithi112 ceph-mon[32617]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:29 smithi078 ceph-mon[29163]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:31 smithi078 ceph-mon[29163]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:31.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:31 smithi112 ceph-mon[32617]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:31 smithi129 ceph-mon[32569]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:32.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:32 smithi112 ceph-mon[32617]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:32.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:32.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:32 smithi129 ceph-mon[32569]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:32 smithi078 ceph-mon[29163]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:34.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.620+0000 7fe00d7fa700 1 -- 172.21.15.112:0/3893209736 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe0100780f0 con 0x7fdff80afb80 2024-04-25T13:54:34.622 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 7 on host 'smithi112' 2024-04-25T13:54:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdff80afb80 msgr2=0x7fdff80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdff80afb80 0x7fdff80b2090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe010105240 tx=0x7fe00000a3b0 comp rx=0 tx=0).stop 2024-04-25T13:54:34.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010105850 msgr2=0x7fe0101a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010105850 0x7fe0101a3fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe008007ee0 tx=0x7fe00800d3b0 comp rx=0 tx=0).stop 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 shutdown_connections 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdff80afb80 0x7fdff80b2090 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe010105850 0x7fe0101a3fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe010103e00 0x7fe01019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 --2- 172.21.15.112:0/3893209736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe010104880 0x7fe01019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 >> 172.21.15.112:0/3893209736 conn(0x7fe0100ff870 msgr2=0x7fe0100769d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 shutdown_connections 2024-04-25T13:54:34.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:54:34.623+0000 7fe016915700 1 -- 172.21.15.112:0/3893209736 wait complete. 2024-04-25T13:54:35.207 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:35 smithi112 ceph-mon[32617]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:35.207 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:35.208 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:35.233 DEBUG:teuthology.orchestra.run.smithi112:osd.7> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T13:54:35.235 INFO:tasks.cephadm:Deploying osd.8 on smithi129 with /dev/vg_nvme/lv_2... 2024-04-25T13:54:35.236 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:54:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:35 smithi129 ceph-mon[32569]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:35 smithi078 ceph-mon[29163]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:35.484 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:54:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:36 smithi112 ceph-mon[32617]: from='osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:54:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:36 smithi112 ceph-mon[32617]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:54:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:36 smithi129 ceph-mon[32569]: from='osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:54:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:36 smithi129 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:54:36.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:36.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:36 smithi078 ceph-mon[29163]: from='osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:54:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:36 smithi078 ceph-mon[29163]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:54:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:37.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:37 smithi129 ceph-mon[32569]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:37.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:37 smithi129 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:54:37.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:37 smithi129 ceph-mon[32569]: from='osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:54:37.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:37 smithi129 ceph-mon[32569]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T13:54:37.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:37 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:37.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:37 smithi129 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:54:37.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:37 smithi129 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:54:37.250 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-mon[32617]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:37.251 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-mon[32617]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:54:37.251 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-mon[32617]: from='osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:54:37.251 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-mon[32617]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T13:54:37.251 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:37.251 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-mon[32617]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:54:37.251 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-mon[32617]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:54:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:37 smithi078 ceph-mon[29163]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:37 smithi078 ceph-mon[29163]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:54:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:37 smithi078 ceph-mon[29163]: from='osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:54:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:37 smithi078 ceph-mon[29163]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T13:54:37.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:37 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:37.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:37 smithi078 ceph-mon[29163]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:54:37.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:37 smithi078 ceph-mon[29163]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:54:37.496 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:54:37.496 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:54:37.496 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-25T13:54:37.496 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-25T13:54:37.496 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129817 s, 808 MB/s 2024-04-25T13:54:37.497 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-25T13:54:38.016 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T13:54:37.619+0000 7fbe26b8f700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T13:54:38.016 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 13:54:37 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T13:54:37.633+0000 7fbe1e93c700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:54:38.095 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-04-25T13:54:38.342 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:54:38.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:38 smithi129 ceph-mon[32569]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T13:54:38.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:38.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:38.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:38 smithi129 ceph-mon[32569]: from='osd.7 ' entity='osd.7' 2024-04-25T13:54:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:38 smithi078 ceph-mon[29163]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T13:54:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:38 smithi078 ceph-mon[29163]: from='osd.7 ' entity='osd.7' 2024-04-25T13:54:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:38 smithi112 ceph-mon[32617]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T13:54:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:38 smithi112 ceph-mon[32617]: from='osd.7 ' entity='osd.7' 2024-04-25T13:54:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:54:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:54:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] boot 2024-04-25T13:54:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T13:54:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:39.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] boot 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:39.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:39.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:54:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:54:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:54:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: osd.7 [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] boot 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:40.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 -- 172.21.15.129:0/635848187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488104230 msgr2=0x7f5488104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 --2- 172.21.15.129:0/635848187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488104230 0x7f5488104640 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5474009990 tx=0x7f5474009ca0 comp rx=0 tx=0).stop 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 -- 172.21.15.129:0/635848187 shutdown_connections 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 --2- 172.21.15.129:0/635848187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488104bd0 0x7f548810b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 --2- 172.21.15.129:0/635848187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5488103100 0x7f5488103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 --2- 172.21.15.129:0/635848187 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488104230 0x7f5488104640 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 -- 172.21.15.129:0/635848187 >> 172.21.15.129:0/635848187 conn(0x7f54880fe760 msgr2=0x7f5488100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 -- 172.21.15.129:0/635848187 shutdown_connections 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.059+0000 7f548ee3b700 1 -- 172.21.15.129:0/635848187 wait complete. 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 Processor -- start 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 -- start start 2024-04-25T13:54:40.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488103100 0x7f54881996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488104230 0x7f5488199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5488104bd0 0x7f54881a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f548810eb70 con 0x7f5488104bd0 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f548810e9f0 con 0x7f5488104230 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548ee3b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f548810ecf0 con 0x7f5488103100 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548cbd7700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488103100 0x7f54881996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548cbd7700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488103100 0x7f54881996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57864/0 (socket says 172.21.15.129:57864) 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548cbd7700 1 -- 172.21.15.129:0/4162782757 learned_addr learned my addr 172.21.15.129:0/4162782757 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f5487fff700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488104230 0x7f5488199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.060+0000 7f548d3d8700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5488104bd0 0x7f54881a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548cbd7700 1 -- 172.21.15.129:0/4162782757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488104230 msgr2=0x7f5488199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548cbd7700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488104230 0x7f5488199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548cbd7700 1 -- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5488104bd0 msgr2=0x7f54881a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548cbd7700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5488104bd0 0x7f54881a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:40.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548cbd7700 1 -- 172.21.15.129:0/4162782757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54881a1220 con 0x7f5488103100 2024-04-25T13:54:40.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548cbd7700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488103100 0x7f54881996c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f547c007900 tx=0x7f547c007c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:40.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f5485ffb700 1 -- 172.21.15.129:0/4162782757 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c01d070 con 0x7f5488103100 2024-04-25T13:54:40.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f5485ffb700 1 -- 172.21.15.129:0/4162782757 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f547c009990 con 0x7f5488103100 2024-04-25T13:54:40.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f5485ffb700 1 -- 172.21.15.129:0/4162782757 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c018ac0 con 0x7f5488103100 2024-04-25T13:54:40.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5488075920 con 0x7f5488103100 2024-04-25T13:54:40.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.061+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5488075eb0 con 0x7f5488103100 2024-04-25T13:54:40.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.062+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5488051190 con 0x7f5488103100 2024-04-25T13:54:40.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.064+0000 7f5485ffb700 1 -- 172.21.15.129:0/4162782757 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f547c0094e0 con 0x7f5488103100 2024-04-25T13:54:40.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.064+0000 7f5485ffb700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54780afc00 0x7f54780b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:54:40.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.064+0000 7f5485ffb700 1 -- 172.21.15.129:0/4162782757 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5397+0+0 (secure 0 0 0) 0x7f547c0df830 con 0x7f5488103100 2024-04-25T13:54:40.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.064+0000 7f5487fff700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54780afc00 0x7f54780b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:54:40.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.065+0000 7f5487fff700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54780afc00 0x7f54780b2110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f547400b5c0 tx=0x7f547401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:54:40.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.070+0000 7f5485ffb700 1 -- 172.21.15.129:0/4162782757 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f547c026020 con 0x7f5488103100 2024-04-25T13:54:40.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:40.279+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5488076170 con 0x7f54780afc00 2024-04-25T13:54:40.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:40 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi112 2024-04-25T13:54:40.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:40 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T13:54:40.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:40 smithi129 ceph-mon[32569]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T13:54:40.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:40.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:40.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:40.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:40 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi112 2024-04-25T13:54:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:40 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T13:54:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:40 smithi078 ceph-mon[29163]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T13:54:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:40 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi112 2024-04-25T13:54:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:40 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T13:54:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:40 smithi112 ceph-mon[32617]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T13:54:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:54:40.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:41 smithi078 ceph-mon[29163]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:41 smithi078 ceph-mon[29163]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T13:54:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:54:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:54:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:41 smithi112 ceph-mon[32617]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:41 smithi112 ceph-mon[32617]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T13:54:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:54:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:54:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:41 smithi129 ceph-mon[32569]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:41 smithi129 ceph-mon[32569]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T13:54:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:54:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:54:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:42.448 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:42 smithi129 ceph-mon[32569]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:54:42.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:42 smithi078 ceph-mon[29163]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:54:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:42 smithi112 ceph-mon[32617]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:54:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:43 smithi078 ceph-mon[29163]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:43 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/1871767187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]: dispatch 2024-04-25T13:54:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:43 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]: dispatch 2024-04-25T13:54:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:43 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]': finished 2024-04-25T13:54:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:43 smithi078 ceph-mon[29163]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T13:54:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:54:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:43 smithi112 ceph-mon[32617]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:43 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/1871767187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]: dispatch 2024-04-25T13:54:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:43 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]: dispatch 2024-04-25T13:54:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:43 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]': finished 2024-04-25T13:54:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:43 smithi112 ceph-mon[32617]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T13:54:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:54:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:43 smithi129 ceph-mon[32569]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:43 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/1871767187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]: dispatch 2024-04-25T13:54:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:43 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]: dispatch 2024-04-25T13:54:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:43 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2220b155-081a-4bdd-96c2-e46f8e20b97d"}]': finished 2024-04-25T13:54:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:43 smithi129 ceph-mon[32569]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T13:54:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:54:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:44 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/3783102702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:54:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:44 smithi078 ceph-mon[29163]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:44 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/3783102702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:54:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:44 smithi112 ceph-mon[32617]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:44 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/3783102702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:54:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:44 smithi129 ceph-mon[32569]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:47.270 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:47 smithi129 ceph-mon[32569]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 166 KiB/s, 0 objects/s recovering 2024-04-25T13:54:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:47 smithi078 ceph-mon[29163]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 166 KiB/s, 0 objects/s recovering 2024-04-25T13:54:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:47 smithi112 ceph-mon[32617]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 166 KiB/s, 0 objects/s recovering 2024-04-25T13:54:49.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:49 smithi129 ceph-mon[32569]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:54:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:49 smithi078 ceph-mon[29163]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:54:49.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:49 smithi112 ceph-mon[32617]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:54:51.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:51 smithi129 ceph-mon[32569]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T13:54:51.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:54:51.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:51 smithi078 ceph-mon[29163]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T13:54:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:54:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:51.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:51 smithi112 ceph-mon[32617]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T13:54:51.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:54:51.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:54:52.392 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:52 smithi129 ceph-mon[32569]: Deploying daemon osd.8 on smithi129 2024-04-25T13:54:52.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:52 smithi078 ceph-mon[29163]: Deploying daemon osd.8 on smithi129 2024-04-25T13:54:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:52 smithi112 ceph-mon[32617]: Deploying daemon osd.8 on smithi129 2024-04-25T13:54:53.383 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:53 smithi129 ceph-mon[32569]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T13:54:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:53 smithi078 ceph-mon[29163]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T13:54:53.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:53 smithi112 ceph-mon[32617]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T13:54:55.266 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:55 smithi129 ceph-mon[32569]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T13:54:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:55 smithi078 ceph-mon[29163]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T13:54:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:55 smithi112 ceph-mon[32617]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T13:54:57.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:57 smithi129 ceph-mon[32569]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:54:57.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:57.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:57.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:57 smithi078 ceph-mon[29163]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:54:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:57 smithi112 ceph-mon[32617]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:54:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:54:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:54:59.372 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:54:59 smithi129 ceph-mon[32569]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:54:59 smithi078 ceph-mon[29163]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:54:59 smithi112 ceph-mon[32617]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:54:59.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.696+0000 7f5485ffb700 1 -- 172.21.15.129:0/4162782757 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5488076170 con 0x7f54780afc00 2024-04-25T13:54:59.696 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 8 on host 'smithi129' 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54780afc00 msgr2=0x7f54780b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54780afc00 0x7f54780b2110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f547400b5c0 tx=0x7f547401a040 comp rx=0 tx=0).stop 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488103100 msgr2=0x7f54881996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488103100 0x7f54881996c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f547c007900 tx=0x7f547c007c10 comp rx=0 tx=0).stop 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 shutdown_connections 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54780afc00 0x7f54780b2110 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5488104bd0 0x7f54881a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5488103100 0x7f54881996c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 --2- 172.21.15.129:0/4162782757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488104230 0x7f5488199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 >> 172.21.15.129:0/4162782757 conn(0x7f54880fe760 msgr2=0x7f5488107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:54:59.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 shutdown_connections 2024-04-25T13:54:59.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:54:59.698+0000 7f548ee3b700 1 -- 172.21.15.129:0/4162782757 wait complete. 2024-04-25T13:55:00.309 DEBUG:teuthology.orchestra.run.smithi129:osd.8> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T13:55:00.312 INFO:tasks.cephadm:Deploying osd.9 on smithi078 with /dev/vg_nvme/lv_1... 2024-04-25T13:55:00.312 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:55:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:55:00.921 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:00 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:00.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:00 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:00.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:00 smithi078 ceph-mon[29163]: from='osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:55:00.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:00 smithi078 ceph-mon[29163]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:55:00.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:00 smithi078 ceph-mon[29163]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:55:00.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:00 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:00.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:00 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:00 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:00 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:00 smithi112 ceph-mon[32617]: from='osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:55:01.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:00 smithi112 ceph-mon[32617]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:55:01.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:00 smithi112 ceph-mon[32617]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:55:01.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:00 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:00 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:00 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:00 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:00 smithi129 ceph-mon[32569]: from='osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:55:01.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:00 smithi129 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:55:01.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:00 smithi129 ceph-mon[32569]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:55:01.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:00 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.160 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:00 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:01 smithi129 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:55:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:01 smithi129 ceph-mon[32569]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T13:55:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:01.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:01 smithi129 ceph-mon[32569]: from='osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:55:01.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:01 smithi129 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:55:01.948 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:01 smithi078 ceph-mon[29163]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:55:01.948 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:01 smithi078 ceph-mon[29163]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T13:55:01.948 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:01.949 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:01 smithi078 ceph-mon[29163]: from='osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:55:01.949 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:01 smithi078 ceph-mon[29163]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:55:02.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:01 smithi112 ceph-mon[32617]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:55:02.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:01 smithi112 ceph-mon[32617]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T13:55:02.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:02.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:01 smithi112 ceph-mon[32617]: from='osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:55:02.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:01 smithi112 ceph-mon[32617]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:55:02.337 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T13:55:02.130+0000 7fd252966700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T13:55:02.337 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T13:55:02.141+0000 7fd24a713700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:55:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:55:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:55:02.697 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T13:55:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T13:55:02.697 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150286 s, 698 MB/s 2024-04-25T13:55:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T13:55:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:02 smithi078 ceph-mon[29163]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:55:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:02 smithi078 ceph-mon[29163]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T13:55:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:02 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:02 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:02 smithi078 ceph-mon[29163]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:55:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:02 smithi078 ceph-mon[29163]: from='osd.8 ' entity='osd.8' 2024-04-25T13:55:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:02 smithi112 ceph-mon[32617]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:55:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:02 smithi112 ceph-mon[32617]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T13:55:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:02 smithi112 ceph-mon[32617]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:55:03.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:02 smithi112 ceph-mon[32617]: from='osd.8 ' entity='osd.8' 2024-04-25T13:55:03.099 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:55:03.099 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-mon[32569]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T13:55:03.099 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:03.099 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:03.099 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-mon[32569]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:55:03.099 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:02 smithi129 ceph-mon[32569]: from='osd.8 ' entity='osd.8' 2024-04-25T13:55:03.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-04-25T13:55:03.692 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:55:03.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:03 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:55:03.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:03 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:55:03.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:03 smithi129 ceph-mon[32569]: osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] boot 2024-04-25T13:55:03.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:03 smithi129 ceph-mon[32569]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T13:55:03.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:03.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:03 smithi129 ceph-mon[32569]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T13:55:03.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:03 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:55:03.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:03 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:55:03.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:03 smithi078 ceph-mon[29163]: osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] boot 2024-04-25T13:55:03.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:03 smithi078 ceph-mon[29163]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T13:55:03.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:03.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:03 smithi078 ceph-mon[29163]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T13:55:04.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:03 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:55:04.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:03 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:55:04.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:03 smithi112 ceph-mon[32617]: osd.8 [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] boot 2024-04-25T13:55:04.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:03 smithi112 ceph-mon[32617]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T13:55:04.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:55:04.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:03 smithi112 ceph-mon[32617]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T13:55:05.142 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi129 2024-04-25T13:55:05.142 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.142 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.142 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.142 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:05.143 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:04 smithi078 ceph-mon[29163]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T13:55:05.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi129 2024-04-25T13:55:05.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:05.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:04 smithi129 ceph-mon[32569]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T13:55:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi129 2024-04-25T13:55:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:05.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:04 smithi112 ceph-mon[32617]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 -- 172.21.15.78:0/3647374059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c104040 msgr2=0x7fbd1c104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 --2- 172.21.15.78:0/3647374059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c104040 0x7fbd1c104510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009990 tx=0x7fbd0c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 -- 172.21.15.78:0/3647374059 shutdown_connections 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 --2- 172.21.15.78:0/3647374059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd1c104ba0 0x7fbd1c10b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 --2- 172.21.15.78:0/3647374059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd1c103660 0x7fbd1c103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 --2- 172.21.15.78:0/3647374059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c104040 0x7fbd1c104510 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 -- 172.21.15.78:0/3647374059 >> 172.21.15.78:0/3647374059 conn(0x7fbd1c0fded0 msgr2=0x7fbd1c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 -- 172.21.15.78:0/3647374059 shutdown_connections 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 -- 172.21.15.78:0/3647374059 wait complete. 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.371+0000 7fbd23c95700 1 Processor -- start 2024-04-25T13:55:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 -- start start 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c103660 0x7fbd1c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd1c104040 0x7fbd1c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd1c104ba0 0x7fbd1c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10eb90 con 0x7fbd1c103660 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10ea10 con 0x7fbd1c104ba0 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c10ed10 con 0x7fbd1c104040 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c103660 0x7fbd1c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c103660 0x7fbd1c1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54924/0 (socket says 172.21.15.78:54924) 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 -- 172.21.15.78:0/4234104510 learned_addr learned my addr 172.21.15.78:0/4234104510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21230700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd1c104040 0x7fbd1c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 -- 172.21.15.78:0/4234104510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd1c104040 msgr2=0x7fbd1c199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd22232700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd1c104ba0 0x7fbd1c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd1c104040 0x7fbd1c199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 -- 172.21.15.78:0/4234104510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd1c104ba0 msgr2=0x7fbd1c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd1c104ba0 0x7fbd1c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 -- 172.21.15.78:0/4234104510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c1a1230 con 0x7fbd1c103660 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd21a31700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c103660 0x7fbd1c1996d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800d950 tx=0x7fbd1800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd12ffd700 1 -- 172.21.15.78:0/4234104510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18017070 con 0x7fbd1c103660 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.372+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c1a1520 con 0x7fbd1c103660 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.373+0000 7fbd12ffd700 1 -- 172.21.15.78:0/4234104510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd1800de30 con 0x7fbd1c103660 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.373+0000 7fbd12ffd700 1 -- 172.21.15.78:0/4234104510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800f3c0 con 0x7fbd1c103660 2024-04-25T13:55:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.373+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a1a60 con 0x7fbd1c103660 2024-04-25T13:55:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.374+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1c051190 con 0x7fbd1c103660 2024-04-25T13:55:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.375+0000 7fbd12ffd700 1 -- 172.21.15.78:0/4234104510 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbd1801a070 con 0x7fbd1c103660 2024-04-25T13:55:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.375+0000 7fbd12ffd700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd080afb80 0x7fbd080b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.376+0000 7fbd21230700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd080afb80 0x7fbd080b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.376+0000 7fbd12ffd700 1 -- 172.21.15.78:0/4234104510 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7fbd180ddbd0 con 0x7fbd1c103660 2024-04-25T13:55:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.376+0000 7fbd21230700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd080afb80 0x7fbd080b2090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c006010 tx=0x7fbd0c00b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:05.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.380+0000 7fbd12ffd700 1 -- 172.21.15.78:0/4234104510 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd180a7160 con 0x7fbd1c103660 2024-04-25T13:55:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:05.592+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbd1c063b90 con 0x7fbd080afb80 2024-04-25T13:55:06.335 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:06.335 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:06.335 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:06 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:06.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:06.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:06.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:06 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:06 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:07.206 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:07 smithi078 ceph-mon[29163]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:55:07.206 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:07 smithi078 ceph-mon[29163]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:07.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:07 smithi129 ceph-mon[32569]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:55:07.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:07 smithi129 ceph-mon[32569]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:07.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:07 smithi112 ceph-mon[32617]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:55:07.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:07 smithi112 ceph-mon[32617]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:09.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:09 smithi129 ceph-mon[32569]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1743392029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6395fc39-cdba-4a65-bfd5-1dd36eaaa385"}]: dispatch 2024-04-25T13:55:09.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1743392029' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6395fc39-cdba-4a65-bfd5-1dd36eaaa385"}]': finished 2024-04-25T13:55:09.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:09 smithi129 ceph-mon[32569]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T13:55:09.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:09.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3820516326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:55:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:09 smithi078 ceph-mon[29163]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1743392029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6395fc39-cdba-4a65-bfd5-1dd36eaaa385"}]: dispatch 2024-04-25T13:55:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1743392029' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6395fc39-cdba-4a65-bfd5-1dd36eaaa385"}]': finished 2024-04-25T13:55:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:09 smithi078 ceph-mon[29163]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T13:55:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3820516326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:55:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:09 smithi112 ceph-mon[32617]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1743392029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6395fc39-cdba-4a65-bfd5-1dd36eaaa385"}]: dispatch 2024-04-25T13:55:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1743392029' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6395fc39-cdba-4a65-bfd5-1dd36eaaa385"}]': finished 2024-04-25T13:55:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:09 smithi112 ceph-mon[32617]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T13:55:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3820516326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:55:11.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:11 smithi129 ceph-mon[32569]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T13:55:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:11 smithi078 ceph-mon[29163]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T13:55:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:11 smithi112 ceph-mon[32617]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T13:55:13.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:13 smithi129 ceph-mon[32569]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:55:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:13 smithi078 ceph-mon[29163]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:55:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:13 smithi112 ceph-mon[32617]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:55:15.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:15 smithi078 ceph-mon[29163]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T13:55:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:15 smithi112 ceph-mon[32617]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T13:55:15.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:15 smithi129 ceph-mon[32569]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T13:55:17.319 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:17 smithi078 ceph-mon[29163]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:55:17.319 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:17 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:55:17.319 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:17 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:17.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:17 smithi112 ceph-mon[32617]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:55:17.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:17 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:55:17.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:17 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:17.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:17 smithi129 ceph-mon[32569]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:55:17.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:17 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:55:17.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:17 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:18.398 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:18 smithi078 ceph-mon[29163]: Deploying daemon osd.9 on smithi078 2024-04-25T13:55:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:18 smithi112 ceph-mon[32617]: Deploying daemon osd.9 on smithi078 2024-04-25T13:55:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:18 smithi129 ceph-mon[32569]: Deploying daemon osd.9 on smithi078 2024-04-25T13:55:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:19 smithi078 ceph-mon[29163]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:55:19.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:19 smithi112 ceph-mon[32617]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:55:19.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:19 smithi129 ceph-mon[32569]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:55:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:21 smithi078 ceph-mon[29163]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T13:55:21.489 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:21 smithi129 ceph-mon[32569]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T13:55:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:21 smithi112 ceph-mon[32617]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T13:55:23.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:23 smithi078 ceph-mon[29163]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:23.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:23.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:23.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:23.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:23 smithi112 ceph-mon[32617]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:23.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:23.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:23.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:23.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:23 smithi129 ceph-mon[32569]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:24.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:24 smithi078 ceph-mon[29163]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:24.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:24 smithi112 ceph-mon[32617]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:24 smithi129 ceph-mon[32569]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.802+0000 7fbd12ffd700 1 -- 172.21.15.78:0/4234104510 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd1c063b90 con 0x7fbd080afb80 2024-04-25T13:55:26.802 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 9 on host 'smithi078' 2024-04-25T13:55:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd080afb80 msgr2=0x7fbd080b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd080afb80 0x7fbd080b2090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c006010 tx=0x7fbd0c00b580 comp rx=0 tx=0).stop 2024-04-25T13:55:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c103660 msgr2=0x7fbd1c1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c103660 0x7fbd1c1996d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800d950 tx=0x7fbd1800dd10 comp rx=0 tx=0).stop 2024-04-25T13:55:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 shutdown_connections 2024-04-25T13:55:26.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd080afb80 0x7fbd080b2090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd1c104ba0 0x7fbd1c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd1c103660 0x7fbd1c1996d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 --2- 172.21.15.78:0/4234104510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd1c104040 0x7fbd1c199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 >> 172.21.15.78:0/4234104510 conn(0x7fbd1c0fded0 msgr2=0x7fbd1c107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 shutdown_connections 2024-04-25T13:55:26.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:55:26.804+0000 7fbd23c95700 1 -- 172.21.15.78:0/4234104510 wait complete. 2024-04-25T13:55:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:26 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:26 smithi112 ceph-mon[32617]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:26 smithi112 ceph-mon[32617]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:55:27.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:27.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:27.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:26 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:26 smithi129 ceph-mon[32569]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:27.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:26 smithi129 ceph-mon[32569]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:55:27.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:27.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:27.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:26 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:26 smithi078 ceph-mon[29163]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:27.191 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:26 smithi078 ceph-mon[29163]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:55:27.516 DEBUG:teuthology.orchestra.run.smithi078:osd.9> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T13:55:27.519 INFO:tasks.cephadm:Deploying osd.10 on smithi112 with /dev/vg_nvme/lv_1... 2024-04-25T13:55:27.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:55:27.765 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:55:27.887 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:27 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.887 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:27 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:27.887 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:27 smithi112 ceph-mon[32617]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:55:27.887 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:27 smithi112 ceph-mon[32617]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T13:55:27.887 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:27 smithi112 ceph-mon[32617]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:55:27.887 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:27 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:27.887 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:27 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:28.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:27 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:28.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:27 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:28.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:27 smithi078 ceph-mon[29163]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:55:28.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:27 smithi078 ceph-mon[29163]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T13:55:28.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:27 smithi078 ceph-mon[29163]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:55:28.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:27 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:28.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:27 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:28.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:27 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:28.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:27 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:28.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:27 smithi129 ceph-mon[32569]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:55:28.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:27 smithi129 ceph-mon[32569]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T13:55:28.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:27 smithi129 ceph-mon[32569]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T13:55:28.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:27 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:28.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:27 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:28.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T13:55:28.289+0000 7fce617d3700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T13:55:28.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T13:55:28.301+0000 7fce58d7f700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:55:29.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-mon[29163]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:55:29.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-mon[29163]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T13:55:29.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:29.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:29.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-mon[29163]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:29.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:28 smithi078 ceph-mon[29163]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' 2024-04-25T13:55:29.142 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:28 smithi112 ceph-mon[32617]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:55:29.143 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:28 smithi112 ceph-mon[32617]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T13:55:29.143 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:28 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:29.143 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:28 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:29.143 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:28 smithi112 ceph-mon[32617]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:29.143 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:28 smithi112 ceph-mon[32617]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' 2024-04-25T13:55:29.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:28 smithi129 ceph-mon[32569]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T13:55:29.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:28 smithi129 ceph-mon[32569]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T13:55:29.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:28 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:29.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:28 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:29.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:28 smithi129 ceph-mon[32569]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:55:29.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:28 smithi129 ceph-mon[32569]: from='osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298]' entity='osd.9' 2024-04-25T13:55:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:55:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:55:29.698 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T13:55:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T13:55:29.698 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130826 s, 802 MB/s 2024-04-25T13:55:29.698 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T13:55:30.087 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:29 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:55:30.087 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:29 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:55:30.087 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:30.087 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:29 smithi112 ceph-mon[32617]: osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] boot 2024-04-25T13:55:30.087 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:29 smithi112 ceph-mon[32617]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T13:55:30.088 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:29 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:30.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:29 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:55:30.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:29 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:55:30.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:30.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:29 smithi129 ceph-mon[32569]: osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] boot 2024-04-25T13:55:30.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:29 smithi129 ceph-mon[32569]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T13:55:30.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:29 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:30.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:29 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:55:30.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:29 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:55:30.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:30.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:29 smithi078 ceph-mon[29163]: osd.9 [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] boot 2024-04-25T13:55:30.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:29 smithi078 ceph-mon[29163]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T13:55:30.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:29 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:55:30.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-04-25T13:55:30.660 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:55:31.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:30 smithi112 ceph-mon[32617]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:31.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:30 smithi112 ceph-mon[32617]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T13:55:31.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:30 smithi129 ceph-mon[32569]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:31.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:30 smithi129 ceph-mon[32569]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T13:55:31.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:30 smithi078 ceph-mon[29163]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:31.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:30 smithi078 ceph-mon[29163]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi078 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-25T13:55:32.115 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.116 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:32.116 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:32.116 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.364+0000 7fa30e419700 1 -- 172.21.15.112:0/3381498437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104250 msgr2=0x7fa308104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:32.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 --2- 172.21.15.112:0/3381498437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104250 0x7fa308104660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009990 tx=0x7fa2fc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:55:32.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 -- 172.21.15.112:0/3381498437 shutdown_connections 2024-04-25T13:55:32.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 --2- 172.21.15.112:0/3381498437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104bf0 0x7fa30810b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:32.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 --2- 172.21.15.112:0/3381498437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa308103120 0x7fa308103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:32.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 --2- 172.21.15.112:0/3381498437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104250 0x7fa308104660 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:32.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 -- 172.21.15.112:0/3381498437 >> 172.21.15.112:0/3381498437 conn(0x7fa3080fe760 msgr2=0x7fa308100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 -- 172.21.15.112:0/3381498437 shutdown_connections 2024-04-25T13:55:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 -- 172.21.15.112:0/3381498437 wait complete. 2024-04-25T13:55:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 Processor -- start 2024-04-25T13:55:32.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.365+0000 7fa30e419700 1 -- start start 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30e419700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa308103120 0x7fa30819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30e419700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104250 0x7fa30819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30e419700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104bf0 0x7fa3081a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30e419700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810eb90 con 0x7fa308104250 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30e419700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810ea10 con 0x7fa308104bf0 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30e419700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810ed10 con 0x7fa308103120 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104250 0x7fa30819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104250 0x7fa30819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.112:44028/0 (socket says 172.21.15.112:44028) 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa307fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa308103120 0x7fa30819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30c9b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104bf0 0x7fa3081a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:32.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 -- 172.21.15.112:0/4228320107 learned_addr learned my addr 172.21.15.112:0/4228320107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 -- 172.21.15.112:0/4228320107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa308103120 msgr2=0x7fa30819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa308103120 0x7fa30819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 -- 172.21.15.112:0/4228320107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104bf0 msgr2=0x7fa3081a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104bf0 0x7fa3081a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa3077fe700 1 -- 172.21.15.112:0/4228320107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3081a2330 con 0x7fa308104250 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.366+0000 7fa30c9b6700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104bf0 0x7fa3081a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.367+0000 7fa3077fe700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104250 0x7fa30819ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006010 tx=0x7fa2fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.367+0000 7fa3057fa700 1 -- 172.21.15.112:0/4228320107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc027070 con 0x7fa308104250 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.367+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa308105470 con 0x7fa308104250 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.367+0000 7fa3057fa700 1 -- 172.21.15.112:0/4228320107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2fc004000 con 0x7fa308104250 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.368+0000 7fa3057fa700 1 -- 172.21.15.112:0/4228320107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc022d30 con 0x7fa308104250 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.368+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3081059a0 con 0x7fa308104250 2024-04-25T13:55:32.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.368+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa308051190 con 0x7fa308104250 2024-04-25T13:55:32.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.370+0000 7fa3057fa700 1 -- 172.21.15.112:0/4228320107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa2fc02a070 con 0x7fa308104250 2024-04-25T13:55:32.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.370+0000 7fa3057fa700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2f00afb80 0x7fa2f00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:32.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.370+0000 7fa3057fa700 1 -- 172.21.15.112:0/4228320107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fa2fc0df160 con 0x7fa308104250 2024-04-25T13:55:32.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.370+0000 7fa307fff700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2f00afb80 0x7fa2f00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:32.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.371+0000 7fa307fff700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2f00afb80 0x7fa2f00b2090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa308104150 tx=0x7fa2f8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:32.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.376+0000 7fa3057fa700 1 -- 172.21.15.112:0/4228320107 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2fc013070 con 0x7fa308104250 2024-04-25T13:55:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi078 2024-04-25T13:55:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-25T13:55:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:32.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi078 2024-04-25T13:55:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-25T13:55:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:32.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:32.583+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa308100780 con 0x7fa2f00afb80 2024-04-25T13:55:33.329 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:33 smithi112 ceph-mon[32617]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:33.329 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:33 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:33.329 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:33 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:33.329 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:33 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:33.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:33 smithi129 ceph-mon[32569]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:33.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:33 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:33.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:33 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:33.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:33 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:33 smithi078 ceph-mon[29163]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:33 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:33 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:33 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:34.291 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:34 smithi112 ceph-mon[32617]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:55:34.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:34 smithi129 ceph-mon[32569]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:55:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:34 smithi078 ceph-mon[29163]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:55:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:35 smithi129 ceph-mon[32569]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:35 smithi078 ceph-mon[29163]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:35 smithi112 ceph-mon[32617]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:36 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/3365690793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]: dispatch 2024-04-25T13:55:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:36 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]: dispatch 2024-04-25T13:55:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:36 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]': finished 2024-04-25T13:55:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:36 smithi112 ceph-mon[32617]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T13:55:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:36 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/3365690793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]: dispatch 2024-04-25T13:55:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:36 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]: dispatch 2024-04-25T13:55:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:36 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]': finished 2024-04-25T13:55:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:36 smithi129 ceph-mon[32569]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T13:55:36.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:36 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/3365690793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]: dispatch 2024-04-25T13:55:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:36 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]: dispatch 2024-04-25T13:55:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:36 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e1b875-eeac-47df-a4e1-df92fa867c4b"}]': finished 2024-04-25T13:55:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:36 smithi078 ceph-mon[29163]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T13:55:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:37 smithi129 ceph-mon[32569]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:37 smithi129 ceph-mon[32569]: from='client.? 172.21.15.112:0/982506519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:55:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:37 smithi078 ceph-mon[29163]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:37 smithi078 ceph-mon[29163]: from='client.? 172.21.15.112:0/982506519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:55:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:37 smithi112 ceph-mon[32617]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:37 smithi112 ceph-mon[32617]: from='client.? 172.21.15.112:0/982506519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:55:39.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:39 smithi129 ceph-mon[32569]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:39 smithi078 ceph-mon[29163]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:39.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:39 smithi112 ceph-mon[32617]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:41.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:41 smithi078 ceph-mon[29163]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:41.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:41 smithi129 ceph-mon[32569]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:41.508 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:41 smithi112 ceph-mon[32617]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:43 smithi078 ceph-mon[29163]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:55:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:43 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:43 smithi112 ceph-mon[32617]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:55:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:43 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:43 smithi129 ceph-mon[32569]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:55:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:43 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:44.432 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:44 smithi112 ceph-mon[32617]: Deploying daemon osd.10 on smithi112 2024-04-25T13:55:44.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:44 smithi078 ceph-mon[29163]: Deploying daemon osd.10 on smithi112 2024-04-25T13:55:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:44 smithi129 ceph-mon[32569]: Deploying daemon osd.10 on smithi112 2024-04-25T13:55:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:45 smithi078 ceph-mon[29163]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:45 smithi112 ceph-mon[32617]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:45.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:45 smithi129 ceph-mon[32569]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:47.446 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:47 smithi112 ceph-mon[32617]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:47 smithi078 ceph-mon[29163]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:47.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:47 smithi129 ceph-mon[32569]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:49.322 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:49 smithi112 ceph-mon[32617]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:49.322 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:49.322 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:49.322 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:49 smithi078 ceph-mon[29163]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:49.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:49 smithi129 ceph-mon[32569]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:49.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:51.454 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:51 smithi112 ceph-mon[32617]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:51 smithi078 ceph-mon[29163]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:51 smithi129 ceph-mon[32569]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:52.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.655+0000 7fa3057fa700 1 -- 172.21.15.112:0/4228320107 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa308100780 con 0x7fa2f00afb80 2024-04-25T13:55:52.656 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 10 on host 'smithi112' 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2f00afb80 msgr2=0x7fa2f00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2f00afb80 0x7fa2f00b2090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa308104150 tx=0x7fa2f8010040 comp rx=0 tx=0).stop 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104250 msgr2=0x7fa30819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104250 0x7fa30819ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006010 tx=0x7fa2fc0047b0 comp rx=0 tx=0).stop 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 shutdown_connections 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2f00afb80 0x7fa2f00b2090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa308104bf0 0x7fa3081a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa308103120 0x7fa30819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 --2- 172.21.15.112:0/4228320107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa308104250 0x7fa30819ad10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:52.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 >> 172.21.15.112:0/4228320107 conn(0x7fa3080fe760 msgr2=0x7fa308100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:52.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 shutdown_connections 2024-04-25T13:55:52.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T13:55:52.658+0000 7fa30e419700 1 -- 172.21.15.112:0/4228320107 wait complete. 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:55:52.978 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:52.979 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:52.979 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:53.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:55:53.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:55:53.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:53.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:55:53.306 DEBUG:teuthology.orchestra.run.smithi112:osd.10> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T13:55:53.308 INFO:tasks.cephadm:Deploying osd.11 on smithi129 with /dev/vg_nvme/lv_1... 2024-04-25T13:55:53.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:55:53.554 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:55:54.014 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:53 smithi129 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:55:54.014 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:53 smithi129 ceph-mon[32569]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T13:55:54.014 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:53 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:54.014 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:53 smithi129 ceph-mon[32569]: from='osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:55:54.014 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:53 smithi129 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:55:54.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:53 smithi112 ceph-mon[32617]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:55:54.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:53 smithi112 ceph-mon[32617]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T13:55:54.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:53 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:54.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:53 smithi112 ceph-mon[32617]: from='osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:55:54.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:53 smithi112 ceph-mon[32617]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:55:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:53 smithi078 ceph-mon[29163]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:55:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:53 smithi078 ceph-mon[29163]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T13:55:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:53 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:53 smithi078 ceph-mon[29163]: from='osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:55:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:53 smithi078 ceph-mon[29163]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T13:55:54.462 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T13:55:54.187+0000 7f1f331d5700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T13:55:54.462 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T13:55:54.203+0000 7f1f2af82700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:55:54.957 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:54 smithi129 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:55:54.957 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:54 smithi129 ceph-mon[32569]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T13:55:54.957 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:54 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:54.957 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:54 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:54.957 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:54 smithi129 ceph-mon[32569]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:54.957 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:54 smithi129 ceph-mon[32569]: from='osd.10 ' entity='osd.10' 2024-04-25T13:55:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-mon[32617]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:55:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-mon[32617]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T13:55:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-mon[32617]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:54 smithi112 ceph-mon[32617]: from='osd.10 ' entity='osd.10' 2024-04-25T13:55:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:54 smithi078 ceph-mon[29163]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T13:55:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:54 smithi078 ceph-mon[29163]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T13:55:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:54 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:54 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:54 smithi078 ceph-mon[29163]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:55:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:54 smithi078 ceph-mon[29163]: from='osd.10 ' entity='osd.10' 2024-04-25T13:55:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:55:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:55:55.650 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-25T13:55:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-25T13:55:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142748 s, 735 MB/s 2024-04-25T13:55:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-25T13:55:55.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:55 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:55:55.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:55 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:55:55.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:55.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:55 smithi129 ceph-mon[32569]: osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] boot 2024-04-25T13:55:55.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:55 smithi129 ceph-mon[32569]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T13:55:55.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:56.004 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:55 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:55:56.004 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:55 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:55:56.004 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:56.004 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:55 smithi112 ceph-mon[32617]: osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] boot 2024-04-25T13:55:56.004 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:55 smithi112 ceph-mon[32617]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T13:55:56.004 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:56.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:55 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:55:56.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:55 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:55:56.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:56.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:55 smithi078 ceph-mon[29163]: osd.10 [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] boot 2024-04-25T13:55:56.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:55 smithi078 ceph-mon[29163]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T13:55:56.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:55:56.316 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-04-25T13:55:56.560 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:55:57.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T13:55:57.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:57.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T13:55:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:55:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T13:55:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:55:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:55:57.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:57.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:57.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:55:57.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:55:58.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 -- 172.21.15.129:0/4266678596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 msgr2=0x7f7e5410c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:58.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/4266678596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5410c5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b0a0 tx=0x7f7e4c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:55:58.225 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:58 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi112 2024-04-25T13:55:58.225 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:58 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T13:55:58.225 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:58 smithi129 ceph-mon[32569]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 -- 172.21.15.129:0/4266678596 shutdown_connections 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/4266678596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5410c5c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/4266678596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e54105880 0x7f7e54105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/4266678596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e54103e30 0x7f7e54104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 -- 172.21.15.129:0/4266678596 >> 172.21.15.129:0/4266678596 conn(0x7f7e540ff8a0 msgr2=0x7f7e54101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 -- 172.21.15.129:0/4266678596 shutdown_connections 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 -- 172.21.15.129:0/4266678596 wait complete. 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 Processor -- start 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.224+0000 7f7e5a3de700 1 -- start start 2024-04-25T13:55:58.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5a3de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e54103e30 0x7f7e5419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5a3de700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5a3de700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e54105880 0x7f7e541a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5a3de700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410fe30 con 0x7f7e54105880 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5a3de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410fcb0 con 0x7f7e54103e30 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5a3de700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410ffb0 con 0x7f7e541048b0 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33424/0 (socket says 172.21.15.129:33424) 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 -- 172.21.15.129:0/827820372 learned_addr learned my addr 172.21.15.129:0/827820372 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e53fff700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e54103e30 0x7f7e5419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5897b700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e54105880 0x7f7e541a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 -- 172.21.15.129:0/827820372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e54103e30 msgr2=0x7f7e5419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e54103e30 0x7f7e5419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 -- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e54105880 msgr2=0x7f7e541a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:55:58.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e54105880 0x7f7e541a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e537fe700 1 -- 172.21.15.129:0/827820372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e541a2420 con 0x7f7e541048b0 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.225+0000 7f7e5897b700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e54105880 0x7f7e541a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.226+0000 7f7e537fe700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5419ae00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400d950 tx=0x7f7e4400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.226+0000 7f7e517fa700 1 -- 172.21.15.129:0/827820372 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44016070 con 0x7f7e541048b0 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.226+0000 7f7e517fa700 1 -- 172.21.15.129:0/827820372 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e44009df0 con 0x7f7e541048b0 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.226+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e541a2710 con 0x7f7e541048b0 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.226+0000 7f7e517fa700 1 -- 172.21.15.129:0/827820372 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400f3c0 con 0x7f7e541048b0 2024-04-25T13:55:58.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.226+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e541a2d20 con 0x7f7e541048b0 2024-04-25T13:55:58.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.227+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e54051190 con 0x7f7e541048b0 2024-04-25T13:55:58.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.229+0000 7f7e517fa700 1 -- 172.21.15.129:0/827820372 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7e44019070 con 0x7f7e541048b0 2024-04-25T13:55:58.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.230+0000 7f7e517fa700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7e400afc00 0x7f7e400b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:55:58.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.230+0000 7f7e517fa700 1 -- 172.21.15.129:0/827820372 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f7e440de120 con 0x7f7e541048b0 2024-04-25T13:55:58.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.230+0000 7f7e53fff700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7e400afc00 0x7f7e400b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:55:58.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.231+0000 7f7e53fff700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7e400afc00 0x7f7e400b2110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e54105270 tx=0x7f7e3c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:55:58.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.234+0000 7f7e517fa700 1 -- 172.21.15.129:0/827820372 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e440a7350 con 0x7f7e541048b0 2024-04-25T13:55:58.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:55:58.438+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7e5402e880 con 0x7f7e400afc00 2024-04-25T13:55:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:58 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi112 2024-04-25T13:55:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:58 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T13:55:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:58 smithi078 ceph-mon[29163]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T13:55:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:58 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi112 2024-04-25T13:55:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:58 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T13:55:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:58 smithi112 ceph-mon[32617]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T13:55:59.312 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:59 smithi129 ceph-mon[32569]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:55:59.313 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:59 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:59.313 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:59 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:59.313 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:55:59 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:59.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:59 smithi078 ceph-mon[29163]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:55:59.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:59 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:59.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:59 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:59.371 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:55:59 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:55:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:59 smithi112 ceph-mon[32617]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:55:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:59 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:55:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:59 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:55:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:55:59 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:00.358 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:00 smithi129 ceph-mon[32569]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:56:00.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:00 smithi078 ceph-mon[29163]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:56:00.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:00 smithi112 ceph-mon[32617]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:56:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:01 smithi129 ceph-mon[32569]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:56:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:01 smithi078 ceph-mon[29163]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:56:01.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:01 smithi112 ceph-mon[32617]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:56:02.072 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:02 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/2041866872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]: dispatch 2024-04-25T13:56:02.072 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:02 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]: dispatch 2024-04-25T13:56:02.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:02 smithi129 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]': finished 2024-04-25T13:56:02.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:02 smithi129 ceph-mon[32569]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T13:56:02.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:02 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:02.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:02 smithi129 ceph-mon[32569]: from='client.? 172.21.15.129:0/3233608291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:56:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:02 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/2041866872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]: dispatch 2024-04-25T13:56:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:02 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]: dispatch 2024-04-25T13:56:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:02 smithi078 ceph-mon[29163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]': finished 2024-04-25T13:56:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:02 smithi078 ceph-mon[29163]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T13:56:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:02 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:02 smithi078 ceph-mon[29163]: from='client.? 172.21.15.129:0/3233608291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:56:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/2041866872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]: dispatch 2024-04-25T13:56:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:02 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]: dispatch 2024-04-25T13:56:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:02 smithi112 ceph-mon[32617]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78830a45-c9f6-4c81-93ae-dbb8a1dc61db"}]': finished 2024-04-25T13:56:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:02 smithi112 ceph-mon[32617]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T13:56:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:02 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.129:0/3233608291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:56:03.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:03 smithi129 ceph-mon[32569]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:56:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:03 smithi078 ceph-mon[29163]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:56:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:03 smithi112 ceph-mon[32617]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:56:05.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:05 smithi129 ceph-mon[32569]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:56:05.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:05 smithi078 ceph-mon[29163]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:56:05.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:05 smithi112 ceph-mon[32617]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:56:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:07 smithi078 ceph-mon[29163]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:56:07.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:07 smithi112 ceph-mon[32617]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:56:07.643 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:07 smithi129 ceph-mon[32569]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:56:09.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:09 smithi129 ceph-mon[32569]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:56:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:09 smithi078 ceph-mon[29163]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:56:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:09 smithi112 ceph-mon[32617]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:56:10.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:10 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:56:10.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:10 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:10 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:56:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:10 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:10 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:56:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:10 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:11.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:11 smithi129 ceph-mon[32569]: Deploying daemon osd.11 on smithi129 2024-04-25T13:56:11.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:11 smithi129 ceph-mon[32569]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:11 smithi078 ceph-mon[29163]: Deploying daemon osd.11 on smithi129 2024-04-25T13:56:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:11 smithi078 ceph-mon[29163]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:11 smithi112 ceph-mon[32617]: Deploying daemon osd.11 on smithi129 2024-04-25T13:56:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:11 smithi112 ceph-mon[32617]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:13.358 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:13 smithi129 ceph-mon[32569]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:13 smithi078 ceph-mon[29163]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:13 smithi112 ceph-mon[32617]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:15 smithi129 ceph-mon[32569]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:15 smithi078 ceph-mon[29163]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:15 smithi112 ceph-mon[32617]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:16 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:16 smithi078 ceph-mon[29163]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:16 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:16 smithi112 ceph-mon[32617]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:16.551 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:16.552 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:16.552 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:16 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:16.552 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:16 smithi129 ceph-mon[32569]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:19.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.256+0000 7f7e517fa700 1 -- 172.21.15.129:0/827820372 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7e5402e880 con 0x7f7e400afc00 2024-04-25T13:56:19.256 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 11 on host 'smithi129' 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.258+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7e400afc00 msgr2=0x7f7e400b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.258+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7e400afc00 0x7f7e400b2110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e54105270 tx=0x7f7e3c009190 comp rx=0 tx=0).stop 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.258+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 msgr2=0x7f7e5419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.258+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5419ae00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400d950 tx=0x7f7e4400dc60 comp rx=0 tx=0).stop 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 shutdown_connections 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7e400afc00 0x7f7e400b2110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e54105880 0x7f7e541a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:19.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e54103e30 0x7f7e5419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:19.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 --2- 172.21.15.129:0/827820372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e541048b0 0x7f7e5419ae00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:19.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 >> 172.21.15.129:0/827820372 conn(0x7f7e540ff8a0 msgr2=0x7f7e541085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:19.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 shutdown_connections 2024-04-25T13:56:19.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:19.259+0000 7f7e5a3de700 1 -- 172.21.15.129:0/827820372 wait complete. 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: from='osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:56:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:19 smithi078 ceph-mon[29163]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:56:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: from='osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:56:19.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:19 smithi112 ceph-mon[32617]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:56:19.559 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:19.559 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.560 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.560 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:19.560 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:19.560 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:19.560 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: from='osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:56:19.560 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:19 smithi129 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:56:19.880 DEBUG:teuthology.orchestra.run.smithi129:osd.11> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T13:56:19.882 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T13:56:19.883 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:56:20.127 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:56:20.490 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:20.490 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:20.490 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:20.490 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:56:20.490 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: from='osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:56:20.490 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T13:56:20.491 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:20.491 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:56:20.491 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:20 smithi112 ceph-mon[32617]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:20.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:20.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:20.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:20.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:56:20.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: from='osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:56:20.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T13:56:20.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:20.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:56:20.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:20 smithi078 ceph-mon[29163]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: from='osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-25T13:56:20.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-mon[32569]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:20.665 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T13:56:20.314+0000 7fd1311ef700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T13:56:20.665 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 13:56:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T13:56:20.330+0000 7fd12cfa4700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:56:21.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:21 smithi078 ceph-mon[29163]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:56:21.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:21 smithi078 ceph-mon[29163]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T13:56:21.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:21.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:21 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:21 smithi129 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:56:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:21 smithi129 ceph-mon[32569]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T13:56:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:21 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:21 smithi112 ceph-mon[32617]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-25T13:56:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:21 smithi112 ceph-mon[32617]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T13:56:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:21 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 -- 172.21.15.78:0/2684708849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 msgr2=0x7efcf8103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 --2- 172.21.15.78:0/2684708849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8103510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efce8009990 tx=0x7efce8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 -- 172.21.15.78:0/2684708849 shutdown_connections 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 --2- 172.21.15.78:0/2684708849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf8104bd0 0x7efcf810b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 --2- 172.21.15.78:0/2684708849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8103510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 --2- 172.21.15.78:0/2684708849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcf8104230 0x7efcf8104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 -- 172.21.15.78:0/2684708849 >> 172.21.15.78:0/2684708849 conn(0x7efcf80fe740 msgr2=0x7efcf8100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 -- 172.21.15.78:0/2684708849 shutdown_connections 2024-04-25T13:56:21.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.892+0000 7efcfd21d700 1 -- 172.21.15.78:0/2684708849 wait complete. 2024-04-25T13:56:21.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 Processor -- start 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 -- start start 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf8104230 0x7efcf81a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcf8104bd0 0x7efcf81a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcf810eb40 con 0x7efcf8103100 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcf810e9c0 con 0x7efcf8104230 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcfd21d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcf810ecc0 con 0x7efcf8104bd0 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8076320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49140/0 (socket says 172.21.15.78:49140) 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 -- 172.21.15.78:0/3300345794 learned_addr learned my addr 172.21.15.78:0/3300345794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf759e700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcf8104bd0 0x7efcf81a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf659c700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf8104230 0x7efcf81a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:21.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 -- 172.21.15.78:0/3300345794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcf8104bd0 msgr2=0x7efcf81a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcf8104bd0 0x7efcf81a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 -- 172.21.15.78:0/3300345794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf8104230 msgr2=0x7efcf81a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf8104230 0x7efcf81a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.893+0000 7efcf6d9d700 1 -- 172.21.15.78:0/3300345794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf81a4560 con 0x7efcf8103100 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.894+0000 7efcf6d9d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8076320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efce8000c00 tx=0x7efce80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.894+0000 7efcdffff700 1 -- 172.21.15.78:0/3300345794 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8004830 con 0x7efcf8103100 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.894+0000 7efcdffff700 1 -- 172.21.15.78:0/3300345794 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efce8005020 con 0x7efcf8103100 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.894+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf81a47f0 con 0x7efcf8103100 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.894+0000 7efcdffff700 1 -- 172.21.15.78:0/3300345794 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8003d90 con 0x7efcf8103100 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.894+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81a4be0 con 0x7efcf8103100 2024-04-25T13:56:21.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.895+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf8051190 con 0x7efcf8103100 2024-04-25T13:56:21.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.896+0000 7efcdffff700 1 -- 172.21.15.78:0/3300345794 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7efce80051d0 con 0x7efcf8103100 2024-04-25T13:56:21.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.897+0000 7efcdffff700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efce00afc00 0x7efce00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:21.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.897+0000 7efcf659c700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efce00afc00 0x7efce00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:21.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.897+0000 7efcdffff700 1 -- 172.21.15.78:0/3300345794 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7efce80dfba0 con 0x7efcf8103100 2024-04-25T13:56:21.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.897+0000 7efcf659c700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efce00afc00 0x7efce00b2110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efcf8075cd0 tx=0x7efcec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:21.901+0000 7efcdffff700 1 -- 172.21.15.78:0/3300345794 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efce8013070 con 0x7efcf8103100 2024-04-25T13:56:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.099+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7efcf80642a0 con 0x7efcf8103100 2024-04-25T13:56:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.099+0000 7efcdffff700 1 -- 172.21.15.78:0/3300345794 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7efce80a8c10 con 0x7efcf8103100 2024-04-25T13:56:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.101+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efce00afc00 msgr2=0x7efce00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.101+0000 7efcfd21d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efce00afc00 0x7efce00b2110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efcf8075cd0 tx=0x7efcec00b410 comp rx=0 tx=0).stop 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 msgr2=0x7efcf8076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8076320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efce8000c00 tx=0x7efce80047b0 comp rx=0 tx=0).stop 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 shutdown_connections 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efce00afc00 0x7efce00b2110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcf8104bd0 0x7efcf81a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcf8103100 0x7efcf8076320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 --2- 172.21.15.78:0/3300345794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcf8104230 0x7efcf81a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 >> 172.21.15.78:0/3300345794 conn(0x7efcf80fe740 msgr2=0x7efcf8107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 shutdown_connections 2024-04-25T13:56:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:22.102+0000 7efcfd21d700 1 -- 172.21.15.78:0/3300345794 wait complete. 2024-04-25T13:56:22.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T13:56:22.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T13:56:22.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:22.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] boot 2024-04-25T13:56:22.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T13:56:22.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:22.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3300345794' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:56:22.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:22 smithi129 ceph-mon[32569]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: purged_snaps scrub starts 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: purged_snaps scrub ok 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] boot 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3300345794' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:56:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:22 smithi078 ceph-mon[29163]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: purged_snaps scrub starts 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: purged_snaps scrub ok 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: osd.11 [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] boot 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3300345794' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:56:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:22 smithi112 ceph-mon[32617]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:56:22.832 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1714053381,"num_in_osds":12,"osd_in_since":1714053361,"num_remapped_pgs":0} 2024-04-25T13:56:22.833 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:56:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: Detected new or changed devices on smithi129 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-25T13:56:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:23 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: Detected new or changed devices on smithi129 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:23 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T13:56:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: Detected new or changed devices on smithi129 2024-04-25T13:56:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:56:23.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-25T13:56:23.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:23.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:23.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:23.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:23 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:24.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:24 smithi078 ceph-mon[29163]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:24 smithi129 ceph-mon[32569]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:24.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:24 smithi112 ceph-mon[32617]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:24.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 -- 172.21.15.78:0/3047287671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 msgr2=0x7f4cd4102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:24.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 --2- 172.21.15.78:0/3047287671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd4102b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009990 tx=0x7f4cc4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:56:24.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 -- 172.21.15.78:0/3047287671 shutdown_connections 2024-04-25T13:56:24.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 --2- 172.21.15.78:0/3047287671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cd41030a0 0x7f4cd4111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:24.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 --2- 172.21.15.78:0/3047287671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd41091d0 0x7f4cd41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:24.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 --2- 172.21.15.78:0/3047287671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd4102b60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:24.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 -- 172.21.15.78:0/3047287671 >> 172.21.15.78:0/3047287671 conn(0x7f4cd40fe5c0 msgr2=0x7f4cd41009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:24.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 -- 172.21.15.78:0/3047287671 shutdown_connections 2024-04-25T13:56:24.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.821+0000 7f4cdc212700 1 -- 172.21.15.78:0/3047287671 wait complete. 2024-04-25T13:56:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 Processor -- start 2024-04-25T13:56:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 -- start start 2024-04-25T13:56:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd41030a0 0x7f4cd419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cd41091d0 0x7f4cd419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4114470 con 0x7f4cd4102750 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41142f0 con 0x7f4cd41030a0 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cdc212700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41145f0 con 0x7f4cd41091d0 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49150/0 (socket says 172.21.15.78:49150) 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 -- 172.21.15.78:0/2577176754 learned_addr learned my addr 172.21.15.78:0/2577176754 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cda7af700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cd41091d0 0x7f4cd419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd97ad700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd41030a0 0x7f4cd419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 -- 172.21.15.78:0/2577176754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cd41091d0 msgr2=0x7f4cd419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cd41091d0 0x7f4cd419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 -- 172.21.15.78:0/2577176754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd41030a0 msgr2=0x7f4cd419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd41030a0 0x7f4cd419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd9fae700 1 -- 172.21.15.78:0/2577176754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd419fd10 con 0x7f4cd4102750 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.822+0000 7f4cd97ad700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd41030a0 0x7f4cd419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.823+0000 7f4cd9fae700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd419ae40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000c970 tx=0x7f4cd000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.823+0000 7f4ccaffd700 1 -- 172.21.15.78:0/2577176754 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000ce60 con 0x7f4cd4102750 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.823+0000 7f4ccaffd700 1 -- 172.21.15.78:0/2577176754 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd0007900 con 0x7f4cd4102750 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.823+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd41a0000 con 0x7f4cd4102750 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.823+0000 7f4ccaffd700 1 -- 172.21.15.78:0/2577176754 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0003c70 con 0x7f4cd4102750 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.823+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd41a7920 con 0x7f4cd4102750 2024-04-25T13:56:24.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.824+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd4051190 con 0x7f4cd4102750 2024-04-25T13:56:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.826+0000 7f4ccaffd700 1 -- 172.21.15.78:0/2577176754 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4cd0004220 con 0x7f4cd4102750 2024-04-25T13:56:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.827+0000 7f4ccaffd700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4cc00afc00 0x7f4cc00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.827+0000 7f4cd97ad700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4cc00afc00 0x7f4cc00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.827+0000 7f4ccaffd700 1 -- 172.21.15.78:0/2577176754 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f4cd00de5b0 con 0x7f4cd4102750 2024-04-25T13:56:24.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.827+0000 7f4cd97ad700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4cc00afc00 0x7f4cc00b2110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4003800 tx=0x7f4cc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:24.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:24.830+0000 7f4ccaffd700 1 -- 172.21.15.78:0/2577176754 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cd0014030 con 0x7f4cd4102750 2024-04-25T13:56:25.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.030+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cd419c1a0 con 0x7f4cd4102750 2024-04-25T13:56:25.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.031+0000 7f4ccaffd700 1 -- 172.21.15.78:0/2577176754 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18480 (secure 0 0 0) 0x7f4cd0004510 con 0x7f4cd4102750 2024-04-25T13:56:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:25.031 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:56:22.314977+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:56:25.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4cc00afc00 msgr2=0x7f4cc00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4cc00afc00 0x7f4cc00b2110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4003800 tx=0x7f4cc4005fb0 comp rx=0 tx=0).stop 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 msgr2=0x7f4cd419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd419ae40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000c970 tx=0x7f4cd000cc80 comp rx=0 tx=0).stop 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 shutdown_connections 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4cc00afc00 0x7f4cc00b2110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4cd41091d0 0x7f4cd419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd4102750 0x7f4cd419ae40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 --2- 172.21.15.78:0/2577176754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd41030a0 0x7f4cd419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.033+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 >> 172.21.15.78:0/2577176754 conn(0x7f4cd40fe5c0 msgr2=0x7f4cd410a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.034+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 shutdown_connections 2024-04-25T13:56:25.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:25.034+0000 7f4cdc212700 1 -- 172.21.15.78:0/2577176754 wait complete. 2024-04-25T13:56:25.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:25 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2577176754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:56:25.709 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:25 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2577176754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:56:25.738 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:56:25.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T13:56:25.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:25 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2577176754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:56:25.970 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:56:26.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:26 smithi078 ceph-mon[29163]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:26.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:26 smithi129 ceph-mon[32569]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:26.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:26 smithi112 ceph-mon[32617]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 -- 172.21.15.78:0/2704529561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30108db0 msgr2=0x7fbe301091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 --2- 172.21.15.78:0/2704529561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30108db0 0x7fbe301091c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe200099c0 tx=0x7fbe20009cd0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 -- 172.21.15.78:0/2704529561 shutdown_connections 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 --2- 172.21.15.78:0/2704529561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe30109700 0x7fbe30111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 --2- 172.21.15.78:0/2704529561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe30104480 0x7fbe30104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 --2- 172.21.15.78:0/2704529561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30108db0 0x7fbe301091c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 -- 172.21.15.78:0/2704529561 >> 172.21.15.78:0/2704529561 conn(0x7fbe300ffd70 msgr2=0x7fbe301021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 -- 172.21.15.78:0/2704529561 shutdown_connections 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 -- 172.21.15.78:0/2704529561 wait complete. 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.765+0000 7fbe37a33700 1 Processor -- start 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe37a33700 1 -- start start 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe37a33700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe30104480 0x7fbe301a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe37a33700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe30108db0 0x7fbe301a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe37a33700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30109700 0x7fbe3019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe37a33700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30114310 con 0x7fbe30109700 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe37a33700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30114190 con 0x7fbe30108db0 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe37a33700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30114490 con 0x7fbe30104480 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30109700 0x7fbe3019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30109700 0x7fbe3019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40868/0 (socket says 172.21.15.78:40868) 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 -- 172.21.15.78:0/3997800052 learned_addr learned my addr 172.21.15.78:0/3997800052 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe34fce700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe30108db0 0x7fbe301a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe357cf700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe30104480 0x7fbe301a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 -- 172.21.15.78:0/3997800052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe30104480 msgr2=0x7fbe301a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe30104480 0x7fbe301a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 -- 172.21.15.78:0/3997800052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe30108db0 msgr2=0x7fbe301a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe30108db0 0x7fbe301a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 -- 172.21.15.78:0/3997800052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3019b160 con 0x7fbe30109700 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe34fce700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe30108db0 0x7fbe301a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.766+0000 7fbe35fd0700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30109700 0x7fbe3019aab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe28007ee0 tx=0x7fbe2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.767+0000 7fbe267fc700 1 -- 172.21.15.78:0/3997800052 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800dbd0 con 0x7fbe30109700 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.767+0000 7fbe267fc700 1 -- 172.21.15.78:0/3997800052 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe2800dd70 con 0x7fbe30109700 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.767+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3019b450 con 0x7fbe30109700 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.767+0000 7fbe267fc700 1 -- 172.21.15.78:0/3997800052 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28005990 con 0x7fbe30109700 2024-04-25T13:56:27.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.767+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe301a7720 con 0x7fbe30109700 2024-04-25T13:56:27.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.768+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe30105930 con 0x7fbe30109700 2024-04-25T13:56:27.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.769+0000 7fbe267fc700 1 -- 172.21.15.78:0/3997800052 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbe2801a020 con 0x7fbe30109700 2024-04-25T13:56:27.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.769+0000 7fbe267fc700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbe1c0afb80 0x7fbe1c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:27.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.770+0000 7fbe357cf700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbe1c0afb80 0x7fbe1c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:27.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.770+0000 7fbe267fc700 1 -- 172.21.15.78:0/3997800052 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fbe280ddfa0 con 0x7fbe30109700 2024-04-25T13:56:27.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.770+0000 7fbe357cf700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbe1c0afb80 0x7fbe1c0b2090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c005950 tx=0x7fbe2c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:27.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.774+0000 7fbe267fc700 1 -- 172.21.15.78:0/3997800052 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe28010070 con 0x7fbe30109700 2024-04-25T13:56:27.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.970+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe3019bfe0 con 0x7fbe30109700 2024-04-25T13:56:27.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.970+0000 7fbe267fc700 1 -- 172.21.15.78:0/3997800052 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fbe280a7020 con 0x7fbe30109700 2024-04-25T13:56:27.970 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:56:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.972+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbe1c0afb80 msgr2=0x7fbe1c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.972+0000 7fbe37a33700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbe1c0afb80 0x7fbe1c0b2090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c005950 tx=0x7fbe2c0058e0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.972+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30109700 msgr2=0x7fbe3019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.972+0000 7fbe37a33700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30109700 0x7fbe3019aab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe28007ee0 tx=0x7fbe2800d3b0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.972+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 shutdown_connections 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.972+0000 7fbe37a33700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbe1c0afb80 0x7fbe1c0b2090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.972+0000 7fbe37a33700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe30109700 0x7fbe3019aab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.973+0000 7fbe37a33700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe30104480 0x7fbe301a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.973+0000 7fbe37a33700 1 --2- 172.21.15.78:0/3997800052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe30108db0 0x7fbe301a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.973+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 >> 172.21.15.78:0/3997800052 conn(0x7fbe300ffd70 msgr2=0x7fbe30108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.973+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 shutdown_connections 2024-04-25T13:56:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:27.973+0000 7fbe37a33700 1 -- 172.21.15.78:0/3997800052 wait complete. 2024-04-25T13:56:28.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:28 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3997800052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:56:28.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:28 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3997800052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:56:28.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:28 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3997800052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:56:28.782 INFO:tasks.cephadm:Adding mds.a on smithi078 2024-04-25T13:56:28.782 INFO:tasks.cephadm:Adding mds.d on smithi078 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.g on smithi078 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.j on smithi078 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.b on smithi112 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.e on smithi112 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.h on smithi112 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.k on smithi112 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.c on smithi129 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.f on smithi129 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.i on smithi129 2024-04-25T13:56:28.783 INFO:tasks.cephadm:Adding mds.l on smithi129 2024-04-25T13:56:28.783 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l' 2024-04-25T13:56:29.034 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:56:29.156 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:29 smithi129 ceph-mon[32569]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:29 smithi078 ceph-mon[29163]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:29 smithi112 ceph-mon[32617]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:30.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 -- 172.21.15.129:0/1295532472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104210 msgr2=0x7fc4e0104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/1295532472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104210 0x7fc4e0104620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009990 tx=0x7fc4c8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 -- 172.21.15.129:0/1295532472 shutdown_connections 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/1295532472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4e0104bb0 0x7fc4e010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/1295532472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e01030e0 0x7fc4e01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/1295532472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104210 0x7fc4e0104620 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 -- 172.21.15.129:0/1295532472 >> 172.21.15.129:0/1295532472 conn(0x7fc4e00fe740 msgr2=0x7fc4e0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 -- 172.21.15.129:0/1295532472 shutdown_connections 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.701+0000 7fc4e5b10700 1 -- 172.21.15.129:0/1295532472 wait complete. 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 Processor -- start 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 -- start start 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e01030e0 0x7fc4e01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:30.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4e0104210 0x7fc4e0199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104bb0 0x7fc4e01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e010eb60 con 0x7fc4e01030e0 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e010e9e0 con 0x7fc4e0104210 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.702+0000 7fc4e5b10700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e010ece0 con 0x7fc4e0104bb0 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104bb0 0x7fc4e01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104bb0 0x7fc4e01a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43634/0 (socket says 172.21.15.129:43634) 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 -- 172.21.15.129:0/2403549534 learned_addr learned my addr 172.21.15.129:0/2403549534 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4df7fe700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e01030e0 0x7fc4e01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4d6ffd700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4e0104210 0x7fc4e0199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 -- 172.21.15.129:0/2403549534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4e0104210 msgr2=0x7fc4e0199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4e0104210 0x7fc4e0199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 -- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e01030e0 msgr2=0x7fc4e01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e01030e0 0x7fc4e01996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 -- 172.21.15.129:0/2403549534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e01a11a0 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4d6ffd700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4e0104210 0x7fc4e0199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dffff700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104bb0 0x7fc4e01a0bb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800ada0 tx=0x7fc4d800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dd7fa700 1 -- 172.21.15.129:0/2403549534 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8017020 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dd7fa700 1 -- 172.21.15.129:0/2403549534 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d800ce30 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4dd7fa700 1 -- 172.21.15.129:0/2403549534 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8005350 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e01a1490 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.703+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e01a19d0 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.704+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e0051190 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.706+0000 7fc4dd7fa700 1 -- 172.21.15.129:0/2403549534 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc4d800c980 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.707+0000 7fc4dd7fa700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4cc0afc00 0x7fc4cc0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.707+0000 7fc4dd7fa700 1 -- 172.21.15.129:0/2403549534 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc4d80de3f0 con 0x7fc4e0104bb0 2024-04-25T13:56:30.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.707+0000 7fc4df7fe700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4cc0afc00 0x7fc4cc0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:30.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.708+0000 7fc4df7fe700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4cc0afc00 0x7fc4cc0b2110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0104110 tx=0x7fc4d000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:30.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.710+0000 7fc4dd7fa700 1 -- 172.21.15.129:0/2403549534 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4d8021020 con 0x7fc4e0104bb0 2024-04-25T13:56:30.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.919+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}) -- 0x7fc4e0063d90 con 0x7fc4cc0afc00 2024-04-25T13:56:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.928+0000 7fc4dd7fa700 1 -- 172.21.15.129:0/2403549534 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc4e0063d90 con 0x7fc4cc0afc00 2024-04-25T13:56:30.927 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mds.all update... 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4cc0afc00 msgr2=0x7fc4cc0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4cc0afc00 0x7fc4cc0b2110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0104110 tx=0x7fc4d000a3b0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104bb0 msgr2=0x7fc4e01a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104bb0 0x7fc4e01a0bb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800ada0 tx=0x7fc4d800c330 comp rx=0 tx=0).stop 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 shutdown_connections 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4cc0afc00 0x7fc4cc0b2110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4e0104bb0 0x7fc4e01a0bb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e01030e0 0x7fc4e01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 --2- 172.21.15.129:0/2403549534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4e0104210 0x7fc4e0199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:30.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 >> 172.21.15.129:0/2403549534 conn(0x7fc4e00fe740 msgr2=0x7fc4e0107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:30.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.930+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 shutdown_connections 2024-04-25T13:56:30.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T13:56:30.931+0000 7fc4e5b10700 1 -- 172.21.15.129:0/2403549534 wait complete. 2024-04-25T13:56:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:31 smithi078 ceph-mon[29163]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:31.481 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:31 smithi112 ceph-mon[32617]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:31.481 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:31.481 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:31.559 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:31 smithi129 ceph-mon[32569]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:31.559 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:31.559 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:56:31.590 DEBUG:teuthology.orchestra.run.smithi078:mds.a> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T13:56:31.593 DEBUG:teuthology.orchestra.run.smithi078:mds.d> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T13:56:31.636 DEBUG:teuthology.orchestra.run.smithi078:mds.g> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T13:56:31.678 DEBUG:teuthology.orchestra.run.smithi078:mds.j> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T13:56:31.722 DEBUG:teuthology.orchestra.run.smithi112:mds.b> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T13:56:31.725 DEBUG:teuthology.orchestra.run.smithi112:mds.e> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T13:56:31.768 DEBUG:teuthology.orchestra.run.smithi112:mds.h> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T13:56:31.811 DEBUG:teuthology.orchestra.run.smithi112:mds.k> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T13:56:31.854 DEBUG:teuthology.orchestra.run.smithi129:mds.c> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T13:56:31.857 DEBUG:teuthology.orchestra.run.smithi129:mds.f> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T13:56:31.899 DEBUG:teuthology.orchestra.run.smithi129:mds.i> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T13:56:31.943 DEBUG:teuthology.orchestra.run.smithi129:mds.l> sudo journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T13:56:31.986 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T13:56:31.986 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 -- 172.21.15.78:0/492612911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 msgr2=0x7fe44c10be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 --2- 172.21.15.78:0/492612911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c10be70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe444009ca0 tx=0x7fe44402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 -- 172.21.15.78:0/492612911 shutdown_connections 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 --2- 172.21.15.78:0/492612911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c10be70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 --2- 172.21.15.78:0/492612911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe44c104d50 0x7fe44c105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 --2- 172.21.15.78:0/492612911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44c103b10 0x7fe44c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 -- 172.21.15.78:0/492612911 >> 172.21.15.78:0/492612911 conn(0x7fe44c0fed40 msgr2=0x7fe44c1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 -- 172.21.15.78:0/492612911 shutdown_connections 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 -- 172.21.15.78:0/492612911 wait complete. 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 Processor -- start 2024-04-25T13:56:32.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.103+0000 7fe452357640 1 -- start start 2024-04-25T13:56:32.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44c103b10 0x7fe44c077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe44c104d50 0x7fe44c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c10f640 con 0x7fe44c109a50 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c10f4c0 con 0x7fe44c103b10 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c10f7c0 con 0x7fe44c104d50 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40882/0 (socket says 172.21.15.78:40882) 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 -- 172.21.15.78:0/3292393789 learned_addr learned my addr 172.21.15.78:0/3292393789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe44b7fe640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe44c104d50 0x7fe44c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe44bfff640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44c103b10 0x7fe44c077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 -- 172.21.15.78:0/3292393789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe44c104d50 msgr2=0x7fe44c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe44c104d50 0x7fe44c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 -- 172.21.15.78:0/3292393789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44c103b10 msgr2=0x7fe44c077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44c103b10 0x7fe44c077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 -- 172.21.15.78:0/3292393789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c076c70 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe44bfff640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44c103b10 0x7fe44c077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4508f4640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c076540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe444004af0 tx=0x7fe444031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4497fa640 1 -- 172.21.15.78:0/3292393789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444003b70 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4497fa640 1 -- 172.21.15.78:0/3292393789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe444003d10 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c1a42a0 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe4497fa640 1 -- 172.21.15.78:0/3292393789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44402fb20 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.104+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1a4700 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.105+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c06a190 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.107+0000 7fe4497fa640 1 -- 172.21.15.78:0/3292393789 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe444004580 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.107+0000 7fe4497fa640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4340afaf0 0x7fe4340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.107+0000 7fe44bfff640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4340afaf0 0x7fe4340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.107+0000 7fe4497fa640 1 -- 172.21.15.78:0/3292393789 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe4440fff90 con 0x7fe44c109a50 2024-04-25T13:56:32.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.108+0000 7fe44bfff640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4340afaf0 0x7fe4340b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe44c104710 tx=0x7fe43c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.111+0000 7fe4497fa640 1 -- 172.21.15.78:0/3292393789 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4440c9010 con 0x7fe44c109a50 2024-04-25T13:56:32.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.290+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe44c0637a0 con 0x7fe44c109a50 2024-04-25T13:56:32.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.291+0000 7fe4497fa640 1 -- 172.21.15.78:0/3292393789 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18480 (secure 0 0 0) 0x7fe4440c8910 con 0x7fe44c109a50 2024-04-25T13:56:32.291 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:32.291 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:56:22.314977+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4340afaf0 msgr2=0x7fe4340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4340afaf0 0x7fe4340b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe44c104710 tx=0x7fe43c005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 msgr2=0x7fe44c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c076540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe444004af0 tx=0x7fe444031700 comp rx=0 tx=0).stop 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 shutdown_connections 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4340afaf0 0x7fe4340b2000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe44c109a50 0x7fe44c076540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe44c104d50 0x7fe44c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 --2- 172.21.15.78:0/3292393789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe44c103b10 0x7fe44c077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 >> 172.21.15.78:0/3292393789 conn(0x7fe44c0fed40 msgr2=0x7fe44c105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 shutdown_connections 2024-04-25T13:56:32.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.293+0000 7fe452357640 1 -- 172.21.15.78:0/3292393789 wait complete. 2024-04-25T13:56:32.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:56:32.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-04-25T13:56:32.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:32.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:32.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:32.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:32.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:32.304 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:32 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:32.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:56:32.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:56:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 -- 172.21.15.78:0/2491146934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741080d0 msgr2=0x7f177410a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 --2- 172.21.15.78:0/2491146934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741080d0 0x7f177410a4f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f176c009ca0 tx=0x7f176c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 -- 172.21.15.78:0/2491146934 shutdown_connections 2024-04-25T13:56:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 --2- 172.21.15.78:0/2491146934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741080d0 0x7f177410a4f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 --2- 172.21.15.78:0/2491146934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17741033d0 0x7f17741037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 --2- 172.21.15.78:0/2491146934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1774102190 0x7f1774102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 -- 172.21.15.78:0/2491146934 >> 172.21.15.78:0/2491146934 conn(0x7f17740fd380 msgr2=0x7f17740ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 -- 172.21.15.78:0/2491146934 shutdown_connections 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.417+0000 7f177adb2640 1 -- 172.21.15.78:0/2491146934 wait complete. 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 Processor -- start 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 -- start start 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1774102190 0x7f17741990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741033d0 0x7f1774199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17741080d0 0x7f17741a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f177410e530 con 0x7f17741033d0 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f177410e3b0 con 0x7f1774102190 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177adb2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f177410e6b0 con 0x7f17741080d0 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f1773fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741033d0 0x7f1774199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f1773fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741033d0 0x7f1774199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40902/0 (socket says 172.21.15.78:40902) 2024-04-25T13:56:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f1773fff640 1 -- 172.21.15.78:0/1335237904 learned_addr learned my addr 172.21.15.78:0/1335237904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.418+0000 7f177934f640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17741080d0 0x7f17741a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1778b4e640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1774102190 0x7f17741990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1773fff640 1 -- 172.21.15.78:0/1335237904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17741080d0 msgr2=0x7f17741a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1773fff640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17741080d0 0x7f17741a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1773fff640 1 -- 172.21.15.78:0/1335237904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1774102190 msgr2=0x7f17741990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1773fff640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1774102190 0x7f17741990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1773fff640 1 -- 172.21.15.78:0/1335237904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17741a0c10 con 0x7f17741033d0 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1778b4e640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1774102190 0x7f17741990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1773fff640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741033d0 0x7f1774199610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f176000e9f0 tx=0x7f176000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1771ffb640 1 -- 172.21.15.78:0/1335237904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176000cd80 con 0x7f17741033d0 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1771ffb640 1 -- 172.21.15.78:0/1335237904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17600054d0 con 0x7f17741033d0 2024-04-25T13:56:32.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17741a0f00 con 0x7f17741033d0 2024-04-25T13:56:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f1771ffb640 1 -- 172.21.15.78:0/1335237904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760010690 con 0x7f17741033d0 2024-04-25T13:56:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.419+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17741a1370 con 0x7f17741033d0 2024-04-25T13:56:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.420+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177406a160 con 0x7f17741033d0 2024-04-25T13:56:32.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.421+0000 7f1771ffb640 1 -- 172.21.15.78:0/1335237904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1760004e90 con 0x7f17741033d0 2024-04-25T13:56:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.422+0000 7f1771ffb640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17640afaf0 0x7f17640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.422+0000 7f1778b4e640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17640afaf0 0x7f17640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.422+0000 7f1771ffb640 1 -- 172.21.15.78:0/1335237904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f17600dec40 con 0x7f17741033d0 2024-04-25T13:56:32.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.423+0000 7f1778b4e640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17640afaf0 0x7f17640b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1774102d90 tx=0x7f1768005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.427+0000 7f1771ffb640 1 -- 172.21.15.78:0/1335237904 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17600a7cc0 con 0x7f17741033d0 2024-04-25T13:56:32.472 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:56:32.472 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-04-25T13:56:32.472 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:32.472 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:32.472 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:32.472 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:32.473 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:32.473 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:32 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:32.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:32 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.605+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f177402db50 con 0x7f17741033d0 2024-04-25T13:56:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.606+0000 7f1771ffb640 1 -- 172.21.15.78:0/1335237904 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f17600a75c0 con 0x7f17741033d0 2024-04-25T13:56:32.606 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17640afaf0 msgr2=0x7f17640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17640afaf0 0x7f17640b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1774102d90 tx=0x7f1768005e70 comp rx=0 tx=0).stop 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741033d0 msgr2=0x7f1774199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741033d0 0x7f1774199610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f176000e9f0 tx=0x7f176000edf0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 shutdown_connections 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17741080d0 0x7f17741a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17640afaf0 0x7f17640b2000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17741033d0 0x7f1774199610 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 --2- 172.21.15.78:0/1335237904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1774102190 0x7f17741990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 >> 172.21.15.78:0/1335237904 conn(0x7f17740fd380 msgr2=0x7f17741040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 shutdown_connections 2024-04-25T13:56:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.608+0000 7f177adb2640 1 -- 172.21.15.78:0/1335237904 wait complete. 2024-04-25T13:56:32.619 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 -- 172.21.15.78:0/3766819561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50104d50 msgr2=0x7f0e50105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3766819561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50104d50 0x7f0e50105160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009900 tx=0x7f0e4402a410 comp rx=0 tx=0).stop 2024-04-25T13:56:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 -- 172.21.15.78:0/3766819561 shutdown_connections 2024-04-25T13:56:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3766819561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e50109a50 0x7f0e5010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3766819561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50104d50 0x7f0e50105160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3766819561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50103b10 0x7f0e50104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 -- 172.21.15.78:0/3766819561 >> 172.21.15.78:0/3766819561 conn(0x7f0e500fed40 msgr2=0x7f0e501011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 -- 172.21.15.78:0/3766819561 shutdown_connections 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.727+0000 7f0e5636c640 1 -- 172.21.15.78:0/3766819561 wait complete. 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 Processor -- start 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 -- start start 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e50103b10 0x7f0e5019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50104d50 0x7f0e5019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50109a50 0x7f0e501a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010f690 con 0x7f0e50109a50 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010f510 con 0x7f0e50104d50 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e5636c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5010f810 con 0x7f0e50103b10 2024-04-25T13:56:32.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e54909640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50109a50 0x7f0e501a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e54909640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50109a50 0x7f0e501a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40912/0 (socket says 172.21.15.78:40912) 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e54909640 1 -- 172.21.15.78:0/3663346799 learned_addr learned my addr 172.21.15.78:0/3663346799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.728+0000 7f0e4ffff640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e50103b10 0x7f0e5019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e4f7fe640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50104d50 0x7f0e5019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e54909640 1 -- 172.21.15.78:0/3663346799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e50103b10 msgr2=0x7f0e5019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e54909640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e50103b10 0x7f0e5019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e54909640 1 -- 172.21.15.78:0/3663346799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50104d50 msgr2=0x7f0e5019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e54909640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50104d50 0x7f0e5019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e54909640 1 -- 172.21.15.78:0/3663346799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e501a1e30 con 0x7f0e50109a50 2024-04-25T13:56:32.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e54909640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50109a50 0x7f0e501a17c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009fd0 tx=0x7f0e4800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e4d7fa640 1 -- 172.21.15.78:0/3663346799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48017070 con 0x7f0e50109a50 2024-04-25T13:56:32.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e4d7fa640 1 -- 172.21.15.78:0/3663346799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e48003e20 con 0x7f0e50109a50 2024-04-25T13:56:32.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e501a2120 con 0x7f0e50109a50 2024-04-25T13:56:32.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e4d7fa640 1 -- 172.21.15.78:0/3663346799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48010460 con 0x7f0e50109a50 2024-04-25T13:56:32.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.729+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e501a2620 con 0x7f0e50109a50 2024-04-25T13:56:32.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.730+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5006a190 con 0x7f0e50109a50 2024-04-25T13:56:32.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.731+0000 7f0e4d7fa640 1 -- 172.21.15.78:0/3663346799 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0e48004020 con 0x7f0e50109a50 2024-04-25T13:56:32.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.732+0000 7f0e4d7fa640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0e380afaf0 0x7f0e380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:32.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.732+0000 7f0e4ffff640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0e380afaf0 0x7f0e380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:32.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.732+0000 7f0e4d7fa640 1 -- 172.21.15.78:0/3663346799 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0e480de4c0 con 0x7f0e50109a50 2024-04-25T13:56:32.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.732+0000 7f0e4ffff640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0e380afaf0 0x7f0e380b2000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e50104710 tx=0x7f0e40009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:32.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.737+0000 7f0e4d7fa640 1 -- 172.21.15.78:0/3663346799 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e480a7540 con 0x7f0e50109a50 2024-04-25T13:56:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.951+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e50108860 con 0x7f0e50109a50 2024-04-25T13:56:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.951+0000 7f0e4d7fa640 1 -- 172.21.15.78:0/3663346799 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f0e480a6e40 con 0x7f0e50109a50 2024-04-25T13:56:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:32.951 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T13:56:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 1 2024-04-25T13:56:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0e380afaf0 msgr2=0x7f0e380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0e380afaf0 0x7f0e380b2000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e50104710 tx=0x7f0e40009110 comp rx=0 tx=0).stop 2024-04-25T13:56:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50109a50 msgr2=0x7f0e501a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50109a50 0x7f0e501a17c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009fd0 tx=0x7f0e4800def0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 shutdown_connections 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0e380afaf0 0x7f0e380b2000 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e50109a50 0x7f0e501a17c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e50104d50 0x7f0e5019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 --2- 172.21.15.78:0/3663346799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e50103b10 0x7f0e5019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 >> 172.21.15.78:0/3663346799 conn(0x7f0e500fed40 msgr2=0x7f0e50105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 shutdown_connections 2024-04-25T13:56:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:32.953+0000 7f0e5636c640 1 -- 172.21.15.78:0/3663346799 wait complete. 2024-04-25T13:56:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:33 smithi078 ceph-mon[29163]: Deploying daemon mds.l on smithi129 2024-04-25T13:56:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:33 smithi078 ceph-mon[29163]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:33 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3292393789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:56:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:33 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1335237904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:56:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:33 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3663346799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:33 smithi112 ceph-mon[32617]: Deploying daemon mds.l on smithi129 2024-04-25T13:56:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:33 smithi112 ceph-mon[32617]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:33 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3292393789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:56:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:33 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1335237904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:56:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:33 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3663346799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:33.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:33 smithi129 ceph-mon[32569]: Deploying daemon mds.l on smithi129 2024-04-25T13:56:33.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:33 smithi129 ceph-mon[32569]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:33.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:33 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3292393789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:56:33.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:33 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1335237904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:56:33.526 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:33 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3663346799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:33.913 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:33 smithi129 systemd[1]: Starting Ceph mds.l for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:34.215 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:33 smithi129 podman[62057]: 2024-04-25 13:56:33.827882538 +0000 UTC m=+0.018303935 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:34.216 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:33 smithi129 podman[62057]: 2024-04-25 13:56:33.960129296 +0000 UTC m=+0.150550693 container create 659f89faefcbf33ba6ab8a55ff4f014048be7a28c88f09147bdbda17824bc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:56:34.216 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:34 smithi129 podman[62057]: 2024-04-25 13:56:34.216642275 +0000 UTC m=+0.407063673 container init 659f89faefcbf33ba6ab8a55ff4f014048be7a28c88f09147bdbda17824bc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:56:34.663 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:34 smithi129 podman[62057]: 2024-04-25 13:56:34.21945954 +0000 UTC m=+0.409880945 container start 659f89faefcbf33ba6ab8a55ff4f014048be7a28c88f09147bdbda17824bc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T13:56:34.664 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:34 smithi129 ceph-mds[62076]: -- 172.21.15.129:0/3985462475 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a54d47a40 con 0x564a54d8fa80 2024-04-25T13:56:34.664 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:34 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l[62072]: starting mds.l at 2024-04-25T13:56:34.664 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:34 smithi129 bash[62057]: 659f89faefcbf33ba6ab8a55ff4f014048be7a28c88f09147bdbda17824bc24c 2024-04-25T13:56:34.664 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 13:56:34 smithi129 systemd[1]: Started Ceph mds.l for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:34.966 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 -- 172.21.15.78:0/4109187816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 msgr2=0x7f29d0104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 --2- 172.21.15.78:0/4109187816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d0104010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009900 tx=0x7f29c002a410 comp rx=0 tx=0).stop 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 -- 172.21.15.78:0/4109187816 shutdown_connections 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 --2- 172.21.15.78:0/4109187816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29d0108900 0x7f29d010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 --2- 172.21.15.78:0/4109187816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d0104010 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 --2- 172.21.15.78:0/4109187816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29d01029c0 0x7f29d01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 -- 172.21.15.78:0/4109187816 >> 172.21.15.78:0/4109187816 conn(0x7f29d00fdbd0 msgr2=0x7f29d0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 -- 172.21.15.78:0/4109187816 shutdown_connections 2024-04-25T13:56:35.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.086+0000 7f29d8508640 1 -- 172.21.15.78:0/4109187816 wait complete. 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 Processor -- start 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 -- start start 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29d01029c0 0x7f29d019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29d0108900 0x7f29d01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29d010f650 con 0x7f29d0103c00 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29d010f4d0 con 0x7f29d0108900 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d8508640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29d010f7d0 con 0x7f29d01029c0 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40928/0 (socket says 172.21.15.78:40928) 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 -- 172.21.15.78:0/3010275903 learned_addr learned my addr 172.21.15.78:0/3010275903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d62a4640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29d01029c0 0x7f29d019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d6aa5640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29d0108900 0x7f29d01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 -- 172.21.15.78:0/3010275903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29d01029c0 msgr2=0x7f29d019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29d01029c0 0x7f29d019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 -- 172.21.15.78:0/3010275903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29d0108900 msgr2=0x7f29d01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29d0108900 0x7f29d01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.087+0000 7f29d5aa3640 1 -- 172.21.15.78:0/3010275903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d01a1db0 con 0x7f29d0103c00 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.088+0000 7f29d5aa3640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d019a7b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.088+0000 7f29c77fe640 1 -- 172.21.15.78:0/3010275903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c003f030 con 0x7f29d0103c00 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.088+0000 7f29c77fe640 1 -- 172.21.15.78:0/3010275903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c0003cb0 con 0x7f29d0103c00 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.088+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d01a2040 con 0x7f29d0103c00 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.088+0000 7f29c77fe640 1 -- 172.21.15.78:0/3010275903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0035070 con 0x7f29d0103c00 2024-04-25T13:56:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.088+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01a2600 con 0x7f29d0103c00 2024-04-25T13:56:35.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.089+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d0107710 con 0x7f29d0103c00 2024-04-25T13:56:35.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.090+0000 7f29c77fe640 1 -- 172.21.15.78:0/3010275903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f29c0042030 con 0x7f29d0103c00 2024-04-25T13:56:35.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.091+0000 7f29c77fe640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29bc0afb70 0x7f29bc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:35.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.091+0000 7f29d62a4640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29bc0afb70 0x7f29bc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:35.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.091+0000 7f29c77fe640 1 -- 172.21.15.78:0/3010275903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f29c00ff430 con 0x7f29d0103c00 2024-04-25T13:56:35.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.091+0000 7f29d62a4640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29bc0afb70 0x7f29bc0b2080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f29d01035c0 tx=0x7f29cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:35.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.095+0000 7f29c77fe640 1 -- 172.21.15.78:0/3010275903 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29c00c7e60 con 0x7f29d0103c00 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6832/1065513259,v1:172.21.15.129:6833/1065513259] up:boot 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: fsmap 1 up:standby 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:35.307 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:35.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.314+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29d002db50 con 0x7f29d0103c00 2024-04-25T13:56:35.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.314+0000 7f29c77fe640 1 -- 172.21.15.78:0/3010275903 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f29c00cc160 con 0x7f29d0103c00 2024-04-25T13:56:35.314 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:35.314 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:56:35.314 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 2 2024-04-25T13:56:35.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29bc0afb70 msgr2=0x7f29bc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29bc0afb70 0x7f29bc0b2080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f29d01035c0 tx=0x7f29cc00a2d0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 msgr2=0x7f29d019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:35.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d019a7b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c00043b0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 shutdown_connections 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29d0108900 0x7f29d01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29bc0afb70 0x7f29bc0b2080 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29d0103c00 0x7f29d019a7b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 --2- 172.21.15.78:0/3010275903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29d01029c0 0x7f29d019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 >> 172.21.15.78:0/3010275903 conn(0x7f29d00fdbd0 msgr2=0x7f29d01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 shutdown_connections 2024-04-25T13:56:35.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:35.316+0000 7f29d8508640 1 -- 172.21.15.78:0/3010275903 wait complete. 2024-04-25T13:56:35.326 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:35.326 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6832/1065513259,v1:172.21.15.129:6833/1065513259] up:boot 2024-04-25T13:56:35.326 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: fsmap 1 up:standby 2024-04-25T13:56:35.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:56:35.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:35.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:35.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:35.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6832/1065513259,v1:172.21.15.129:6833/1065513259] up:boot 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: fsmap 1 up:standby 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:35.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:36 smithi078 ceph-mon[29163]: Deploying daemon mds.k on smithi112 2024-04-25T13:56:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:36 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3010275903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:36.499 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:36 smithi112 ceph-mon[32617]: Deploying daemon mds.k on smithi112 2024-04-25T13:56:36.500 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:36 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3010275903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:36 smithi129 ceph-mon[32569]: Deploying daemon mds.k on smithi112 2024-04-25T13:56:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:36 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3010275903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:36.766 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:36 smithi112 systemd[1]: Starting Ceph mds.k for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:37.183 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:36 smithi112 podman[62651]: 2024-04-25 13:56:36.798917409 +0000 UTC m=+0.017961425 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:37.183 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:36 smithi112 podman[62651]: 2024-04-25 13:56:36.916244144 +0000 UTC m=+0.135288155 container create e2927c9c209bc072541bef21aa11a705f873d868ebce8b0dad3bdb2271ac4dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T13:56:37.183 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:37 smithi112 podman[62651]: 2024-04-25 13:56:37.102264812 +0000 UTC m=+0.321308823 container init e2927c9c209bc072541bef21aa11a705f873d868ebce8b0dad3bdb2271ac4dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T13:56:37.183 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:37 smithi112 podman[62651]: 2024-04-25 13:56:37.104907123 +0000 UTC m=+0.323951136 container start e2927c9c209bc072541bef21aa11a705f873d868ebce8b0dad3bdb2271ac4dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T13:56:37.183 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:37 smithi112 ceph-mds[62670]: -- 172.21.15.112:0/3867986405 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608111b9a40 con 0x560811201a80 2024-04-25T13:56:37.183 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:37 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k[62666]: starting mds.k at 2024-04-25T13:56:37.329 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 -- 172.21.15.78:0/3907029998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1044d0 msgr2=0x7f113c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 --2- 172.21.15.78:0/3907029998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1044d0 0x7f113c10ac20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1134009ca0 tx=0x7f113402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 -- 172.21.15.78:0/3907029998 shutdown_connections 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 --2- 172.21.15.78:0/3907029998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1044d0 0x7f113c10ac20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 --2- 172.21.15.78:0/3907029998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f113c1039b0 0x7f113c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 --2- 172.21.15.78:0/3907029998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c102fd0 0x7f113c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 -- 172.21.15.78:0/3907029998 >> 172.21.15.78:0/3907029998 conn(0x7f113c0fd3b0 msgr2=0x7f113c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.455+0000 7f1140842640 1 -- 172.21.15.78:0/3907029998 shutdown_connections 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.456+0000 7f1140842640 1 -- 172.21.15.78:0/3907029998 wait complete. 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.456+0000 7f1140842640 1 Processor -- start 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.456+0000 7f1140842640 1 -- start start 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f113c102fd0 0x7f113c102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1039b0 0x7f113c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c1044d0 0x7f113c1a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f113c10e530 con 0x7f113c1039b0 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f113c10e3b0 con 0x7f113c102fd0 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f113c10e6b0 con 0x7f113c1044d0 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1039b0 0x7f113c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1039b0 0x7f113c19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35080/0 (socket says 172.21.15.78:35080) 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 -- 172.21.15.78:0/4194549644 learned_addr learned my addr 172.21.15.78:0/4194549644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f113ad9d640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c1044d0 0x7f113c1a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f113a59c640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f113c102fd0 0x7f113c102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 -- 172.21.15.78:0/4194549644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c1044d0 msgr2=0x7f113c1a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c1044d0 0x7f113c1a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 -- 172.21.15.78:0/4194549644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f113c102fd0 msgr2=0x7f113c102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f113c102fd0 0x7f113c102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 -- 172.21.15.78:0/4194549644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c1a1c90 con 0x7f113c1039b0 2024-04-25T13:56:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1139d9b640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1039b0 0x7f113c19ef20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f112c00d950 tx=0x7f112c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f112b7fe640 1 -- 172.21.15.78:0/4194549644 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c017070 con 0x7f113c1039b0 2024-04-25T13:56:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f112b7fe640 1 -- 172.21.15.78:0/4194549644 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f112c009c30 con 0x7f113c1039b0 2024-04-25T13:56:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c1a1f80 con 0x7f113c1039b0 2024-04-25T13:56:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f112b7fe640 1 -- 172.21.15.78:0/4194549644 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c00f620 con 0x7f113c1039b0 2024-04-25T13:56:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.457+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c1a23f0 con 0x7f113c1039b0 2024-04-25T13:56:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.458+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113c109980 con 0x7f113c1039b0 2024-04-25T13:56:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.461+0000 7f112b7fe640 1 -- 172.21.15.78:0/4194549644 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f112c010460 con 0x7f113c1039b0 2024-04-25T13:56:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.461+0000 7f112b7fe640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11240afaf0 0x7f11240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.461+0000 7f113a59c640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11240afaf0 0x7f11240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.462+0000 7f112b7fe640 1 -- 172.21.15.78:0/4194549644 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f112c0de360 con 0x7f113c1039b0 2024-04-25T13:56:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.462+0000 7f113a59c640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11240afaf0 0x7f11240b2000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1130005fd0 tx=0x7f1130005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.465+0000 7f112b7fe640 1 -- 172.21.15.78:0/4194549644 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f112c0a73e0 con 0x7f113c1039b0 2024-04-25T13:56:37.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:37 smithi078 ceph-mon[29163]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:37 smithi112 ceph-mon[32617]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:37.516 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:37 smithi112 bash[62651]: e2927c9c209bc072541bef21aa11a705f873d868ebce8b0dad3bdb2271ac4dc0 2024-04-25T13:56:37.516 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 13:56:37 smithi112 systemd[1]: Started Ceph mds.k for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:37.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:37 smithi129 ceph-mon[32569]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.688+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f113c02db50 con 0x7f113c1039b0 2024-04-25T13:56:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.689+0000 7f112b7fe640 1 -- 172.21.15.78:0/4194549644 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f112c009de0 con 0x7f113c1039b0 2024-04-25T13:56:37.689 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:37.689 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:56:37.689 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11240afaf0 msgr2=0x7f11240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11240afaf0 0x7f11240b2000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1130005fd0 tx=0x7f1130005850 comp rx=0 tx=0).stop 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1039b0 msgr2=0x7f113c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1039b0 0x7f113c19ef20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f112c00d950 tx=0x7f112c00de10 comp rx=0 tx=0).stop 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 shutdown_connections 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f113c1044d0 0x7f113c1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11240afaf0 0x7f11240b2000 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f113c1039b0 0x7f113c19ef20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 --2- 172.21.15.78:0/4194549644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f113c102fd0 0x7f113c102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:37.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 >> 172.21.15.78:0/4194549644 conn(0x7f113c0fd3b0 msgr2=0x7f113c106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:37.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 shutdown_connections 2024-04-25T13:56:37.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:37.691+0000 7f1140842640 1 -- 172.21.15.78:0/4194549644 wait complete. 2024-04-25T13:56:38.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6832/3808802757,v1:172.21.15.112:6833/3808802757] up:boot 2024-04-25T13:56:38.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: fsmap 2 up:standby 2024-04-25T13:56:38.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:56:38.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:38.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:38.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:38.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:38 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4194549644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6832/3808802757,v1:172.21.15.112:6833/3808802757] up:boot 2024-04-25T13:56:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: fsmap 2 up:standby 2024-04-25T13:56:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:56:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:38.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:38.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:38.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:38 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4194549644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6832/3808802757,v1:172.21.15.112:6833/3808802757] up:boot 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: fsmap 2 up:standby 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:38 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4194549644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:39 smithi078 ceph-mon[29163]: Deploying daemon mds.j on smithi078 2024-04-25T13:56:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:39 smithi078 ceph-mon[29163]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:39.454 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:39 smithi112 ceph-mon[32617]: Deploying daemon mds.j on smithi078 2024-04-25T13:56:39.454 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:39 smithi112 ceph-mon[32617]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:39.565 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:39 smithi129 ceph-mon[32569]: Deploying daemon mds.j on smithi078 2024-04-25T13:56:39.565 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:39 smithi129 ceph-mon[32569]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:39.703 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:39.719 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:39 smithi078 systemd[1]: Starting Ceph mds.j for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 -- 172.21.15.78:0/3924793538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4410103c20 msgr2=0x7f4410104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 --2- 172.21.15.78:0/3924793538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4410103c20 0x7f4410104030 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4404009900 tx=0x7f440402a410 comp rx=0 tx=0).stop 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 -- 172.21.15.78:0/3924793538 shutdown_connections 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 --2- 172.21.15.78:0/3924793538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4410108920 0x7f441010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 --2- 172.21.15.78:0/3924793538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4410103c20 0x7f4410104030 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 --2- 172.21.15.78:0/3924793538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44101029e0 0x7f44101036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 -- 172.21.15.78:0/3924793538 >> 172.21.15.78:0/3924793538 conn(0x7f44100fdbd0 msgr2=0x7f4410100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 -- 172.21.15.78:0/3924793538 shutdown_connections 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 -- 172.21.15.78:0/3924793538 wait complete. 2024-04-25T13:56:39.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 Processor -- start 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.817+0000 7f4416634640 1 -- start start 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44101029e0 0x7f441019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4410103c20 0x7f441019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4410108920 0x7f44101a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f441010f670 con 0x7f44101029e0 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f441010f4f0 con 0x7f4410103c20 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f441010f7f0 con 0x7f4410108920 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44101029e0 0x7f441019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44101029e0 0x7f441019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35092/0 (socket says 172.21.15.78:35092) 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 -- 172.21.15.78:0/2424762017 learned_addr learned my addr 172.21.15.78:0/2424762017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440f7fe640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4410103c20 0x7f441019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 -- 172.21.15.78:0/2424762017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4410108920 msgr2=0x7f44101a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4410108920 0x7f44101a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:39.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 -- 172.21.15.78:0/2424762017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4410103c20 msgr2=0x7f441019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4410103c20 0x7f441019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 -- 172.21.15.78:0/2424762017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44101a1df0 con 0x7f44101029e0 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440f7fe640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4410103c20 0x7f441019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440ffff640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44101029e0 0x7f441019a2b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f440000d740 tx=0x7f440000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440d7fa640 1 -- 172.21.15.78:0/2424762017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400017070 con 0x7f44101029e0 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44101a20e0 con 0x7f44101029e0 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440d7fa640 1 -- 172.21.15.78:0/2424762017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440000de50 con 0x7f44101029e0 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f440d7fa640 1 -- 172.21.15.78:0/2424762017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400003c30 con 0x7f44101029e0 2024-04-25T13:56:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.818+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44101a2700 con 0x7f44101029e0 2024-04-25T13:56:39.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.820+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4410107730 con 0x7f44101029e0 2024-04-25T13:56:39.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.821+0000 7f440d7fa640 1 -- 172.21.15.78:0/2424762017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4400003dd0 con 0x7f44101029e0 2024-04-25T13:56:39.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.821+0000 7f440d7fa640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43f80afaf0 0x7f43f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:39.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.821+0000 7f440f7fe640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43f80afaf0 0x7f43f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:39.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.822+0000 7f440d7fa640 1 -- 172.21.15.78:0/2424762017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f44000de240 con 0x7f44101029e0 2024-04-25T13:56:39.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.822+0000 7f440f7fe640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43f80afaf0 0x7f43f80b2000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4404005f70 tx=0x7f4404031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:39.826+0000 7f440d7fa640 1 -- 172.21.15.78:0/2424762017 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44000a72c0 con 0x7f44101029e0 2024-04-25T13:56:40.036 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:39 smithi078 podman[68794]: 2024-04-25 13:56:39.672048367 +0000 UTC m=+0.017712720 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:40.036 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:39 smithi078 podman[68794]: 2024-04-25 13:56:39.792687993 +0000 UTC m=+0.138352346 container create d98f796c6f6019ec2a3e0c0bef51fed334498c2c0f10651e9e55e19e47485585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:56:40.036 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:40 smithi078 podman[68794]: 2024-04-25 13:56:40.037220662 +0000 UTC m=+0.382885015 container init d98f796c6f6019ec2a3e0c0bef51fed334498c2c0f10651e9e55e19e47485585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T13:56:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.047+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f441002db50 con 0x7f44101029e0 2024-04-25T13:56:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.047+0000 7f440d7fa640 1 -- 172.21.15.78:0/2424762017 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f44000a6bc0 con 0x7f44101029e0 2024-04-25T13:56:40.047 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:40.047 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:56:40.047 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2024-04-25T13:56:40.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43f80afaf0 msgr2=0x7f43f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:40.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43f80afaf0 0x7f43f80b2000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4404005f70 tx=0x7f4404031400 comp rx=0 tx=0).stop 2024-04-25T13:56:40.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44101029e0 msgr2=0x7f441019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44101029e0 0x7f441019a2b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f440000d740 tx=0x7f440000dc00 comp rx=0 tx=0).stop 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 shutdown_connections 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4410108920 0x7f44101a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4410103c20 0x7f441019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43f80afaf0 0x7f43f80b2000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 --2- 172.21.15.78:0/2424762017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44101029e0 0x7f441019a2b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 >> 172.21.15.78:0/2424762017 conn(0x7f44100fdbd0 msgr2=0x7f4410104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 shutdown_connections 2024-04-25T13:56:40.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:40.050+0000 7f4416634640 1 -- 172.21.15.78:0/2424762017 wait complete. 2024-04-25T13:56:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:40 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2424762017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:40.469 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:40 smithi078 podman[68794]: 2024-04-25 13:56:40.040587664 +0000 UTC m=+0.386252029 container start d98f796c6f6019ec2a3e0c0bef51fed334498c2c0f10651e9e55e19e47485585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T13:56:40.469 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:40 smithi078 ceph-mds[68852]: -- 172.21.15.78:0/2879834351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d2cd53a40 con 0x555d2cd9b600 2024-04-25T13:56:40.469 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j[68848]: starting mds.j at 2024-04-25T13:56:40.470 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:40 smithi078 bash[68794]: d98f796c6f6019ec2a3e0c0bef51fed334498c2c0f10651e9e55e19e47485585 2024-04-25T13:56:40.470 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 13:56:40 smithi078 systemd[1]: Started Ceph mds.j for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:40 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2424762017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:40.575 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:40 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2424762017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.78:6834/835077889,v1:172.21.15.78:6835/835077889] up:boot 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: fsmap 3 up:standby 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:41 smithi078 ceph-mon[29163]: Deploying daemon mds.d on smithi078 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.78:6834/835077889,v1:172.21.15.78:6835/835077889] up:boot 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: fsmap 3 up:standby 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:41.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:41 smithi112 ceph-mon[32617]: Deploying daemon mds.d on smithi078 2024-04-25T13:56:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.78:6834/835077889,v1:172.21.15.78:6835/835077889] up:boot 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: fsmap 3 up:standby 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:41 smithi129 ceph-mon[32569]: Deploying daemon mds.d on smithi078 2024-04-25T13:56:42.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:42.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 -- 172.21.15.78:0/1033259858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0108900 msgr2=0x7fb3b010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/1033259858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0108900 0x7fb3b010ad20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009ca0 tx=0x7fb3a802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 -- 172.21.15.78:0/1033259858 shutdown_connections 2024-04-25T13:56:42.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/1033259858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0108900 0x7fb3b010ad20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/1033259858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3b0103c00 0x7fb3b0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/1033259858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 0x7fb3b01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 -- 172.21.15.78:0/1033259858 >> 172.21.15.78:0/1033259858 conn(0x7fb3b00fdbd0 msgr2=0x7fb3b0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 -- 172.21.15.78:0/1033259858 shutdown_connections 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.204+0000 7fb3b6f84640 1 -- 172.21.15.78:0/1033259858 wait complete. 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 Processor -- start 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 -- start start 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 0x7fb3b006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0103c00 0x7fb3b019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3b0108900 0x7fb3b01a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010e4f0 con 0x7fb3b01029c0 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010e370 con 0x7fb3b0103c00 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.205+0000 7fb3b6f84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010e670 con 0x7fb3b0108900 2024-04-25T13:56:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 0x7fb3b006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 0x7fb3b006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35118/0 (socket says 172.21.15.78:35118) 2024-04-25T13:56:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 -- 172.21.15.78:0/819592178 learned_addr learned my addr 172.21.15.78:0/819592178 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3affff640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0103c00 0x7fb3b019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b5521640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3b0108900 0x7fb3b01a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 -- 172.21.15.78:0/819592178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3b0108900 msgr2=0x7fb3b01a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3b0108900 0x7fb3b01a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 -- 172.21.15.78:0/819592178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0103c00 msgr2=0x7fb3b019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0103c00 0x7fb3b019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b4d20640 1 -- 172.21.15.78:0/819592178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b0198da0 con 0x7fb3b01029c0 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.206+0000 7fb3b5521640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3b0108900 0x7fb3b01a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.207+0000 7fb3b4d20640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 0x7fb3b006a920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e980 tx=0x7fb3a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.218+0000 7fb3adffb640 1 -- 172.21.15.78:0/819592178 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a400cd80 con 0x7fb3b01029c0 2024-04-25T13:56:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.218+0000 7fb3adffb640 1 -- 172.21.15.78:0/819592178 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a40108e0 con 0x7fb3b01029c0 2024-04-25T13:56:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.218+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b0199090 con 0x7fb3b01029c0 2024-04-25T13:56:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.218+0000 7fb3adffb640 1 -- 172.21.15.78:0/819592178 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4018990 con 0x7fb3b01029c0 2024-04-25T13:56:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.218+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01995e0 con 0x7fb3b01029c0 2024-04-25T13:56:42.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.220+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3940052c0 con 0x7fb3b01029c0 2024-04-25T13:56:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.222+0000 7fb3adffb640 1 -- 172.21.15.78:0/819592178 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb3a401f020 con 0x7fb3b01029c0 2024-04-25T13:56:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.222+0000 7fb3adffb640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb3a00afc40 0x7fb3a00b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.222+0000 7fb3affff640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb3a00afc40 0x7fb3a00b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.222+0000 7fb3adffb640 1 -- 172.21.15.78:0/819592178 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb3a40dec40 con 0x7fb3b01029c0 2024-04-25T13:56:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.223+0000 7fb3affff640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb3a00afc40 0x7fb3a00b2150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3b006a2c0 tx=0x7fb39c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:42.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.226+0000 7fb3adffb640 1 -- 172.21.15.78:0/819592178 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3a4026020 con 0x7fb3b01029c0 2024-04-25T13:56:42.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:42 smithi078 ceph-mon[29163]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.457+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3940055b0 con 0x7fb3b01029c0 2024-04-25T13:56:42.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.458+0000 7fb3adffb640 1 -- 172.21.15.78:0/819592178 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fb3a40a7d40 con 0x7fb3b01029c0 2024-04-25T13:56:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:42.458 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:56:42.458 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 4 2024-04-25T13:56:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb3a00afc40 msgr2=0x7fb3a00b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb3a00afc40 0x7fb3a00b2150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3b006a2c0 tx=0x7fb39c005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 msgr2=0x7fb3b006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 0x7fb3b006a920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e980 tx=0x7fb3a400ee40 comp rx=0 tx=0).stop 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 shutdown_connections 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3b0108900 0x7fb3b01a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3b0103c00 0x7fb3b019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb3a00afc40 0x7fb3a00b2150 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 --2- 172.21.15.78:0/819592178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b01029c0 0x7fb3b006a920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 >> 172.21.15.78:0/819592178 conn(0x7fb3b00fdbd0 msgr2=0x7fb3b01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 shutdown_connections 2024-04-25T13:56:42.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:42.460+0000 7fb3b6f84640 1 -- 172.21.15.78:0/819592178 wait complete. 2024-04-25T13:56:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:42 smithi112 ceph-mon[32617]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:42.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:42 smithi129 ceph-mon[32569]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:42.719 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:42 smithi078 systemd[1]: Starting Ceph mds.d for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:43.126 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:42 smithi078 podman[69187]: 2024-04-25 13:56:42.678686937 +0000 UTC m=+0.018200445 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:43.126 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:42 smithi078 podman[69187]: 2024-04-25 13:56:42.807759891 +0000 UTC m=+0.147273394 container create 6a0ff26c9602e2963aafbd70ded55ae1ed55ba1fcf4dd684c4949c4f07799a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:56:43.126 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:43 smithi078 podman[69187]: 2024-04-25 13:56:43.127272355 +0000 UTC m=+0.466785858 container init 6a0ff26c9602e2963aafbd70ded55ae1ed55ba1fcf4dd684c4949c4f07799a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T13:56:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:43 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/819592178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:43.469 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:43 smithi078 podman[69187]: 2024-04-25 13:56:43.13000837 +0000 UTC m=+0.469521873 container start 6a0ff26c9602e2963aafbd70ded55ae1ed55ba1fcf4dd684c4949c4f07799a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:56:43.469 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:43 smithi078 ceph-mds[69207]: -- 172.21.15.78:0/833786837 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560267e61a40 con 0x560267ea9600 2024-04-25T13:56:43.469 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:43 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d[69203]: starting mds.d at 2024-04-25T13:56:43.469 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:43 smithi078 bash[69187]: 6a0ff26c9602e2963aafbd70ded55ae1ed55ba1fcf4dd684c4949c4f07799a34 2024-04-25T13:56:43.469 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 13:56:43 smithi078 systemd[1]: Started Ceph mds.d for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:43 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/819592178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:43 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/819592178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.78:6836/3031062077,v1:172.21.15.78:6837/3031062077] up:boot 2024-04-25T13:56:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: fsmap 4 up:standby 2024-04-25T13:56:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:56:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: Deploying daemon mds.e on smithi112 2024-04-25T13:56:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:44 smithi078 ceph-mon[29163]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:44.472 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.78:6836/3031062077,v1:172.21.15.78:6837/3031062077] up:boot 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: fsmap 4 up:standby 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:44.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: Deploying daemon mds.e on smithi112 2024-04-25T13:56:44.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:44 smithi112 ceph-mon[32617]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 -- 172.21.15.78:0/590422414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11201097a0 msgr2=0x7f112010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 --2- 172.21.15.78:0/590422414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11201097a0 0x7f112010bcb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1118009ca0 tx=0x7f111802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 -- 172.21.15.78:0/590422414 shutdown_connections 2024-04-25T13:56:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 --2- 172.21.15.78:0/590422414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11201097a0 0x7f112010bcb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 --2- 172.21.15.78:0/590422414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1120106c30 0x7f1120109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 --2- 172.21.15.78:0/590422414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1120101ea0 0x7f11201022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 -- 172.21.15.78:0/590422414 >> 172.21.15.78:0/590422414 conn(0x7f11200fd3b0 msgr2=0x7f11200ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 -- 172.21.15.78:0/590422414 shutdown_connections 2024-04-25T13:56:44.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.600+0000 7f1127ef0640 1 -- 172.21.15.78:0/590422414 wait complete. 2024-04-25T13:56:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.601+0000 7f1127ef0640 1 Processor -- start 2024-04-25T13:56:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.601+0000 7f1127ef0640 1 -- start start 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1127ef0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1120101ea0 0x7f112019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1127ef0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1120106c30 0x7f112019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1127ef0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11201097a0 0x7f11201a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1127ef0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f112010f610 con 0x7f1120106c30 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1127ef0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f112010f490 con 0x7f11201097a0 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1127ef0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f112010f790 con 0x7f1120101ea0 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1120106c30 0x7f112019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1120106c30 0x7f112019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35146/0 (socket says 172.21.15.78:35146) 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 -- 172.21.15.78:0/3304483660 learned_addr learned my addr 172.21.15.78:0/3304483660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112648d640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11201097a0 0x7f11201a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1125c8c640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1120101ea0 0x7f112019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 -- 172.21.15.78:0/3304483660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1120101ea0 msgr2=0x7f112019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1120101ea0 0x7f112019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 -- 172.21.15.78:0/3304483660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11201097a0 msgr2=0x7f11201a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11201097a0 0x7f11201a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 -- 172.21.15.78:0/3304483660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11201a1bf0 con 0x7f1120106c30 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1125c8c640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1120101ea0 0x7f112019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f112548b640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1120106c30 0x7f112019a6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f111c00d950 tx=0x7f111c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1116ffd640 1 -- 172.21.15.78:0/3304483660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c017070 con 0x7f1120106c30 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1116ffd640 1 -- 172.21.15.78:0/3304483660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f111c009c30 con 0x7f1120106c30 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11201a1ee0 con 0x7f1120106c30 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.602+0000 7f1116ffd640 1 -- 172.21.15.78:0/3304483660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c00f620 con 0x7f1120106c30 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.603+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11201a2350 con 0x7f1120106c30 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.603+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11200feed0 con 0x7f1120106c30 2024-04-25T13:56:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.605+0000 7f1116ffd640 1 -- 172.21.15.78:0/3304483660 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f111c010460 con 0x7f1120106c30 2024-04-25T13:56:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.605+0000 7f1116ffd640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f110c0afaf0 0x7f110c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.606+0000 7f1125c8c640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f110c0afaf0 0x7f110c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.606+0000 7f1116ffd640 1 -- 172.21.15.78:0/3304483660 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f111c0de360 con 0x7f1120106c30 2024-04-25T13:56:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.606+0000 7f1125c8c640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f110c0afaf0 0x7f110c0b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f112006b5b0 tx=0x7f1110005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.609+0000 7f1116ffd640 1 -- 172.21.15.78:0/3304483660 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f111c0a73e0 con 0x7f1120106c30 2024-04-25T13:56:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.78:6836/3031062077,v1:172.21.15.78:6837/3031062077] up:boot 2024-04-25T13:56:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: fsmap 4 up:standby 2024-04-25T13:56:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: Deploying daemon mds.e on smithi112 2024-04-25T13:56:44.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:44 smithi129 ceph-mon[32569]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.826+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f112002db50 con 0x7f1120106c30 2024-04-25T13:56:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.827+0000 7f1116ffd640 1 -- 172.21.15.78:0/3304483660 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f111c009de0 con 0x7f1120106c30 2024-04-25T13:56:44.826 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:44.826 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:56:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 5 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f110c0afaf0 msgr2=0x7f110c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f110c0afaf0 0x7f110c0b2000 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f112006b5b0 tx=0x7f1110005db0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1120106c30 msgr2=0x7f112019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1120106c30 0x7f112019a6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f111c00d950 tx=0x7f111c00de10 comp rx=0 tx=0).stop 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 shutdown_connections 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11201097a0 0x7f11201a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f110c0afaf0 0x7f110c0b2000 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1120106c30 0x7f112019a6a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 --2- 172.21.15.78:0/3304483660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1120101ea0 0x7f112019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:44.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 >> 172.21.15.78:0/3304483660 conn(0x7f11200fd3b0 msgr2=0x7f1120107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:44.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 shutdown_connections 2024-04-25T13:56:44.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:44.829+0000 7f1127ef0640 1 -- 172.21.15.78:0/3304483660 wait complete. 2024-04-25T13:56:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:45 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3304483660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:45.517 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:45 smithi112 systemd[1]: Starting Ceph mds.e for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:45.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:45 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3304483660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:45 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3304483660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:45.945 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:45 smithi112 podman[62974]: 2024-04-25 13:56:45.568492904 +0000 UTC m=+0.018116990 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:45.945 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:45 smithi112 podman[62974]: 2024-04-25 13:56:45.666775363 +0000 UTC m=+0.116399444 container create 94e0751fdcd2b34eeeef1eab2662b7baf98c760426d52280d4e4a4b56f164ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:56:45.945 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:45 smithi112 podman[62974]: 2024-04-25 13:56:45.91134134 +0000 UTC m=+0.360965418 container init 94e0751fdcd2b34eeeef1eab2662b7baf98c760426d52280d4e4a4b56f164ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T13:56:45.946 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:45 smithi112 podman[62974]: 2024-04-25 13:56:45.913783591 +0000 UTC m=+0.363407678 container start 94e0751fdcd2b34eeeef1eab2662b7baf98c760426d52280d4e4a4b56f164ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:56:45.946 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:45 smithi112 ceph-mds[62993]: -- 172.21.15.112:0/2988563357 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55deb7f9ba40 con 0x55deb7fe3180 2024-04-25T13:56:46.266 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:45 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e[62989]: starting mds.e at 2024-04-25T13:56:46.266 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:46 smithi112 bash[62974]: 94e0751fdcd2b34eeeef1eab2662b7baf98c760426d52280d4e4a4b56f164ea4 2024-04-25T13:56:46.266 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 13:56:46 smithi112 systemd[1]: Started Ceph mds.e for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:46.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:46.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 -- 172.21.15.78:0/3704603080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 msgr2=0x7f9aec1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3704603080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec1036c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009900 tx=0x7f9adc02a410 comp rx=0 tx=0).stop 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 -- 172.21.15.78:0/3704603080 shutdown_connections 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3704603080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec108900 0x7f9aec10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3704603080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9aec103c00 0x7f9aec104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3704603080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec1036c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 -- 172.21.15.78:0/3704603080 >> 172.21.15.78:0/3704603080 conn(0x7f9aec0fdbd0 msgr2=0x7f9aec100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 -- 172.21.15.78:0/3704603080 shutdown_connections 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.961+0000 7f9af0bee640 1 -- 172.21.15.78:0/3704603080 wait complete. 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 Processor -- start 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 -- start start 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:46.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec103c00 0x7f9aec199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9aec108900 0x7f9aec1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec10e510 con 0x7f9aec1029c0 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec10e390 con 0x7f9aec103c00 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9af0bee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec10e690 con 0x7f9aec108900 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59984/0 (socket says 172.21.15.78:59984) 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 -- 172.21.15.78:0/3819143545 learned_addr learned my addr 172.21.15.78:0/3819143545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aead9d640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9aec108900 0x7f9aec1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9ae9d9b640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec103c00 0x7f9aec199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 -- 172.21.15.78:0/3819143545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9aec108900 msgr2=0x7f9aec1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9aec108900 0x7f9aec1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 -- 172.21.15.78:0/3819143545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec103c00 msgr2=0x7f9aec199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:46.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec103c00 0x7f9aec199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.962+0000 7f9aea59c640 1 -- 172.21.15.78:0/3819143545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec1a0c80 con 0x7f9aec1029c0 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.963+0000 7f9aea59c640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec199140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9adc031440 tx=0x7f9adc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.963+0000 7f9adb7fe640 1 -- 172.21.15.78:0/3819143545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc03f070 con 0x7f9aec1029c0 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.963+0000 7f9adb7fe640 1 -- 172.21.15.78:0/3819143545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9adc03b460 con 0x7f9aec1029c0 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.963+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec1a0f10 con 0x7f9aec1029c0 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.963+0000 7f9adb7fe640 1 -- 172.21.15.78:0/3819143545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc035030 con 0x7f9aec1029c0 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.963+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec1a1310 con 0x7f9aec1029c0 2024-04-25T13:56:46.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.964+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec107710 con 0x7f9aec1029c0 2024-04-25T13:56:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.965+0000 7f9adb7fe640 1 -- 172.21.15.78:0/3819143545 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9adc042070 con 0x7f9aec1029c0 2024-04-25T13:56:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.966+0000 7f9adb7fe640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ad40afaf0 0x7f9ad40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.966+0000 7f9ae9d9b640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ad40afaf0 0x7f9ad40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.966+0000 7f9adb7fe640 1 -- 172.21.15.78:0/3819143545 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f9adc0ff330 con 0x7f9aec1029c0 2024-04-25T13:56:46.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.967+0000 7f9ae9d9b640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ad40afaf0 0x7f9ad40b2000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9aec19a1e0 tx=0x7f9ae000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:46.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:46.970+0000 7f9adb7fe640 1 -- 172.21.15.78:0/3819143545 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9adc0c7d60 con 0x7f9aec1029c0 2024-04-25T13:56:47.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.186+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9aec02db50 con 0x7f9aec1029c0 2024-04-25T13:56:47.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.187+0000 7f9adb7fe640 1 -- 172.21.15.78:0/3819143545 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f9adc0cc060 con 0x7f9aec1029c0 2024-04-25T13:56:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:47.187 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:56:47.187 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 6 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ad40afaf0 msgr2=0x7f9ad40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ad40afaf0 0x7f9ad40b2000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9aec19a1e0 tx=0x7f9ae000b390 comp rx=0 tx=0).stop 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 msgr2=0x7f9aec199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec199140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9adc031440 tx=0x7f9adc003d50 comp rx=0 tx=0).stop 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 shutdown_connections 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9aec108900 0x7f9aec1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec103c00 0x7f9aec199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:47.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ad40afaf0 0x7f9ad40b2000 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:47.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 --2- 172.21.15.78:0/3819143545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9aec1029c0 0x7f9aec199140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:47.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 >> 172.21.15.78:0/3819143545 conn(0x7f9aec0fdbd0 msgr2=0x7f9aec1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:47.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 shutdown_connections 2024-04-25T13:56:47.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:47.189+0000 7f9af0bee640 1 -- 172.21.15.78:0/3819143545 wait complete. 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: Deploying daemon mds.f on smithi129 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6834/2622963442,v1:172.21.15.112:6835/2622963442] up:boot 2024-04-25T13:56:47.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: fsmap 5 up:standby 2024-04-25T13:56:47.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:56:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: Deploying daemon mds.f on smithi129 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6834/2622963442,v1:172.21.15.112:6835/2622963442] up:boot 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: fsmap 5 up:standby 2024-04-25T13:56:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: Deploying daemon mds.f on smithi129 2024-04-25T13:56:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:47.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6834/2622963442,v1:172.21.15.112:6835/2622963442] up:boot 2024-04-25T13:56:47.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: fsmap 5 up:standby 2024-04-25T13:56:47.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:56:48.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:48 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3819143545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:48.414 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 systemd[1]: Starting Ceph mds.f for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:48 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3819143545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:48 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3819143545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:48.703 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 podman[62369]: 2024-04-25 13:56:48.33610813 +0000 UTC m=+0.018396962 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:48.703 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 podman[62369]: 2024-04-25 13:56:48.451676226 +0000 UTC m=+0.133965045 container create c887c63f1db75c467929f167b2f71e0059013c8337a52f2294f69ecfd48faa6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T13:56:48.703 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 podman[62369]: 2024-04-25 13:56:48.704491558 +0000 UTC m=+0.386780377 container init c887c63f1db75c467929f167b2f71e0059013c8337a52f2294f69ecfd48faa6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T13:56:49.090 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 podman[62369]: 2024-04-25 13:56:48.707578484 +0000 UTC m=+0.389867303 container start c887c63f1db75c467929f167b2f71e0059013c8337a52f2294f69ecfd48faa6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T13:56:49.090 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 ceph-mds[62389]: -- 172.21.15.129:0/1540711981 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562912f17a40 con 0x562912f5f180 2024-04-25T13:56:49.090 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f[62385]: starting mds.f at 2024-04-25T13:56:49.090 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 bash[62369]: c887c63f1db75c467929f167b2f71e0059013c8337a52f2294f69ecfd48faa6f 2024-04-25T13:56:49.090 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 13:56:48 smithi129 systemd[1]: Started Ceph mds.f for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:49.201 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:49.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:49 smithi112 ceph-mon[32617]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:49.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:49.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:49.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:49 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 -- 172.21.15.78:0/708151798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c108920 msgr2=0x7f6e2c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 --2- 172.21.15.78:0/708151798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c108920 0x7f6e2c10ad40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009ca0 tx=0x7f6e2402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 -- 172.21.15.78:0/708151798 shutdown_connections 2024-04-25T13:56:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 --2- 172.21.15.78:0/708151798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c108920 0x7f6e2c10ad40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 --2- 172.21.15.78:0/708151798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e2c103c20 0x7f6e2c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 --2- 172.21.15.78:0/708151798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e2c1029e0 0x7f6e2c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 -- 172.21.15.78:0/708151798 >> 172.21.15.78:0/708151798 conn(0x7f6e2c0fdbd0 msgr2=0x7f6e2c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:49.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 -- 172.21.15.78:0/708151798 shutdown_connections 2024-04-25T13:56:49.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.320+0000 7f6e34267640 1 -- 172.21.15.78:0/708151798 wait complete. 2024-04-25T13:56:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 Processor -- start 2024-04-25T13:56:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 -- start start 2024-04-25T13:56:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e2c1029e0 0x7f6e2c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c103c20 0x7f6e2c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e2c108920 0x7f6e2c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10f650 con 0x7f6e2c103c20 2024-04-25T13:56:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10f4d0 con 0x7f6e2c108920 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e34267640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10f7d0 con 0x7f6e2c1029e0 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e31802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c103c20 0x7f6e2c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e31802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c103c20 0x7f6e2c19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60000/0 (socket says 172.21.15.78:60000) 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e31802640 1 -- 172.21.15.78:0/3629965031 learned_addr learned my addr 172.21.15.78:0/3629965031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e32003640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e2c1029e0 0x7f6e2c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.321+0000 7f6e32804640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e2c108920 0x7f6e2c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e31802640 1 -- 172.21.15.78:0/3629965031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e2c1029e0 msgr2=0x7f6e2c19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e31802640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e2c1029e0 0x7f6e2c19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e31802640 1 -- 172.21.15.78:0/3629965031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e2c108920 msgr2=0x7f6e2c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e31802640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e2c108920 0x7f6e2c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e31802640 1 -- 172.21.15.78:0/3629965031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c1a1d60 con 0x7f6e2c103c20 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e31802640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c103c20 0x7f6e2c19a760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00e9f0 tx=0x7f6e1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e237fe640 1 -- 172.21.15.78:0/3629965031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00cd80 con 0x7f6e2c103c20 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e237fe640 1 -- 172.21.15.78:0/3629965031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e1c0108e0 con 0x7f6e2c103c20 2024-04-25T13:56:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c1a1ff0 con 0x7f6e2c103c20 2024-04-25T13:56:49.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e237fe640 1 -- 172.21.15.78:0/3629965031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c018980 con 0x7f6e2c103c20 2024-04-25T13:56:49.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.322+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1a24e0 con 0x7f6e2c103c20 2024-04-25T13:56:49.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.323+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c107730 con 0x7f6e2c103c20 2024-04-25T13:56:49.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.324+0000 7f6e237fe640 1 -- 172.21.15.78:0/3629965031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6e1c010430 con 0x7f6e2c103c20 2024-04-25T13:56:49.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.325+0000 7f6e237fe640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e180afaf0 0x7f6e180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:49.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.325+0000 7f6e32003640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e180afaf0 0x7f6e180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:49.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.325+0000 7f6e237fe640 1 -- 172.21.15.78:0/3629965031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f6e1c0dea50 con 0x7f6e2c103c20 2024-04-25T13:56:49.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.325+0000 7f6e32003640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e180afaf0 0x7f6e180b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c1035e0 tx=0x7f6e28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.330+0000 7f6e237fe640 1 -- 172.21.15.78:0/3629965031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e1c0a7480 con 0x7f6e2c103c20 2024-04-25T13:56:49.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:49 smithi129 ceph-mon[32569]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:49.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:49.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:49 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:49 smithi078 ceph-mon[29163]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:49 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:49.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.545+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e2c0637a0 con 0x7f6e2c103c20 2024-04-25T13:56:49.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.545+0000 7f6e237fe640 1 -- 172.21.15.78:0/3629965031 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f6e1c0ab780 con 0x7f6e2c103c20 2024-04-25T13:56:49.546 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:49.546 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:56:49.546 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 7 2024-04-25T13:56:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e180afaf0 msgr2=0x7f6e180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e180afaf0 0x7f6e180b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c1035e0 tx=0x7f6e28005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c103c20 msgr2=0x7f6e2c19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c103c20 0x7f6e2c19a760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00e9f0 tx=0x7f6e1c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 shutdown_connections 2024-04-25T13:56:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e2c108920 0x7f6e2c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e180afaf0 0x7f6e180b2000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2c103c20 0x7f6e2c19a760 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 --2- 172.21.15.78:0/3629965031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e2c1029e0 0x7f6e2c19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:49.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 >> 172.21.15.78:0/3629965031 conn(0x7f6e2c0fdbd0 msgr2=0x7f6e2c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:49.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 shutdown_connections 2024-04-25T13:56:49.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:49.547+0000 7f6e34267640 1 -- 172.21.15.78:0/3629965031 wait complete. 2024-04-25T13:56:50.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:50 smithi129 ceph-mon[32569]: Deploying daemon mds.c on smithi129 2024-04-25T13:56:50.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:50 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] up:boot 2024-04-25T13:56:50.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:50 smithi129 ceph-mon[32569]: fsmap 6 up:standby 2024-04-25T13:56:50.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:50 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:56:50.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:50 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3629965031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:50 smithi078 ceph-mon[29163]: Deploying daemon mds.c on smithi129 2024-04-25T13:56:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:50 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] up:boot 2024-04-25T13:56:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:50 smithi078 ceph-mon[29163]: fsmap 6 up:standby 2024-04-25T13:56:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:50 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:56:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:50 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3629965031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:50 smithi112 ceph-mon[32617]: Deploying daemon mds.c on smithi129 2024-04-25T13:56:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:50 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] up:boot 2024-04-25T13:56:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:50 smithi112 ceph-mon[32617]: fsmap 6 up:standby 2024-04-25T13:56:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:50 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:56:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:50 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3629965031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:51.097 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 systemd[1]: Starting Ceph mds.c for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:51.361 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:51 smithi129 ceph-mon[32569]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:51.361 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 podman[62679]: 2024-04-25 13:56:51.149229416 +0000 UTC m=+0.017566397 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:51.361 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 podman[62679]: 2024-04-25 13:56:51.265407063 +0000 UTC m=+0.133744043 container create 62bd797bd311249dbd00765d888b58a23c45c1d750c2852c3b2c52db7381c229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:56:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:51 smithi078 ceph-mon[29163]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:51.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:51 smithi112 ceph-mon[32617]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:51.560 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:51.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 -- 172.21.15.78:0/2363101359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c102a00 msgr2=0x7f0c4c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:51.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 --2- 172.21.15.78:0/2363101359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c102a00 0x7f0c4c103700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009900 tx=0x7f0c3c02a410 comp rx=0 tx=0).stop 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 -- 172.21.15.78:0/2363101359 shutdown_connections 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 --2- 172.21.15.78:0/2363101359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c4c108940 0x7f0c4c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 --2- 172.21.15.78:0/2363101359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c4c103c40 0x7f0c4c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 --2- 172.21.15.78:0/2363101359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c102a00 0x7f0c4c103700 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 -- 172.21.15.78:0/2363101359 >> 172.21.15.78:0/2363101359 conn(0x7f0c4c0fdbf0 msgr2=0x7f0c4c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 -- 172.21.15.78:0/2363101359 shutdown_connections 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.685+0000 7f0c51972640 1 -- 172.21.15.78:0/2363101359 wait complete. 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 Processor -- start 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 -- start start 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c4c102a00 0x7f0c4c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c103c40 0x7f0c4c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c4c108940 0x7f0c4c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:51.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10e4e0 con 0x7f0c4c103c40 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10e360 con 0x7f0c4c108940 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c51972640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10e660 con 0x7f0c4c102a00 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c103c40 0x7f0c4c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c103c40 0x7f0c4c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60020/0 (socket says 172.21.15.78:60020) 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c43fff640 1 -- 172.21.15.78:0/361295595 learned_addr learned my addr 172.21.15.78:0/361295595 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c4b7fe640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c4c108940 0x7f0c4c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.686+0000 7f0c4affd640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c4c102a00 0x7f0c4c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c43fff640 1 -- 172.21.15.78:0/361295595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c4c102a00 msgr2=0x7f0c4c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c43fff640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c4c102a00 0x7f0c4c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c43fff640 1 -- 172.21.15.78:0/361295595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c4c108940 msgr2=0x7f0c4c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c43fff640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c4c108940 0x7f0c4c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c43fff640 1 -- 172.21.15.78:0/361295595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c1a1d40 con 0x7f0c4c103c40 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c4b7fe640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c4c108940 0x7f0c4c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:56:51.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c43fff640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c103c40 0x7f0c4c19a740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e9f0 tx=0x7f0c3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:51.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c50970640 1 -- 172.21.15.78:0/361295595 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400cd80 con 0x7f0c4c103c40 2024-04-25T13:56:51.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c1a1fd0 con 0x7f0c4c103c40 2024-04-25T13:56:51.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c50970640 1 -- 172.21.15.78:0/361295595 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c340108e0 con 0x7f0c4c103c40 2024-04-25T13:56:51.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c50970640 1 -- 172.21.15.78:0/361295595 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34018980 con 0x7f0c4c103c40 2024-04-25T13:56:51.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.687+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c1a24e0 con 0x7f0c4c103c40 2024-04-25T13:56:51.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.688+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4c1a2740 con 0x7f0c4c103c40 2024-04-25T13:56:51.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.689+0000 7f0c50970640 1 -- 172.21.15.78:0/361295595 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0c34010430 con 0x7f0c4c103c40 2024-04-25T13:56:51.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.690+0000 7f0c50970640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0c380afaf0 0x7f0c380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:51.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.690+0000 7f0c4affd640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0c380afaf0 0x7f0c380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:51.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.690+0000 7f0c50970640 1 -- 172.21.15.78:0/361295595 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0c340def60 con 0x7f0c4c103c40 2024-04-25T13:56:51.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.691+0000 7f0c4affd640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0c380afaf0 0x7f0c380b2000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c005ab0 tx=0x7f0c3c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:51.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.694+0000 7f0c50970640 1 -- 172.21.15.78:0/361295595 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c340a7990 con 0x7f0c4c103c40 2024-04-25T13:56:51.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.909+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c4c1a2910 con 0x7f0c4c103c40 2024-04-25T13:56:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.910+0000 7f0c50970640 1 -- 172.21.15.78:0/361295595 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f0c340b0060 con 0x7f0c4c103c40 2024-04-25T13:56:51.910 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:51.910 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:56:51.910 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 7 2024-04-25T13:56:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0c380afaf0 msgr2=0x7f0c380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0c380afaf0 0x7f0c380b2000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c005ab0 tx=0x7f0c3c031400 comp rx=0 tx=0).stop 2024-04-25T13:56:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c103c40 msgr2=0x7f0c4c19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c103c40 0x7f0c4c19a740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e9f0 tx=0x7f0c3400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 shutdown_connections 2024-04-25T13:56:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c4c108940 0x7f0c4c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0c380afaf0 0x7f0c380b2000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c4c103c40 0x7f0c4c19a740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 --2- 172.21.15.78:0/361295595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c4c102a00 0x7f0c4c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 >> 172.21.15.78:0/361295595 conn(0x7f0c4c0fdbf0 msgr2=0x7f0c4c109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 shutdown_connections 2024-04-25T13:56:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:51.912+0000 7f0c51972640 1 -- 172.21.15.78:0/361295595 wait complete. 2024-04-25T13:56:51.913 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 podman[62679]: 2024-04-25 13:56:51.468131211 +0000 UTC m=+0.336468191 container init 62bd797bd311249dbd00765d888b58a23c45c1d750c2852c3b2c52db7381c229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T13:56:51.913 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 podman[62679]: 2024-04-25 13:56:51.470623461 +0000 UTC m=+0.338960442 container start 62bd797bd311249dbd00765d888b58a23c45c1d750c2852c3b2c52db7381c229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:56:51.913 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 ceph-mds[62699]: -- 172.21.15.129:0/113143229 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc7567ba40 con 0x55bc756c3600 2024-04-25T13:56:51.914 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c[62695]: starting mds.c at 2024-04-25T13:56:51.914 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 bash[62679]: 62bd797bd311249dbd00765d888b58a23c45c1d750c2852c3b2c52db7381c229 2024-04-25T13:56:51.914 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 13:56:51 smithi129 systemd[1]: Started Ceph mds.c for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: Deploying daemon mds.h on smithi112 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/361295595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:52.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6836/750801105,v1:172.21.15.129:6837/750801105] up:boot 2024-04-25T13:56:52.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: fsmap 7 up:standby 2024-04-25T13:56:52.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:56:52.915 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:52 smithi129 ceph-mon[32569]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:52.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:52.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:52.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:52.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:52.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:52.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:52.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: Deploying daemon mds.h on smithi112 2024-04-25T13:56:52.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/361295595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:52.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6836/750801105,v1:172.21.15.129:6837/750801105] up:boot 2024-04-25T13:56:52.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: fsmap 7 up:standby 2024-04-25T13:56:52.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:56:52.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:52 smithi078 ceph-mon[29163]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:53.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:53.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:53.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:53.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:53.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:53.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:53.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: Deploying daemon mds.h on smithi112 2024-04-25T13:56:53.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/361295595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:53.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6836/750801105,v1:172.21.15.129:6837/750801105] up:boot 2024-04-25T13:56:53.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: fsmap 7 up:standby 2024-04-25T13:56:53.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:56:53.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:52 smithi112 ceph-mon[32617]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:53.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:54.016 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:53 smithi112 systemd[1]: Starting Ceph mds.h for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.042+0000 7fec73a97640 1 -- 172.21.15.78:0/1285797841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c109a40 msgr2=0x7fec6c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.042+0000 7fec73a97640 1 --2- 172.21.15.78:0/1285797841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c109a40 0x7fec6c10be60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009900 tx=0x7fec5c02a410 comp rx=0 tx=0).stop 2024-04-25T13:56:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.042+0000 7fec73a97640 1 -- 172.21.15.78:0/1285797841 shutdown_connections 2024-04-25T13:56:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.042+0000 7fec73a97640 1 --2- 172.21.15.78:0/1285797841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c109a40 0x7fec6c10be60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.042+0000 7fec73a97640 1 --2- 172.21.15.78:0/1285797841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c104d40 0x7fec6c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.042+0000 7fec73a97640 1 --2- 172.21.15.78:0/1285797841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec6c103b00 0x7fec6c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.042+0000 7fec73a97640 1 -- 172.21.15.78:0/1285797841 >> 172.21.15.78:0/1285797841 conn(0x7fec6c0fed10 msgr2=0x7fec6c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 -- 172.21.15.78:0/1285797841 shutdown_connections 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 -- 172.21.15.78:0/1285797841 wait complete. 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 Processor -- start 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 -- start start 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c103b00 0x7fec6c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec6c104d40 0x7fec6c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c109a40 0x7fec6c1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.043+0000 7fec73a97640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c10f610 con 0x7fec6c103b00 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec73a97640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c10f490 con 0x7fec6c109a40 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec73a97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c10f790 con 0x7fec6c104d40 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c103b00 0x7fec6c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c103b00 0x7fec6c19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60050/0 (socket says 172.21.15.78:60050) 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 -- 172.21.15.78:0/2894826414 learned_addr learned my addr 172.21.15.78:0/2894826414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec72034640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c109a40 0x7fec6c1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71032640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec6c104d40 0x7fec6c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 -- 172.21.15.78:0/2894826414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec6c104d40 msgr2=0x7fec6c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec6c104d40 0x7fec6c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 -- 172.21.15.78:0/2894826414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c109a40 msgr2=0x7fec6c1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c109a40 0x7fec6c1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 -- 172.21.15.78:0/2894826414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c1a1c40 con 0x7fec6c103b00 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec72034640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c109a40 0x7fec6c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec71833640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c103b00 0x7fec6c19a1b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec6400cce0 tx=0x7fec6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec6affd640 1 -- 172.21.15.78:0/2894826414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6400f450 con 0x7fec6c103b00 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec6affd640 1 -- 172.21.15.78:0/2894826414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6400faa0 con 0x7fec6c103b00 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c1a1f30 con 0x7fec6c103b00 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec6affd640 1 -- 172.21.15.78:0/2894826414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64018b70 con 0x7fec6c103b00 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.044+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c1a23f0 con 0x7fec6c103b00 2024-04-25T13:56:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.045+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6c108850 con 0x7fec6c103b00 2024-04-25T13:56:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.046+0000 7fec6affd640 1 -- 172.21.15.78:0/2894826414 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fec64018d10 con 0x7fec6c103b00 2024-04-25T13:56:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.047+0000 7fec6affd640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec540afb70 0x7fec540b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.047+0000 7fec71032640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec540afb70 0x7fec540b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.047+0000 7fec6affd640 1 -- 172.21.15.78:0/2894826414 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fec640deba0 con 0x7fec6c103b00 2024-04-25T13:56:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.048+0000 7fec71032640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec540afb70 0x7fec540b2080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec6c19b250 tx=0x7fec60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.052+0000 7fec6affd640 1 -- 172.21.15.78:0/2894826414 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec64014030 con 0x7fec6c103b00 2024-04-25T13:56:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.274+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec6c02dd20 con 0x7fec6c103b00 2024-04-25T13:56:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.275+0000 7fec6affd640 1 -- 172.21.15.78:0/2894826414 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7fec640a7c20 con 0x7fec6c103b00 2024-04-25T13:56:54.275 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:54.275 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:56:54.275 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec540afb70 msgr2=0x7fec540b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec540afb70 0x7fec540b2080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec6c19b250 tx=0x7fec60005f90 comp rx=0 tx=0).stop 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c103b00 msgr2=0x7fec6c19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c103b00 0x7fec6c19a1b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec6400cce0 tx=0x7fec6400a620 comp rx=0 tx=0).stop 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 shutdown_connections 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec6c109a40 0x7fec6c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec6c104d40 0x7fec6c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec540afb70 0x7fec540b2080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 --2- 172.21.15.78:0/2894826414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec6c103b00 0x7fec6c19a1b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:54.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 >> 172.21.15.78:0/2894826414 conn(0x7fec6c0fed10 msgr2=0x7fec6c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:54.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 shutdown_connections 2024-04-25T13:56:54.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:54.277+0000 7fec73a97640 1 -- 172.21.15.78:0/2894826414 wait complete. 2024-04-25T13:56:54.358 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 podman[63290]: 2024-04-25 13:56:53.965227896 +0000 UTC m=+0.017988985 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:54.359 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 podman[63290]: 2024-04-25 13:56:54.073763655 +0000 UTC m=+0.126524732 container create 7871954402fe2c9296b63ae806739f234428d846b10718f6479e5d57cf8567a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:56:54.359 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 podman[63290]: 2024-04-25 13:56:54.360046752 +0000 UTC m=+0.412807827 container init 7871954402fe2c9296b63ae806739f234428d846b10718f6479e5d57cf8567a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:56:54.766 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 podman[63290]: 2024-04-25 13:56:54.362763961 +0000 UTC m=+0.415525041 container start 7871954402fe2c9296b63ae806739f234428d846b10718f6479e5d57cf8567a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:56:54.766 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 ceph-mds[63309]: -- 172.21.15.112:0/4095453911 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b249963a40 con 0x55b2499ab180 2024-04-25T13:56:54.766 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h[63305]: starting mds.h at 2024-04-25T13:56:54.766 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 bash[63290]: 7871954402fe2c9296b63ae806739f234428d846b10718f6479e5d57cf8567a3 2024-04-25T13:56:54.766 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 13:56:54 smithi112 systemd[1]: Started Ceph mds.h for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2894826414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:55.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:55 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2894826414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:55.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:55.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:55 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2894826414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:55 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:56.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:56.404 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:56 smithi129 ceph-mon[32569]: Deploying daemon mds.i on smithi129 2024-04-25T13:56:56.404 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:56 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6836/4152465978,v1:172.21.15.112:6837/4152465978] up:boot 2024-04-25T13:56:56.404 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:56 smithi129 ceph-mon[32569]: fsmap 8 up:standby 2024-04-25T13:56:56.404 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:56 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:56:56.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 -- 172.21.15.78:0/2448653989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 msgr2=0x7f6b481069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/2448653989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b481069d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009900 tx=0x7f6b3c02a410 comp rx=0 tx=0).stop 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 -- 172.21.15.78:0/2448653989 shutdown_connections 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/2448653989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b48109870 0x7f6b4810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/2448653989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b48106f10 0x7f6b48109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/2448653989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b481069d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 -- 172.21.15.78:0/2448653989 >> 172.21.15.78:0/2448653989 conn(0x7f6b480fdbd0 msgr2=0x7f6b48100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 -- 172.21.15.78:0/2448653989 shutdown_connections 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.414+0000 7f6b4cb00640 1 -- 172.21.15.78:0/2448653989 wait complete. 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 Processor -- start 2024-04-25T13:56:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 -- start start 2024-04-25T13:56:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b4819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b48106f10 0x7f6b4819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b48109870 0x7f6b481a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4810f5a0 con 0x7f6b481045b0 2024-04-25T13:56:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4810f420 con 0x7f6b48109870 2024-04-25T13:56:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4cb00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4810f720 con 0x7f6b48106f10 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b4819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b4819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60056/0 (socket says 172.21.15.78:60056) 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b4659c640 1 -- 172.21.15.78:0/1651128058 learned_addr learned my addr 172.21.15.78:0/1651128058 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b45d9b640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b48106f10 0x7f6b4819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.415+0000 7f6b46d9d640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b48109870 0x7f6b481a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4659c640 1 -- 172.21.15.78:0/1651128058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b48106f10 msgr2=0x7f6b4819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4659c640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b48106f10 0x7f6b4819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4659c640 1 -- 172.21.15.78:0/1651128058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b48109870 msgr2=0x7f6b481a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4659c640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b48109870 0x7f6b481a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4659c640 1 -- 172.21.15.78:0/1651128058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b481a1d40 con 0x7f6b481045b0 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4659c640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b4819a200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006010 tx=0x7f6b3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b377fe640 1 -- 172.21.15.78:0/1651128058 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c03f030 con 0x7f6b481045b0 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b481a1fd0 con 0x7f6b481045b0 2024-04-25T13:56:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b377fe640 1 -- 172.21.15.78:0/1651128058 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3c003aa0 con 0x7f6b481045b0 2024-04-25T13:56:56.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b377fe640 1 -- 172.21.15.78:0/1651128058 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c035070 con 0x7f6b481045b0 2024-04-25T13:56:56.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.416+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b481a2430 con 0x7f6b481045b0 2024-04-25T13:56:56.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.417+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b480ff6f0 con 0x7f6b481045b0 2024-04-25T13:56:56.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.420+0000 7f6b377fe640 1 -- 172.21.15.78:0/1651128058 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6b3c042030 con 0x7f6b481045b0 2024-04-25T13:56:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.420+0000 7f6b377fe640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b300afaf0 0x7f6b300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.420+0000 7f6b45d9b640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b300afaf0 0x7f6b300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.420+0000 7f6b377fe640 1 -- 172.21.15.78:0/1651128058 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f6b3c0ff160 con 0x7f6b481045b0 2024-04-25T13:56:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.421+0000 7f6b45d9b640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b300afaf0 0x7f6b300b2000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b4819b2a0 tx=0x7f6b38009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.424+0000 7f6b377fe640 1 -- 172.21.15.78:0/1651128058 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b3c0c7b90 con 0x7f6b481045b0 2024-04-25T13:56:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:56 smithi078 ceph-mon[29163]: Deploying daemon mds.i on smithi129 2024-04-25T13:56:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:56 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6836/4152465978,v1:172.21.15.112:6837/4152465978] up:boot 2024-04-25T13:56:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:56 smithi078 ceph-mon[29163]: fsmap 8 up:standby 2024-04-25T13:56:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:56 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:56:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:56 smithi112 ceph-mon[32617]: Deploying daemon mds.i on smithi129 2024-04-25T13:56:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:56 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6836/4152465978,v1:172.21.15.112:6837/4152465978] up:boot 2024-04-25T13:56:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:56 smithi112 ceph-mon[32617]: fsmap 8 up:standby 2024-04-25T13:56:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:56 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:56:56.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.641+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b4802dc70 con 0x7f6b481045b0 2024-04-25T13:56:56.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.641+0000 7f6b377fe640 1 -- 172.21.15.78:0/1651128058 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f6b3c0cbe90 con 0x7f6b481045b0 2024-04-25T13:56:56.641 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:56.642 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T13:56:56.642 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 9 2024-04-25T13:56:56.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.643+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b300afaf0 msgr2=0x7f6b300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.643+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b300afaf0 0x7f6b300b2000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b4819b2a0 tx=0x7f6b38009680 comp rx=0 tx=0).stop 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 msgr2=0x7f6b4819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b4819a200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006010 tx=0x7f6b3c004270 comp rx=0 tx=0).stop 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 shutdown_connections 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b48109870 0x7f6b481a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b48106f10 0x7f6b4819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b300afaf0 0x7f6b300b2000 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 --2- 172.21.15.78:0/1651128058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b481045b0 0x7f6b4819a200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 >> 172.21.15.78:0/1651128058 conn(0x7f6b480fdbd0 msgr2=0x7f6b48105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 shutdown_connections 2024-04-25T13:56:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:56.644+0000 7f6b4cb00640 1 -- 172.21.15.78:0/1651128058 wait complete. 2024-04-25T13:56:56.663 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:56 smithi129 systemd[1]: Starting Ceph mds.i for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:57.084 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:56 smithi129 podman[62990]: 2024-04-25 13:56:56.713438029 +0000 UTC m=+0.018764681 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:57.084 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:56 smithi129 podman[62990]: 2024-04-25 13:56:56.829560038 +0000 UTC m=+0.134886704 container create 48581fb537166f55162f4880b02673fcb023f3b943d1d2503487971ac1a3fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T13:56:57.084 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:57 smithi129 podman[62990]: 2024-04-25 13:56:57.048908218 +0000 UTC m=+0.354234841 container init 48581fb537166f55162f4880b02673fcb023f3b943d1d2503487971ac1a3fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T13:56:57.084 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:57 smithi129 podman[62990]: 2024-04-25 13:56:57.051533636 +0000 UTC m=+0.356860261 container start 48581fb537166f55162f4880b02673fcb023f3b943d1d2503487971ac1a3fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T13:56:57.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:57 smithi129 ceph-mon[32569]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:57.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:57 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1651128058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:57.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:57 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:57.414 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:57 smithi129 ceph-mds[63009]: -- 172.21.15.129:0/357279588 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56058cc77a40 con 0x56058ccbf600 2024-04-25T13:56:57.414 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:57 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i[63005]: starting mds.i at 2024-04-25T13:56:57.414 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:57 smithi129 bash[62990]: 48581fb537166f55162f4880b02673fcb023f3b943d1d2503487971ac1a3fd85 2024-04-25T13:56:57.414 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 13:56:57 smithi129 systemd[1]: Started Ceph mds.i for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:56:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:57 smithi078 ceph-mon[29163]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:57 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1651128058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:57 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:57 smithi112 ceph-mon[32617]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:57 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1651128058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:57 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] up:boot 2024-04-25T13:56:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: fsmap 9 up:standby 2024-04-25T13:56:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:56:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:58 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] up:boot 2024-04-25T13:56:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: fsmap 9 up:standby 2024-04-25T13:56:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:56:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:58.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:58.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:58 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:58.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:56:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] up:boot 2024-04-25T13:56:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: fsmap 9 up:standby 2024-04-25T13:56:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:56:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:56:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:56:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:56:58.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:58 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:56:58.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.769+0000 7fb80f59e640 1 -- 172.21.15.78:0/764299061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810108740 msgr2=0x7fb81010ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.769+0000 7fb80f59e640 1 --2- 172.21.15.78:0/764299061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810108740 0x7fb81010ab60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb808009ca0 tx=0x7fb80802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 -- 172.21.15.78:0/764299061 shutdown_connections 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 --2- 172.21.15.78:0/764299061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810108740 0x7fb81010ab60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 --2- 172.21.15.78:0/764299061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810103a40 0x7fb810103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 --2- 172.21.15.78:0/764299061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb810102800 0x7fb810103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 -- 172.21.15.78:0/764299061 >> 172.21.15.78:0/764299061 conn(0x7fb8100fd9f0 msgr2=0x7fb8100ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 -- 172.21.15.78:0/764299061 shutdown_connections 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 -- 172.21.15.78:0/764299061 wait complete. 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 Processor -- start 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.770+0000 7fb80f59e640 1 -- start start 2024-04-25T13:56:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810102800 0x7fb810198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb810103a40 0x7fb810199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810108740 0x7fb8101a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb81010e310 con 0x7fb810102800 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb81010e190 con 0x7fb810108740 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb81010e490 con 0x7fb810103a40 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810102800 0x7fb810198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810102800 0x7fb810198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45148/0 (socket says 172.21.15.78:45148) 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 -- 172.21.15.78:0/3713118866 learned_addr learned my addr 172.21.15.78:0/3713118866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80ed9d640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810108740 0x7fb8101a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 -- 172.21.15.78:0/3713118866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb810103a40 msgr2=0x7fb810199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb810103a40 0x7fb810199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 -- 172.21.15.78:0/3713118866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810108740 msgr2=0x7fb8101a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810108740 0x7fb8101a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 -- 172.21.15.78:0/3713118866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8101a0970 con 0x7fb810102800 2024-04-25T13:56:58.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80e59c640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810102800 0x7fb810198ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb800005fd0 tx=0x7fb80000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:58.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb8077fe640 1 -- 172.21.15.78:0/3713118866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800017070 con 0x7fb810102800 2024-04-25T13:56:58.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8101a0c60 con 0x7fb810102800 2024-04-25T13:56:58.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb8077fe640 1 -- 172.21.15.78:0/3713118866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb800009c30 con 0x7fb810102800 2024-04-25T13:56:58.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb8077fe640 1 -- 172.21.15.78:0/3713118866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800003d10 con 0x7fb810102800 2024-04-25T13:56:58.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.771+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8101a11b0 con 0x7fb810102800 2024-04-25T13:56:58.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.773+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb810107550 con 0x7fb810102800 2024-04-25T13:56:58.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.774+0000 7fb8077fe640 1 -- 172.21.15.78:0/3713118866 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb800003eb0 con 0x7fb810102800 2024-04-25T13:56:58.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.775+0000 7fb8077fe640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7fc0afae0 0x7fb7fc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:56:58.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.775+0000 7fb807fff640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7fc0afae0 0x7fb7fc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:56:58.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.775+0000 7fb8077fe640 1 -- 172.21.15.78:0/3713118866 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb8000decd0 con 0x7fb810102800 2024-04-25T13:56:58.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.775+0000 7fb807fff640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7fc0afae0 0x7fb7fc0b1ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb810199f80 tx=0x7fb7f8005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:56:58.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.778+0000 7fb8077fe640 1 -- 172.21.15.78:0/3713118866 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb800020030 con 0x7fb810102800 2024-04-25T13:56:58.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:58.999+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb81002dd10 con 0x7fb810102800 2024-04-25T13:56:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.000+0000 7fb8077fe640 1 -- 172.21.15.78:0/3713118866 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fb800009de0 con 0x7fb810102800 2024-04-25T13:56:59.000 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:56:59.000 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:56:59.000 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 10 2024-04-25T13:56:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7fc0afae0 msgr2=0x7fb7fc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7fc0afae0 0x7fb7fc0b1ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb810199f80 tx=0x7fb7f8005c00 comp rx=0 tx=0).stop 2024-04-25T13:56:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810102800 msgr2=0x7fb810198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:56:59.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810102800 0x7fb810198ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb800005fd0 tx=0x7fb80000dde0 comp rx=0 tx=0).stop 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 shutdown_connections 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb810108740 0x7fb8101a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb810103a40 0x7fb810199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7fc0afae0 0x7fb7fc0b1ff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 --2- 172.21.15.78:0/3713118866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810102800 0x7fb810198ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 >> 172.21.15.78:0/3713118866 conn(0x7fb8100fd9f0 msgr2=0x7fb810104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 shutdown_connections 2024-04-25T13:56:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:56:59.002+0000 7fb80f59e640 1 -- 172.21.15.78:0/3713118866 wait complete. 2024-04-25T13:56:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:59 smithi078 ceph-mon[29163]: Deploying daemon mds.g on smithi078 2024-04-25T13:56:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:59 smithi078 ceph-mon[29163]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:56:59 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3713118866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:59.469 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 systemd[1]: Starting Ceph mds.g for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:56:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:59 smithi112 ceph-mon[32617]: Deploying daemon mds.g on smithi078 2024-04-25T13:56:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:59 smithi112 ceph-mon[32617]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:56:59 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3713118866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:59 smithi129 ceph-mon[32569]: Deploying daemon mds.g on smithi078 2024-04-25T13:56:59.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:59 smithi129 ceph-mon[32569]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:56:59.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:56:59 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3713118866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:56:59.891 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 podman[69784]: 2024-04-25 13:56:59.529249815 +0000 UTC m=+0.018055625 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:56:59.891 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 podman[69784]: 2024-04-25 13:56:59.647466516 +0000 UTC m=+0.136272319 container create 8a2adb1f743e339a518eb76e83581a49ca4d3287574889103f1cade2da3a269c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:56:59.891 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 podman[69784]: 2024-04-25 13:56:59.892017183 +0000 UTC m=+0.380822984 container init 8a2adb1f743e339a518eb76e83581a49ca4d3287574889103f1cade2da3a269c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T13:57:00.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 podman[69784]: 2024-04-25 13:56:59.895001094 +0000 UTC m=+0.383806905 container start 8a2adb1f743e339a518eb76e83581a49ca4d3287574889103f1cade2da3a269c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T13:57:00.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 ceph-mds[69803]: -- 172.21.15.78:0/3405482947 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561bb3605a40 con 0x561bb364da80 2024-04-25T13:57:00.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g[69799]: starting mds.g at 2024-04-25T13:57:00.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 bash[69784]: 8a2adb1f743e339a518eb76e83581a49ca4d3287574889103f1cade2da3a269c 2024-04-25T13:57:00.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 13:56:59 smithi078 systemd[1]: Started Ceph mds.g for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:57:01.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:57:01.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 -- 172.21.15.78:0/3832718890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09501045c0 msgr2=0x7f09501069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:01.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 --2- 172.21.15.78:0/3832718890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09501045c0 0x7f09501069e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0944009900 tx=0x7f094402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:01.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 -- 172.21.15.78:0/3832718890 shutdown_connections 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 --2- 172.21.15.78:0/3832718890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0950109880 0x7f095010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 --2- 172.21.15.78:0/3832718890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950106f20 0x7f0950109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 --2- 172.21.15.78:0/3832718890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09501045c0 0x7f09501069e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 -- 172.21.15.78:0/3832718890 >> 172.21.15.78:0/3832718890 conn(0x7f09500fdba0 msgr2=0x7f0950100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 -- 172.21.15.78:0/3832718890 shutdown_connections 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.129+0000 7f0956172640 1 -- 172.21.15.78:0/3832718890 wait complete. 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 Processor -- start 2024-04-25T13:57:01.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 -- start start 2024-04-25T13:57:01.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09501045c0 0x7f0950103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:01.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950106f20 0x7f09501020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:01.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0950109880 0x7f09501025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f095010f5d0 con 0x7f0950109880 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f095010f450 con 0x7f0950106f20 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f0956172640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f095010f750 con 0x7f09501045c0 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0950109880 0x7f09501025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09501045c0 0x7f0950103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0950109880 0x7f09501025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45192/0 (socket says 172.21.15.78:45192) 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 -- 172.21.15.78:0/3209680927 learned_addr learned my addr 172.21.15.78:0/3209680927 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094effd640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950106f20 0x7f09501020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 -- 172.21.15.78:0/3209680927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09501045c0 msgr2=0x7f0950103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09501045c0 0x7f0950103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 -- 172.21.15.78:0/3209680927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950106f20 msgr2=0x7f09501020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950106f20 0x7f09501020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.130+0000 7f094ffff640 1 -- 172.21.15.78:0/3209680927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0950102d20 con 0x7f0950109880 2024-04-25T13:57:01.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.131+0000 7f094ffff640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0950109880 0x7f09501025f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f094800eb40 tx=0x7f094800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:01.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.131+0000 7f094cff9640 1 -- 172.21.15.78:0/3209680927 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948004b20 con 0x7f0950109880 2024-04-25T13:57:01.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.131+0000 7f094cff9640 1 -- 172.21.15.78:0/3209680927 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0948004cc0 con 0x7f0950109880 2024-04-25T13:57:01.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.131+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09501a42c0 con 0x7f0950109880 2024-04-25T13:57:01.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.131+0000 7f094cff9640 1 -- 172.21.15.78:0/3209680927 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948010720 con 0x7f0950109880 2024-04-25T13:57:01.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.131+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09501a4780 con 0x7f0950109880 2024-04-25T13:57:01.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.132+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09500ff6c0 con 0x7f0950109880 2024-04-25T13:57:01.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.133+0000 7f094cff9640 1 -- 172.21.15.78:0/3209680927 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0948005560 con 0x7f0950109880 2024-04-25T13:57:01.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.134+0000 7f094cff9640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09380afaf0 0x7f09380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:01.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.134+0000 7f094f7fe640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09380afaf0 0x7f09380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:01.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.134+0000 7f094cff9640 1 -- 172.21.15.78:0/3209680927 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f09480deec0 con 0x7f0950109880 2024-04-25T13:57:01.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.134+0000 7f094f7fe640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09380afaf0 0x7f09380b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0944009230 tx=0x7f0944031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:01.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.139+0000 7f094cff9640 1 -- 172.21.15.78:0/3209680927 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09480a7f40 con 0x7f0950109880 2024-04-25T13:57:01.287 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: Deploying daemon mds.a on smithi078 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.78:6838/2808782144,v1:172.21.15.78:6839/2808782144] up:boot 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: fsmap 10 up:standby 2024-04-25T13:57:01.288 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:01 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:57:01.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.363+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f095002db50 con 0x7f0950109880 2024-04-25T13:57:01.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.363+0000 7f094cff9640 1 -- 172.21.15.78:0/3209680927 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f09480a7840 con 0x7f0950109880 2024-04-25T13:57:01.363 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:01.364 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:57:01.364 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 11 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09380afaf0 msgr2=0x7f09380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09380afaf0 0x7f09380b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0944009230 tx=0x7f0944031400 comp rx=0 tx=0).stop 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0950109880 msgr2=0x7f09501025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0950109880 0x7f09501025f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f094800eb40 tx=0x7f094800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 shutdown_connections 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09380afaf0 0x7f09380b2000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0950109880 0x7f09501025f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0950106f20 0x7f09501020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 --2- 172.21.15.78:0/3209680927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09501045c0 0x7f0950103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 >> 172.21.15.78:0/3209680927 conn(0x7f09500fdba0 msgr2=0x7f09501051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:01.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 shutdown_connections 2024-04-25T13:57:01.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:01.366+0000 7f0956172640 1 -- 172.21.15.78:0/3209680927 wait complete. 2024-04-25T13:57:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: Deploying daemon mds.a on smithi078 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.78:6838/2808782144,v1:172.21.15.78:6839/2808782144] up:boot 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: fsmap 10 up:standby 2024-04-25T13:57:01.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:01 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:57:01.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:01.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:57:01.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:57:01.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:01.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: Deploying daemon mds.a on smithi078 2024-04-25T13:57:01.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:01.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.78:6838/2808782144,v1:172.21.15.78:6839/2808782144] up:boot 2024-04-25T13:57:01.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: fsmap 10 up:standby 2024-04-25T13:57:01.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:01 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:57:02.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:02 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3209680927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:02.462 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:02 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3209680927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3209680927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:02.719 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 13:57:02 smithi078 podman[70139]: 2024-04-25 13:57:02.604135282 +0000 UTC m=+0.255525354 container create 7ead8345f63bfed6c133acd66ed42557d27c044ac129e2948aab3b2e249a482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:57:03.114 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 13:57:02 smithi078 podman[70139]: 2024-04-25 13:57:02.848555925 +0000 UTC m=+0.499945955 container init 7ead8345f63bfed6c133acd66ed42557d27c044ac129e2948aab3b2e249a482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:57:03.114 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 13:57:02 smithi078 podman[70139]: 2024-04-25 13:57:02.851037153 +0000 UTC m=+0.502427191 container start 7ead8345f63bfed6c133acd66ed42557d27c044ac129e2948aab3b2e249a482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T13:57:03.114 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 13:57:02 smithi078 ceph-mds[70157]: -- 172.21.15.78:0/465329302 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f437e3a40 con 0x557f4382b600 2024-04-25T13:57:03.114 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 13:57:02 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a[70153]: starting mds.a at 2024-04-25T13:57:03.114 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 13:57:02 smithi078 bash[70139]: 7ead8345f63bfed6c133acd66ed42557d27c044ac129e2948aab3b2e249a482f 2024-04-25T13:57:03.115 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 13:57:02 smithi078 systemd[1]: Started Ceph mds.a for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:57:03.378 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:57:03.401 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:03 smithi078 ceph-mon[29163]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:03.401 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.401 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.401 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.401 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:57:03.402 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:57:03.402 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:03 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:03.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:03 smithi129 ceph-mon[32569]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:03.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:57:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:57:03.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:03 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:03.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 -- 172.21.15.78:0/4285592545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 msgr2=0x7fa9e8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:03.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/4285592545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e8101bc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009900 tx=0x7fa9dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:03.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 -- 172.21.15.78:0/4285592545 shutdown_connections 2024-04-25T13:57:03.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/4285592545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9e8102250 0x7fa9e810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/4285592545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e8101bc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/4285592545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e81050a0 0x7fa9e81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 -- 172.21.15.78:0/4285592545 >> 172.21.15.78:0/4285592545 conn(0x7fa9e80fd370 msgr2=0x7fa9e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 -- 172.21.15.78:0/4285592545 shutdown_connections 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 -- 172.21.15.78:0/4285592545 wait complete. 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.491+0000 7fa9edcc6640 1 Processor -- start 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9edcc6640 1 -- start start 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9edcc6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9edcc6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9e8102250 0x7fa9e819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9edcc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e81050a0 0x7fa9e81a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9edcc6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e810f610 con 0x7fa9e8101750 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9edcc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e810f490 con 0x7fa9e81050a0 2024-04-25T13:57:03.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9edcc6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e810f790 con 0x7fa9e8102250 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45232/0 (socket says 172.21.15.78:45232) 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 -- 172.21.15.78:0/2075870369 learned_addr learned my addr 172.21.15.78:0/2075870369 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e7fff640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e81050a0 0x7fa9e81a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e6ffd640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9e8102250 0x7fa9e819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 -- 172.21.15.78:0/2075870369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9e8102250 msgr2=0x7fa9e819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9e8102250 0x7fa9e819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 -- 172.21.15.78:0/2075870369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e81050a0 msgr2=0x7fa9e81a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e81050a0 0x7fa9e81a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 -- 172.21.15.78:0/2075870369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e81a1d20 con 0x7fa9e8101750 2024-04-25T13:57:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e6ffd640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9e8102250 0x7fa9e819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.492+0000 7fa9e77fe640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e819a1e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800e980 tx=0x7fa9d800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.493+0000 7fa9e4ff9640 1 -- 172.21.15.78:0/2075870369 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800cd80 con 0x7fa9e8101750 2024-04-25T13:57:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.493+0000 7fa9e4ff9640 1 -- 172.21.15.78:0/2075870369 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d80108e0 con 0x7fa9e8101750 2024-04-25T13:57:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.493+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e81a1fb0 con 0x7fa9e8101750 2024-04-25T13:57:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.493+0000 7fa9e4ff9640 1 -- 172.21.15.78:0/2075870369 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8018890 con 0x7fa9e8101750 2024-04-25T13:57:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.493+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e81a2470 con 0x7fa9e8101750 2024-04-25T13:57:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.494+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e8108a30 con 0x7fa9e8101750 2024-04-25T13:57:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.495+0000 7fa9e4ff9640 1 -- 172.21.15.78:0/2075870369 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa9d801f020 con 0x7fa9e8101750 2024-04-25T13:57:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.496+0000 7fa9e4ff9640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9d00afaf0 0x7fa9d00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.496+0000 7fa9e6ffd640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9d00afaf0 0x7fa9d00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.496+0000 7fa9e4ff9640 1 -- 172.21.15.78:0/2075870369 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fa9d80dee60 con 0x7fa9e8101750 2024-04-25T13:57:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.496+0000 7fa9e6ffd640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9d00afaf0 0x7fa9d00b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:03.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.499+0000 7fa9e4ff9640 1 -- 172.21.15.78:0/2075870369 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9d80a7ee0 con 0x7fa9e8101750 2024-04-25T13:57:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:03 smithi112 ceph-mon[32617]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:57:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:57:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:03 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:03.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.716+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9e802db50 con 0x7fa9e8101750 2024-04-25T13:57:03.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.717+0000 7fa9e4ff9640 1 -- 172.21.15.78:0/2075870369 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7fa9d80a77e0 con 0x7fa9e8101750 2024-04-25T13:57:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:03.717 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:57:03.717 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 12 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9d00afaf0 msgr2=0x7fa9d00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9d00afaf0 0x7fa9d00b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc005d50 comp rx=0 tx=0).stop 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 msgr2=0x7fa9e819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e819a1e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800e980 tx=0x7fa9d800ee40 comp rx=0 tx=0).stop 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 shutdown_connections 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e81050a0 0x7fa9e81a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9e8102250 0x7fa9e819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9d00afaf0 0x7fa9d00b2000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 --2- 172.21.15.78:0/2075870369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9e8101750 0x7fa9e819a1e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 >> 172.21.15.78:0/2075870369 conn(0x7fa9e80fd370 msgr2=0x7fa9e8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 shutdown_connections 2024-04-25T13:57:03.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:03.719+0000 7fa9edcc6640 1 -- 172.21.15.78:0/2075870369 wait complete. 2024-04-25T13:57:04.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:04 smithi112 ceph-mon[32617]: Deploying daemon mds.b on smithi112 2024-04-25T13:57:04.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:04 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.78:6840/1109968953,v1:172.21.15.78:6841/1109968953] up:boot 2024-04-25T13:57:04.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:04 smithi112 ceph-mon[32617]: fsmap 11 up:standby 2024-04-25T13:57:04.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:04 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:57:04.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:04 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2075870369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:04.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:04 smithi129 ceph-mon[32569]: Deploying daemon mds.b on smithi112 2024-04-25T13:57:04.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:04 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.78:6840/1109968953,v1:172.21.15.78:6841/1109968953] up:boot 2024-04-25T13:57:04.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:04 smithi129 ceph-mon[32569]: fsmap 11 up:standby 2024-04-25T13:57:04.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:04 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:57:04.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:04 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2075870369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:04 smithi078 ceph-mon[29163]: Deploying daemon mds.b on smithi112 2024-04-25T13:57:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:04 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.78:6840/1109968953,v1:172.21.15.78:6841/1109968953] up:boot 2024-04-25T13:57:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:04 smithi078 ceph-mon[29163]: fsmap 11 up:standby 2024-04-25T13:57:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:04 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:57:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:04 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2075870369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:05.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:05 smithi129 ceph-mon[32569]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:05.445 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:05 smithi112 ceph-mon[32617]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:05.446 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 systemd[1]: Starting Ceph mds.b for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T13:57:05.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:05 smithi078 ceph-mon[29163]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:05.715 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 podman[63601]: 2024-04-25 13:57:05.446614113 +0000 UTC m=+0.111786300 container create 832616ae2e58ed3ef979934d7b2c28f9a357fcd0443ecf520c9c3bb4496fb674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T13:57:05.715 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 podman[63601]: 2024-04-25 13:57:05.352391993 +0000 UTC m=+0.017564179 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T13:57:05.731 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:57:05.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 -- 172.21.15.78:0/1909997405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 msgr2=0x7fa230103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 --2- 172.21.15.78:0/1909997405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa230103dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa224009900 tx=0x7fa22402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 -- 172.21.15.78:0/1909997405 shutdown_connections 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 --2- 172.21.15.78:0/1909997405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa230104450 0x7fa23010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 --2- 172.21.15.78:0/1909997405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa230103dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 --2- 172.21.15.78:0/1909997405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa230102f30 0x7fa230103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 -- 172.21.15.78:0/1909997405 >> 172.21.15.78:0/1909997405 conn(0x7fa2300fd330 msgr2=0x7fa2300ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 -- 172.21.15.78:0/1909997405 shutdown_connections 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 -- 172.21.15.78:0/1909997405 wait complete. 2024-04-25T13:57:05.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 Processor -- start 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.851+0000 7fa2365f3640 1 -- start start 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa2365f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa230102f30 0x7fa2301021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa2365f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa23019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa2365f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa230104450 0x7fa2301a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa2365f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010e560 con 0x7fa230103910 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa2365f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010e3e0 con 0x7fa230104450 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa2365f3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010e6e0 con 0x7fa230102f30 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa23019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa23019f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45242/0 (socket says 172.21.15.78:45242) 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 -- 172.21.15.78:0/87293389 learned_addr learned my addr 172.21.15.78:0/87293389 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa234b90640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa230104450 0x7fa2301a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22ffff640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa230102f30 0x7fa2301021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 -- 172.21.15.78:0/87293389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa230102f30 msgr2=0x7fa2301021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa230102f30 0x7fa2301021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 -- 172.21.15.78:0/87293389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa230104450 msgr2=0x7fa2301a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa230104450 0x7fa2301a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 -- 172.21.15.78:0/87293389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2301a1db0 con 0x7fa230103910 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22ffff640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa230102f30 0x7fa2301021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.852+0000 7fa22f7fe640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa23019f040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa224006010 tx=0x7fa224004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.853+0000 7fa22d7fa640 1 -- 172.21.15.78:0/87293389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22403f030 con 0x7fa230103910 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.853+0000 7fa22d7fa640 1 -- 172.21.15.78:0/87293389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa224003cb0 con 0x7fa230103910 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.853+0000 7fa22d7fa640 1 -- 172.21.15.78:0/87293389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224035070 con 0x7fa230103910 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.853+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2301a2040 con 0x7fa230103910 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.853+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a2530 con 0x7fa230103910 2024-04-25T13:57:05.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.854+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2301078d0 con 0x7fa230103910 2024-04-25T13:57:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.855+0000 7fa22d7fa640 1 -- 172.21.15.78:0/87293389 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa224042030 con 0x7fa230103910 2024-04-25T13:57:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.855+0000 7fa22d7fa640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2180afaf0 0x7fa2180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:05.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.856+0000 7fa22ffff640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2180afaf0 0x7fa2180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.856+0000 7fa22d7fa640 1 -- 172.21.15.78:0/87293389 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fa2240ff490 con 0x7fa230103910 2024-04-25T13:57:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.856+0000 7fa22ffff640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2180afaf0 0x7fa2180b2000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa230069e00 tx=0x7fa220008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:05.859+0000 7fa22d7fa640 1 -- 172.21.15.78:0/87293389 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2240c8510 con 0x7fa230103910 2024-04-25T13:57:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.088+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa23002db50 con 0x7fa230103910 2024-04-25T13:57:06.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.089+0000 7fa22d7fa640 1 -- 172.21.15.78:0/87293389 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7fa2240c7e10 con 0x7fa230103910 2024-04-25T13:57:06.089 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:06.089 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:57:06.089 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 12 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2180afaf0 msgr2=0x7fa2180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2180afaf0 0x7fa2180b2000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa230069e00 tx=0x7fa220008040 comp rx=0 tx=0).stop 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 msgr2=0x7fa23019f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa23019f040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa224006010 tx=0x7fa224004340 comp rx=0 tx=0).stop 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 shutdown_connections 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa230104450 0x7fa2301a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2180afaf0 0x7fa2180b2000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa230103910 0x7fa23019f040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 --2- 172.21.15.78:0/87293389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa230102f30 0x7fa2301021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:06.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 >> 172.21.15.78:0/87293389 conn(0x7fa2300fd330 msgr2=0x7fa230104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:06.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 shutdown_connections 2024-04-25T13:57:06.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:06.091+0000 7fa2365f3640 1 -- 172.21.15.78:0/87293389 wait complete. 2024-04-25T13:57:06.266 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 podman[63601]: 2024-04-25 13:57:05.863925304 +0000 UTC m=+0.529097491 container init 832616ae2e58ed3ef979934d7b2c28f9a357fcd0443ecf520c9c3bb4496fb674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:57:06.266 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 podman[63601]: 2024-04-25 13:57:05.86655146 +0000 UTC m=+0.531723657 container start 832616ae2e58ed3ef979934d7b2c28f9a357fcd0443ecf520c9c3bb4496fb674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:57:06.266 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 ceph-mds[63620]: -- 172.21.15.112:0/1107635087 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff5b3b9a40 con 0x55ff5b401180 2024-04-25T13:57:06.266 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: starting mds.b at 2024-04-25T13:57:06.267 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 bash[63601]: 832616ae2e58ed3ef979934d7b2c28f9a357fcd0443ecf520c9c3bb4496fb674 2024-04-25T13:57:06.267 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:05 smithi112 systemd[1]: Started Ceph mds.b for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/87293389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] up:boot 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: fsmap 12 up:standby 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:57:07.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:07 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:07.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/87293389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] up:boot 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: fsmap 12 up:standby 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:57:07.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:07 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/87293389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] up:boot 2024-04-25T13:57:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: fsmap 12 up:standby 2024-04-25T13:57:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:57:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:07 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:08.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:57:08.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 -- 172.21.15.78:0/2137404319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c102a00 msgr2=0x7fd95c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 --2- 172.21.15.78:0/2137404319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c102a00 0x7fd95c103700 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009900 tx=0x7fd94c02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:08.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 -- 172.21.15.78:0/2137404319 shutdown_connections 2024-04-25T13:57:08.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 --2- 172.21.15.78:0/2137404319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd95c108940 0x7fd95c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 --2- 172.21.15.78:0/2137404319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd95c103c40 0x7fd95c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 --2- 172.21.15.78:0/2137404319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c102a00 0x7fd95c103700 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 -- 172.21.15.78:0/2137404319 >> 172.21.15.78:0/2137404319 conn(0x7fd95c0fdbf0 msgr2=0x7fd95c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 -- 172.21.15.78:0/2137404319 shutdown_connections 2024-04-25T13:57:08.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 -- 172.21.15.78:0/2137404319 wait complete. 2024-04-25T13:57:08.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 Processor -- start 2024-04-25T13:57:08.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.215+0000 7fd96280c640 1 -- start start 2024-04-25T13:57:08.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd96280c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd95c102a00 0x7fd95c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd96280c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd95c103c40 0x7fd95c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd96280c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c108940 0x7fd95c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd96280c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c10e4e0 con 0x7fd95c108940 2024-04-25T13:57:08.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd96280c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c10e360 con 0x7fd95c102a00 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd96280c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c10e660 con 0x7fd95c103c40 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c108940 0x7fd95c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c108940 0x7fd95c1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43714/0 (socket says 172.21.15.78:43714) 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 -- 172.21.15.78:0/1160878272 learned_addr learned my addr 172.21.15.78:0/1160878272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd95b7fe640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd95c103c40 0x7fd95c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd95bfff640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd95c102a00 0x7fd95c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 -- 172.21.15.78:0/1160878272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd95c103c40 msgr2=0x7fd95c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd95c103c40 0x7fd95c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 -- 172.21.15.78:0/1160878272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd95c102a00 msgr2=0x7fd95c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd95c102a00 0x7fd95c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 -- 172.21.15.78:0/1160878272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c1a1d50 con 0x7fd95c108940 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd95bfff640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd95c102a00 0x7fd95c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd960da9640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c108940 0x7fd95c1a16e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd95400eb10 tx=0x7fd95400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.216+0000 7fd9597fa640 1 -- 172.21.15.78:0/1160878272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954004b20 con 0x7fd95c108940 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.217+0000 7fd9597fa640 1 -- 172.21.15.78:0/1160878272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd954004cc0 con 0x7fd95c108940 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.217+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95c1a1fe0 con 0x7fd95c108940 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.217+0000 7fd9597fa640 1 -- 172.21.15.78:0/1160878272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954010670 con 0x7fd95c108940 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.217+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95c1a24a0 con 0x7fd95c108940 2024-04-25T13:57:08.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.218+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95c1a2760 con 0x7fd95c108940 2024-04-25T13:57:08.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.219+0000 7fd9597fa640 1 -- 172.21.15.78:0/1160878272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd954005000 con 0x7fd95c108940 2024-04-25T13:57:08.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.220+0000 7fd9597fa640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9440afb70 0x7fd9440b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.220+0000 7fd95bfff640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9440afb70 0x7fd9440b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.220+0000 7fd9597fa640 1 -- 172.21.15.78:0/1160878272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd9540dee90 con 0x7fd95c108940 2024-04-25T13:57:08.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.220+0000 7fd95bfff640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9440afb70 0x7fd9440b2080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd94c005f70 tx=0x7fd94c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.223+0000 7fd9597fa640 1 -- 172.21.15.78:0/1160878272 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd954053ff0 con 0x7fd95c108940 2024-04-25T13:57:08.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.437+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd95c1a2930 con 0x7fd95c108940 2024-04-25T13:57:08.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.438+0000 7fd9597fa640 1 -- 172.21.15.78:0/1160878272 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7fd9540a7f10 con 0x7fd95c108940 2024-04-25T13:57:08.438 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:08.438 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T13:57:08.438 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 13 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9440afb70 msgr2=0x7fd9440b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9440afb70 0x7fd9440b2080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd94c005f70 tx=0x7fd94c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c108940 msgr2=0x7fd95c1a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c108940 0x7fd95c1a16e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd95400eb10 tx=0x7fd95400c2d0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 shutdown_connections 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9440afb70 0x7fd9440b2080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd95c108940 0x7fd95c1a16e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd95c103c40 0x7fd95c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 --2- 172.21.15.78:0/1160878272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd95c102a00 0x7fd95c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 >> 172.21.15.78:0/1160878272 conn(0x7fd95c0fdbf0 msgr2=0x7fd95c109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 shutdown_connections 2024-04-25T13:57:08.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.440+0000 7fd96280c640 1 -- 172.21.15.78:0/1160878272 wait complete. 2024-04-25T13:57:08.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:57:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 -- 172.21.15.78:0/2363257341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514108940 msgr2=0x7f951410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 --2- 172.21.15.78:0/2363257341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514108940 0x7f951410ad60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009900 tx=0x7f94fc02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 -- 172.21.15.78:0/2363257341 shutdown_connections 2024-04-25T13:57:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 --2- 172.21.15.78:0/2363257341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514108940 0x7f951410ad60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 --2- 172.21.15.78:0/2363257341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9514103c40 0x7f9514104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 --2- 172.21.15.78:0/2363257341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9514102a00 0x7f9514103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 -- 172.21.15.78:0/2363257341 >> 172.21.15.78:0/2363257341 conn(0x7f95140fdbf0 msgr2=0x7f95141000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 -- 172.21.15.78:0/2363257341 shutdown_connections 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 -- 172.21.15.78:0/2363257341 wait complete. 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 Processor -- start 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.561+0000 7f951ab34640 1 -- start start 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f951ab34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9514102a00 0x7f951419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f951ab34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514103c40 0x7f951419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f951ab34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9514108940 0x7f95141a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f951ab34640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f951410f690 con 0x7f9514103c40 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f951ab34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f951410f510 con 0x7f9514108940 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f951ab34640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f951410f810 con 0x7f9514102a00 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514103c40 0x7f951419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514103c40 0x7f951419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43722/0 (socket says 172.21.15.78:43722) 2024-04-25T13:57:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 -- 172.21.15.78:0/2667180437 learned_addr learned my addr 172.21.15.78:0/2667180437 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 -- 172.21.15.78:0/2667180437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9514102a00 msgr2=0x7f951419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9514102a00 0x7f951419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 -- 172.21.15.78:0/2667180437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9514108940 msgr2=0x7f95141a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9514108940 0x7f95141a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 -- 172.21.15.78:0/2667180437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95141a1d10 con 0x7f9514103c40 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9513fff640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514103c40 0x7f951419a710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f950400ba50 tx=0x7f950400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9511ffb640 1 -- 172.21.15.78:0/2667180437 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504019070 con 0x7f9514103c40 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9511ffb640 1 -- 172.21.15.78:0/2667180437 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f950400cbe0 con 0x7f9514103c40 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95141a1fa0 con 0x7f9514103c40 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.562+0000 7f9511ffb640 1 -- 172.21.15.78:0/2667180437 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95040075d0 con 0x7f9514103c40 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.563+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95141a2440 con 0x7f9514103c40 2024-04-25T13:57:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.563+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9514107750 con 0x7f9514103c40 2024-04-25T13:57:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.565+0000 7f9511ffb640 1 -- 172.21.15.78:0/2667180437 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f950401c070 con 0x7f9514103c40 2024-04-25T13:57:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.565+0000 7f9511ffb640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95000afaf0 0x7f95000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.565+0000 7f95188d0640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95000afaf0 0x7f95000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.565+0000 7f9511ffb640 1 -- 172.21.15.78:0/2667180437 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f95040dde40 con 0x7f9514103c40 2024-04-25T13:57:08.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.566+0000 7f95188d0640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95000afaf0 0x7f95000b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9514103600 tx=0x7f950c005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.570+0000 7f9511ffb640 1 -- 172.21.15.78:0/2667180437 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95040a6ec0 con 0x7f9514103c40 2024-04-25T13:57:08.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.774+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f951402dd70 con 0x7f9514103c40 2024-04-25T13:57:08.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.776+0000 7f9511ffb640 1 -- 172.21.15.78:0/2667180437 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18480 (secure 0 0 0) 0x7f95040a67c0 con 0x7f9514103c40 2024-04-25T13:57:08.776 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:08.776 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:56:22.314977+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:57:08.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95000afaf0 msgr2=0x7f95000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95000afaf0 0x7f95000b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9514103600 tx=0x7f950c005ed0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514103c40 msgr2=0x7f951419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514103c40 0x7f951419a710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f950400ba50 tx=0x7f950400bf10 comp rx=0 tx=0).stop 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 shutdown_connections 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9514108940 0x7f95141a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95000afaf0 0x7f95000b2000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9514103c40 0x7f951419a710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 --2- 172.21.15.78:0/2667180437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9514102a00 0x7f951419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 >> 172.21.15.78:0/2667180437 conn(0x7f95140fdbf0 msgr2=0x7f9514104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 shutdown_connections 2024-04-25T13:57:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.778+0000 7f951ab34640 1 -- 172.21.15.78:0/2667180437 wait complete. 2024-04-25T13:57:08.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:57:08.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:57:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.906+0000 7fef4beaf640 1 -- 172.21.15.78:0/2882933063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 msgr2=0x7fef44103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.906+0000 7fef4beaf640 1 --2- 172.21.15.78:0/2882933063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef44103de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fef34009900 tx=0x7fef3402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.906+0000 7fef4beaf640 1 -- 172.21.15.78:0/2882933063 shutdown_connections 2024-04-25T13:57:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.906+0000 7fef4beaf640 1 --2- 172.21.15.78:0/2882933063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44104470 0x7fef4410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.906+0000 7fef4beaf640 1 --2- 172.21.15.78:0/2882933063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef44103de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.906+0000 7fef4beaf640 1 --2- 172.21.15.78:0/2882933063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef44102f50 0x7fef44103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.906+0000 7fef4beaf640 1 -- 172.21.15.78:0/2882933063 >> 172.21.15.78:0/2882933063 conn(0x7fef440fd350 msgr2=0x7fef440ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 -- 172.21.15.78:0/2882933063 shutdown_connections 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 -- 172.21.15.78:0/2882933063 wait complete. 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 Processor -- start 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 -- start start 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef44102f50 0x7fef44102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef4419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44104470 0x7fef441a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410e540 con 0x7fef44103930 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410e3c0 con 0x7fef44104470 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.907+0000 7fef4beaf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410e6c0 con 0x7fef44102f50 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4944a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef4419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4944a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef4419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43744/0 (socket says 172.21.15.78:43744) 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4944a640 1 -- 172.21.15.78:0/765739647 learned_addr learned my addr 172.21.15.78:0/765739647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4a44c640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44104470 0x7fef441a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef49c4b640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef44102f50 0x7fef44102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef49c4b640 1 -- 172.21.15.78:0/765739647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44104470 msgr2=0x7fef441a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef49c4b640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44104470 0x7fef441a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef49c4b640 1 -- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 msgr2=0x7fef4419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef49c4b640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef4419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef49c4b640 1 -- 172.21.15.78:0/765739647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef441a1d70 con 0x7fef44102f50 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4a44c640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44104470 0x7fef441a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4944a640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef4419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef49c4b640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef44102f50 0x7fef44102210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef4406b2f0 tx=0x7fef3c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef3affd640 1 -- 172.21.15.78:0/765739647 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c004b20 con 0x7fef44102f50 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef3affd640 1 -- 172.21.15.78:0/765739647 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef3c004cc0 con 0x7fef44102f50 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef441a2060 con 0x7fef44102f50 2024-04-25T13:57:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef3affd640 1 -- 172.21.15.78:0/765739647 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c010690 con 0x7fef44102f50 2024-04-25T13:57:08.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.908+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef441a2560 con 0x7fef44102f50 2024-04-25T13:57:08.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.909+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef441078f0 con 0x7fef44102f50 2024-04-25T13:57:08.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.911+0000 7fef3affd640 1 -- 172.21.15.78:0/765739647 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fef3c005000 con 0x7fef44102f50 2024-04-25T13:57:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.911+0000 7fef3affd640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fef300afaf0 0x7fef300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.912+0000 7fef4944a640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fef300afaf0 0x7fef300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.912+0000 7fef3affd640 1 -- 172.21.15.78:0/765739647 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fef3c0dee80 con 0x7fef44102f50 2024-04-25T13:57:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.912+0000 7fef4944a640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fef300afaf0 0x7fef300b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef34031440 tx=0x7fef340314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:08.915+0000 7fef3affd640 1 -- 172.21.15.78:0/765739647 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef3c0a7f00 con 0x7fef44102f50 2024-04-25T13:57:09.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.101+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef4402ddc0 con 0x7fef44102f50 2024-04-25T13:57:09.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.101+0000 7fef3affd640 1 -- 172.21.15.78:0/765739647 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fef3c0a7800 con 0x7fef44102f50 2024-04-25T13:57:09.101 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:57:09.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fef300afaf0 msgr2=0x7fef300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fef300afaf0 0x7fef300b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef34031440 tx=0x7fef340314a0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef44102f50 msgr2=0x7fef44102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef44102f50 0x7fef44102210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef4406b2f0 tx=0x7fef3c00ef90 comp rx=0 tx=0).stop 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 shutdown_connections 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef44104470 0x7fef441a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fef300afaf0 0x7fef300b2000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef44103930 0x7fef4419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 --2- 172.21.15.78:0/765739647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef44102f50 0x7fef44102210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 >> 172.21.15.78:0/765739647 conn(0x7fef440fd350 msgr2=0x7fef44104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 shutdown_connections 2024-04-25T13:57:09.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.103+0000 7fef4beaf640 1 -- 172.21.15.78:0/765739647 wait complete. 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:09.115 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1160878272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.116 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:09.116 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:09.116 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.116 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.116 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:08 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2667180437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.117 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:57:09.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1160878272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:08 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2667180437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 -- 172.21.15.78:0/4074370498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7538102160 msgr2=0x7f7538102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 --2- 172.21.15.78:0/4074370498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7538102160 0x7f7538102e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f753400aa10 tx=0x7f753402b720 comp rx=0 tx=0).stop 2024-04-25T13:57:09.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 -- 172.21.15.78:0/4074370498 shutdown_connections 2024-04-25T13:57:09.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 --2- 172.21.15.78:0/4074370498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75381080a0 0x7f753810a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 --2- 172.21.15.78:0/4074370498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75381033a0 0x7f75381037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 --2- 172.21.15.78:0/4074370498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7538102160 0x7f7538102e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 -- 172.21.15.78:0/4074370498 >> 172.21.15.78:0/4074370498 conn(0x7f75380fd330 msgr2=0x7f75380ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:09.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 -- 172.21.15.78:0/4074370498 shutdown_connections 2024-04-25T13:57:09.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.228+0000 7f753f3a8640 1 -- 172.21.15.78:0/4074370498 wait complete. 2024-04-25T13:57:09.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 Processor -- start 2024-04-25T13:57:09.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 -- start start 2024-04-25T13:57:09.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538102160 0x7f7538198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75381033a0 0x7f7538198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75381080a0 0x7f753819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f753810d3f0 con 0x7f75381080a0 2024-04-25T13:57:09.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f753810d270 con 0x7f75381033a0 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753f3a8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f753810d570 con 0x7f7538102160 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753d945640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75381080a0 0x7f753819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753d945640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75381080a0 0x7f753819f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43766/0 (socket says 172.21.15.78:43766) 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753c943640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75381033a0 0x7f7538198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.229+0000 7f753d945640 1 -- 172.21.15.78:0/3996051949 learned_addr learned my addr 172.21.15.78:0/3996051949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753d144640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538102160 0x7f7538198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753d945640 1 -- 172.21.15.78:0/3996051949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538102160 msgr2=0x7f7538198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753d945640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538102160 0x7f7538198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753d945640 1 -- 172.21.15.78:0/3996051949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75381033a0 msgr2=0x7f7538198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753d945640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75381033a0 0x7f7538198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753d945640 1 -- 172.21.15.78:0/3996051949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753819fb60 con 0x7f75381080a0 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753c943640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75381033a0 0x7f7538198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753d945640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75381080a0 0x7f753819f4f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f753000eaa0 tx=0x7f753000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f752e7fc640 1 -- 172.21.15.78:0/3996051949 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753000cdb0 con 0x7f75381080a0 2024-04-25T13:57:09.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f752e7fc640 1 -- 172.21.15.78:0/3996051949 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7530005020 con 0x7f75381080a0 2024-04-25T13:57:09.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753819fe50 con 0x7f75381080a0 2024-04-25T13:57:09.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f752e7fc640 1 -- 172.21.15.78:0/3996051949 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75300116d0 con 0x7f75381080a0 2024-04-25T13:57:09.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.230+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381a0260 con 0x7f75381080a0 2024-04-25T13:57:09.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.232+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7538106eb0 con 0x7f75381080a0 2024-04-25T13:57:09.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.233+0000 7f752e7fc640 1 -- 172.21.15.78:0/3996051949 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f75300051d0 con 0x7f75381080a0 2024-04-25T13:57:09.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.233+0000 7f752e7fc640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75240afaf0 0x7f75240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.234+0000 7f753d144640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75240afaf0 0x7f75240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.234+0000 7f752e7fc640 1 -- 172.21.15.78:0/3996051949 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f75300de870 con 0x7f75381080a0 2024-04-25T13:57:09.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.234+0000 7f753d144640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75240afaf0 0x7f75240b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7534005ab0 tx=0x7f7534030510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:09.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.238+0000 7f752e7fc640 1 -- 172.21.15.78:0/3996051949 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7530016030 con 0x7f75381080a0 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1160878272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:09.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:09.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:09.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.267 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:08 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2667180437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.442+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f753802dd20 con 0x7f75381080a0 2024-04-25T13:57:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.442+0000 7f752e7fc640 1 -- 172.21.15.78:0/3996051949 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f75300a72a0 con 0x7f75381080a0 2024-04-25T13:57:09.442 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:09.442 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-04-25T13:57:09.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T13:57:09.443 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-04-25T13:57:09.443 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-04-25T13:57:09.443 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T13:57:09.443 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-04-25T13:57:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75240afaf0 msgr2=0x7f75240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75240afaf0 0x7f75240b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7534005ab0 tx=0x7f7534030510 comp rx=0 tx=0).stop 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75381080a0 msgr2=0x7f753819f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75381080a0 0x7f753819f4f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f753000eaa0 tx=0x7f753000ef60 comp rx=0 tx=0).stop 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 shutdown_connections 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75240afaf0 0x7f75240b2000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75381080a0 0x7f753819f4f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75381033a0 0x7f7538198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 --2- 172.21.15.78:0/3996051949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538102160 0x7f7538198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.444+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 >> 172.21.15.78:0/3996051949 conn(0x7f75380fd330 msgr2=0x7f7538104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.445+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 shutdown_connections 2024-04-25T13:57:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.445+0000 7f753f3a8640 1 -- 172.21.15.78:0/3996051949 wait complete. 2024-04-25T13:57:09.456 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:57:09.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:57:09.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 -- 172.21.15.78:0/1927553555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4109a30 msgr2=0x7f12b410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 --2- 172.21.15.78:0/1927553555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4109a30 0x7f12b410be50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009ca0 tx=0x7f12ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 -- 172.21.15.78:0/1927553555 shutdown_connections 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 --2- 172.21.15.78:0/1927553555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4109a30 0x7f12b410be50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 --2- 172.21.15.78:0/1927553555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b4104d30 0x7f12b4105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 --2- 172.21.15.78:0/1927553555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b4103af0 0x7f12b41047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 -- 172.21.15.78:0/1927553555 >> 172.21.15.78:0/1927553555 conn(0x7f12b40fece0 msgr2=0x7f12b4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 -- 172.21.15.78:0/1927553555 shutdown_connections 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.567+0000 7f12b9253640 1 -- 172.21.15.78:0/1927553555 wait complete. 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 Processor -- start 2024-04-25T13:57:09.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 -- start start 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4103af0 0x7f12b4077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b4104d30 0x7f12b4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b4109a30 0x7f12b4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12b410f620 con 0x7f12b4103af0 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12b410f4a0 con 0x7f12b4109a30 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b9253640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f12b410f7a0 con 0x7f12b4104d30 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4103af0 0x7f12b4077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.568+0000 7f12b2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4103af0 0x7f12b4077960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43792/0 (socket says 172.21.15.78:43792) 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b2d9d640 1 -- 172.21.15.78:0/2996775197 learned_addr learned my addr 172.21.15.78:0/2996775197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b359e640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b4109a30 0x7f12b4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12abfff640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b4104d30 0x7f12b4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b2d9d640 1 -- 172.21.15.78:0/2996775197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b4104d30 msgr2=0x7f12b4076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b2d9d640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b4104d30 0x7f12b4076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b2d9d640 1 -- 172.21.15.78:0/2996775197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b4109a30 msgr2=0x7f12b4076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b2d9d640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b4109a30 0x7f12b4076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b2d9d640 1 -- 172.21.15.78:0/2996775197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b4076c70 con 0x7f12b4103af0 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b2d9d640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4103af0 0x7f12b4077960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12a400e9f0 tx=0x7f12a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b0d99640 1 -- 172.21.15.78:0/2996775197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a400ccc0 con 0x7f12b4103af0 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b0d99640 1 -- 172.21.15.78:0/2996775197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a40035f0 con 0x7f12b4103af0 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b41a4290 con 0x7f12b4103af0 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b0d99640 1 -- 172.21.15.78:0/2996775197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a400ce60 con 0x7f12b4103af0 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.569+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a4750 con 0x7f12b4103af0 2024-04-25T13:57:09.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.570+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b4108840 con 0x7f12b4103af0 2024-04-25T13:57:09.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.571+0000 7f12b0d99640 1 -- 172.21.15.78:0/2996775197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f12a4003aa0 con 0x7f12b4103af0 2024-04-25T13:57:09.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.572+0000 7f12b0d99640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f12a00afaf0 0x7f12a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:09.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.572+0000 7f12abfff640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f12a00afaf0 0x7f12a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:09.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.572+0000 7f12b0d99640 1 -- 172.21.15.78:0/2996775197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f12a40deb20 con 0x7f12b4103af0 2024-04-25T13:57:09.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.573+0000 7f12abfff640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f12a00afaf0 0x7f12a00b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12b4077300 tx=0x7f129c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:09.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.576+0000 7f12b0d99640 1 -- 172.21.15.78:0/2996775197 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12a40a7ba0 con 0x7f12b4103af0 2024-04-25T13:57:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:09.761+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f12b402dcd0 con 0x7f12b4103af0 2024-04-25T13:57:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.114+0000 7f12b0d99640 1 -- 172.21.15.78:0/2996775197 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f12a40a74a0 con 0x7f12b4103af0 2024-04-25T13:57:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-04-25T13:57:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f12a00afaf0 msgr2=0x7f12a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f12a00afaf0 0x7f12a00b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12b4077300 tx=0x7f129c005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4103af0 msgr2=0x7f12b4077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4103af0 0x7f12b4077960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12a400e9f0 tx=0x7f12a400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 shutdown_connections 2024-04-25T13:57:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12b4109a30 0x7f12b4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b4104d30 0x7f12b4076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f12a00afaf0 0x7f12a00b2000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 --2- 172.21.15.78:0/2996775197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b4103af0 0x7f12b4077960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 >> 172.21.15.78:0/2996775197 conn(0x7f12b40fece0 msgr2=0x7f12b4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:10.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 shutdown_connections 2024-04-25T13:57:10.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.118+0000 7f12b9253640 1 -- 172.21.15.78:0/2996775197 wait complete. 2024-04-25T13:57:10.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/765739647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:57:10.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3996051949' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:57:10.132 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2996775197' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:57:10.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:57:10.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/765739647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:57:10.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3996051949' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:57:10.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2996775197' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:57:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.255+0000 7f1e60bc0640 1 -- 172.21.15.78:0/4009507362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 msgr2=0x7f1e5c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.255+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/4009507362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c102230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009900 tx=0x7f1e4c02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 -- 172.21.15.78:0/4009507362 shutdown_connections 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/4009507362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e5c109770 0x7f1e5c10bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/4009507362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c106c00 0x7f1e5c1090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/4009507362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c102230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 -- 172.21.15.78:0/4009507362 >> 172.21.15.78:0/4009507362 conn(0x7f1e5c0fd330 msgr2=0x7f1e5c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 -- 172.21.15.78:0/4009507362 shutdown_connections 2024-04-25T13:57:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 -- 172.21.15.78:0/4009507362 wait complete. 2024-04-25T13:57:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 Processor -- start 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.256+0000 7f1e60bc0640 1 -- start start 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e60bc0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e60bc0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c106c00 0x7f1e5c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e60bc0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e5c109770 0x7f1e5c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e60bc0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c10f5b0 con 0x7f1e5c101e20 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e60bc0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c10f430 con 0x7f1e5c106c00 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e60bc0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c10f730 con 0x7f1e5c109770 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43814/0 (socket says 172.21.15.78:43814) 2024-04-25T13:57:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 -- 172.21.15.78:0/1085392719 learned_addr learned my addr 172.21.15.78:0/1085392719 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e59d9b640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c106c00 0x7f1e5c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5ad9d640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e5c109770 0x7f1e5c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 -- 172.21.15.78:0/1085392719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e5c109770 msgr2=0x7f1e5c1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e5c109770 0x7f1e5c1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 -- 172.21.15.78:0/1085392719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c106c00 msgr2=0x7f1e5c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c106c00 0x7f1e5c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 -- 172.21.15.78:0/1085392719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c1a1d30 con 0x7f1e5c101e20 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5ad9d640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e5c109770 0x7f1e5c1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.257+0000 7f1e5a59c640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c19a1f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006010 tx=0x7f1e4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.258+0000 7f1e4b7fe640 1 -- 172.21.15.78:0/1085392719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c03f030 con 0x7f1e5c101e20 2024-04-25T13:57:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.258+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1a1fc0 con 0x7f1e5c101e20 2024-04-25T13:57:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.258+0000 7f1e4b7fe640 1 -- 172.21.15.78:0/1085392719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4c003aa0 con 0x7f1e5c101e20 2024-04-25T13:57:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.258+0000 7f1e4b7fe640 1 -- 172.21.15.78:0/1085392719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c035070 con 0x7f1e5c101e20 2024-04-25T13:57:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.258+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c1a2420 con 0x7f1e5c101e20 2024-04-25T13:57:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.259+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c0fee50 con 0x7f1e5c101e20 2024-04-25T13:57:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.261+0000 7f1e4b7fe640 1 -- 172.21.15.78:0/1085392719 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1e4c042030 con 0x7f1e5c101e20 2024-04-25T13:57:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.261+0000 7f1e4b7fe640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e440afaf0 0x7f1e440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.261+0000 7f1e59d9b640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e440afaf0 0x7f1e440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.261+0000 7f1e4b7fe640 1 -- 172.21.15.78:0/1085392719 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7f1e4c0ff380 con 0x7f1e5c101e20 2024-04-25T13:57:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.262+0000 7f1e59d9b640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e440afaf0 0x7f1e440b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c19b290 tx=0x7f1e50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.265+0000 7f1e4b7fe640 1 -- 172.21.15.78:0/1085392719 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e4c0c8290 con 0x7f1e5c101e20 2024-04-25T13:57:10.265 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/765739647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:57:10.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3996051949' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:57:10.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2996775197' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:57:10.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:10.458+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1e5c02db50 con 0x7f1e5c101e20 2024-04-25T13:57:11.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.120+0000 7f1e4b7fe640 1 -- 172.21.15.78:0/1085392719 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f1e4c0c7b90 con 0x7f1e5c101e20 2024-04-25T13:57:11.120 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e440afaf0 msgr2=0x7f1e440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e440afaf0 0x7f1e440b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c19b290 tx=0x7f1e50005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 msgr2=0x7f1e5c19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c19a1f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006010 tx=0x7f1e4c004270 comp rx=0 tx=0).stop 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 shutdown_connections 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e5c109770 0x7f1e5c1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e5c106c00 0x7f1e5c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e440afaf0 0x7f1e440b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 --2- 172.21.15.78:0/1085392719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e5c101e20 0x7f1e5c19a1f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 >> 172.21.15.78:0/1085392719 conn(0x7f1e5c0fd330 msgr2=0x7f1e5c107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 shutdown_connections 2024-04-25T13:57:11.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.123+0000 7f1e60bc0640 1 -- 172.21.15.78:0/1085392719 wait complete. 2024-04-25T13:57:11.143 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:57:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 -- 172.21.15.78:0/194966787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62581029e0 msgr2=0x7f62581036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:11.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 --2- 172.21.15.78:0/194966787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62581029e0 0x7f62581036e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6254009900 tx=0x7f625402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:11.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 -- 172.21.15.78:0/194966787 shutdown_connections 2024-04-25T13:57:11.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 --2- 172.21.15.78:0/194966787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6258108920 0x7f625810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 --2- 172.21.15.78:0/194966787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6258103c20 0x7f6258104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 --2- 172.21.15.78:0/194966787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62581029e0 0x7f62581036e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 -- 172.21.15.78:0/194966787 >> 172.21.15.78:0/194966787 conn(0x7f62580fdbd0 msgr2=0x7f6258100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:11.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 -- 172.21.15.78:0/194966787 shutdown_connections 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 -- 172.21.15.78:0/194966787 wait complete. 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 Processor -- start 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.255+0000 7f625f420640 1 -- start start 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62581029e0 0x7f625819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6258103c20 0x7f625819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6258108920 0x7f62581a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f625810e4e0 con 0x7f6258108920 2024-04-25T13:57:11.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f625810e360 con 0x7f62581029e0 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f625810e660 con 0x7f6258103c20 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6258108920 0x7f62581a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6258108920 0x7f62581a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43828/0 (socket says 172.21.15.78:43828) 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 -- 172.21.15.78:0/2379449510 learned_addr learned my addr 172.21.15.78:0/2379449510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625c9bb640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6258103c20 0x7f625819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d1bc640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62581029e0 0x7f625819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 -- 172.21.15.78:0/2379449510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6258103c20 msgr2=0x7f625819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:11.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6258103c20 0x7f625819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 -- 172.21.15.78:0/2379449510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62581029e0 msgr2=0x7f625819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62581029e0 0x7f625819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 -- 172.21.15.78:0/2379449510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62581a2070 con 0x7f6258108920 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625d9bd640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6258108920 0x7f62581a1a00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f625000eb10 tx=0x7f625000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f624e7fc640 1 -- 172.21.15.78:0/2379449510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250004b20 con 0x7f6258108920 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f624e7fc640 1 -- 172.21.15.78:0/2379449510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6250004cc0 con 0x7f6258108920 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62581a2360 con 0x7f6258108920 2024-04-25T13:57:11.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f624e7fc640 1 -- 172.21.15.78:0/2379449510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250010670 con 0x7f6258108920 2024-04-25T13:57:11.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.256+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62581a2900 con 0x7f6258108920 2024-04-25T13:57:11.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.257+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f625806c2a0 con 0x7f6258108920 2024-04-25T13:57:11.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.259+0000 7f624e7fc640 1 -- 172.21.15.78:0/2379449510 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6250005000 con 0x7f6258108920 2024-04-25T13:57:11.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.259+0000 7f624e7fc640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62440afb70 0x7f62440b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:11.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.260+0000 7f625d1bc640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62440afb70 0x7f62440b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:11.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.260+0000 7f624e7fc640 1 -- 172.21.15.78:0/2379449510 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7879+0+0 (secure 0 0 0) 0x7f62500df120 con 0x7f6258108920 2024-04-25T13:57:11.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.260+0000 7f625d1bc640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62440afb70 0x7f62440b2080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6254005ab0 tx=0x7f6254031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:11.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.263+0000 7f624e7fc640 1 -- 172.21.15.78:0/2379449510 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6250053ff0 con 0x7f6258108920 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2996775197' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: pgmap v273: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1085392719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.408 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.409 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.409 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:11.409 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:11.409 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2996775197' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:57:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T13:57:11.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: pgmap v273: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:11.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1085392719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:57:11.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:11.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:11.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:11.482+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f62581a2c30 con 0x7f6258108920 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2996775197' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: pgmap v273: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1085392719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:11.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:57:11.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:57:11.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:57:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.122+0000 7f624e7fc640 1 -- 172.21.15.78:0/2379449510 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f62500a7eb0 con 0x7f6258108920 2024-04-25T13:57:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:57:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62440afb70 msgr2=0x7f62440b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62440afb70 0x7f62440b2080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6254005ab0 tx=0x7f6254031400 comp rx=0 tx=0).stop 2024-04-25T13:57:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6258108920 msgr2=0x7f62581a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6258108920 0x7f62581a1a00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f625000eb10 tx=0x7f625000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 shutdown_connections 2024-04-25T13:57:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62440afb70 0x7f62440b2080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6258108920 0x7f62581a1a00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6258103c20 0x7f625819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 --2- 172.21.15.78:0/2379449510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62581029e0 0x7f625819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.127+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 >> 172.21.15.78:0/2379449510 conn(0x7f62580fdbd0 msgr2=0x7f6258069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.128+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 shutdown_connections 2024-04-25T13:57:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.129+0000 7f625f420640 1 -- 172.21.15.78:0/2379449510 wait complete. 2024-04-25T13:57:12.153 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.264+0000 7fdc6bf64640 1 -- 172.21.15.78:0/520429352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc641029c0 msgr2=0x7fdc641036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.264+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/520429352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc641029c0 0x7fdc641036c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- 172.21.15.78:0/520429352 shutdown_connections 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/520429352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc64108900 0x7fdc6410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/520429352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc64103c00 0x7fdc64104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/520429352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc641029c0 0x7fdc641036c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- 172.21.15.78:0/520429352 >> 172.21.15.78:0/520429352 conn(0x7fdc640fdbd0 msgr2=0x7fdc64100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- 172.21.15.78:0/520429352 shutdown_connections 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- 172.21.15.78:0/520429352 wait complete. 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 Processor -- start 2024-04-25T13:57:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- start start 2024-04-25T13:57:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc641029c0 0x7fdc6406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc64103c00 0x7fdc6419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64108900 0x7fdc641a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6410e4c0 con 0x7fdc64108900 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6410e340 con 0x7fdc64103c00 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.265+0000 7fdc6bf64640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6410e640 con 0x7fdc641029c0 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64108900 0x7fdc641a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64108900 0x7fdc641a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43834/0 (socket says 172.21.15.78:43834) 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 -- 172.21.15.78:0/1377952521 learned_addr learned my addr 172.21.15.78:0/1377952521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc694ff640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc64103c00 0x7fdc6419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 -- 172.21.15.78:0/1377952521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc641029c0 msgr2=0x7fdc6406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc69d00640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc641029c0 0x7fdc6406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc641029c0 0x7fdc6406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 -- 172.21.15.78:0/1377952521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc64103c00 msgr2=0x7fdc6419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc64103c00 0x7fdc6419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 -- 172.21.15.78:0/1377952521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc641a1d40 con 0x7fdc64108900 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6a501640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64108900 0x7fdc641a1520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00eb10 tx=0x7fdc5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc62ffd640 1 -- 172.21.15.78:0/1377952521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c004b20 con 0x7fdc64108900 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc62ffd640 1 -- 172.21.15.78:0/1377952521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c004cc0 con 0x7fdc64108900 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc641a1fd0 con 0x7fdc64108900 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc62ffd640 1 -- 172.21.15.78:0/1377952521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c010670 con 0x7fdc64108900 2024-04-25T13:57:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.266+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641a2490 con 0x7fdc64108900 2024-04-25T13:57:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.267+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc64107710 con 0x7fdc64108900 2024-04-25T13:57:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.268+0000 7fdc62ffd640 1 -- 172.21.15.78:0/1377952521 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fdc5c005020 con 0x7fdc64108900 2024-04-25T13:57:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.269+0000 7fdc62ffd640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc4c0afaf0 0x7fdc4c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.269+0000 7fdc69d00640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc4c0afaf0 0x7fdc4c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.269+0000 7fdc62ffd640 1 -- 172.21.15.78:0/1377952521 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7fdc5c0df0f0 con 0x7fdc64108900 2024-04-25T13:57:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.270+0000 7fdc69d00640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc4c0afaf0 0x7fdc4c0b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc5402a930 tx=0x7fdc54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:12.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.278+0000 7fdc62ffd640 1 -- 172.21.15.78:0/1377952521 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc5c0a7e20 con 0x7fdc64108900 2024-04-25T13:57:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:12 smithi129 ceph-mon[32569]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:57:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1085392719' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:57:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:12 smithi129 ceph-mon[32569]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T13:57:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2379449510' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:57:12.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:12 smithi129 ceph-mon[32569]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:57:12.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.465+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc6402dd70 con 0x7fdc64108900 2024-04-25T13:57:12.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.467+0000 7fdc62ffd640 1 -- 172.21.15.78:0/1377952521 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22256 (secure 0 0 0) 0x7fdc5c0a7720 con 0x7fdc64108900 2024-04-25T13:57:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:12.467 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":74,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:57:11.484387+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc4c0afaf0 msgr2=0x7fdc4c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc4c0afaf0 0x7fdc4c0b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc5402a930 tx=0x7fdc54005fd0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64108900 msgr2=0x7fdc641a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64108900 0x7fdc641a1520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00eb10 tx=0x7fdc5c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 shutdown_connections 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc4c0afaf0 0x7fdc4c0b2000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64108900 0x7fdc641a1520 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc64103c00 0x7fdc6419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 --2- 172.21.15.78:0/1377952521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc641029c0 0x7fdc6406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 >> 172.21.15.78:0/1377952521 conn(0x7fdc640fdbd0 msgr2=0x7fdc641048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 shutdown_connections 2024-04-25T13:57:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.469+0000 7fdc6bf64640 1 -- 172.21.15.78:0/1377952521 wait complete. 2024-04-25T13:57:12.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:12 smithi078 ceph-mon[29163]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:57:12.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1085392719' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:57:12.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:12 smithi078 ceph-mon[29163]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T13:57:12.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2379449510' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:57:12.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:12 smithi078 ceph-mon[29163]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:57:12.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:57:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:12 smithi112 ceph-mon[32617]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:57:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1085392719' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:57:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:12 smithi112 ceph-mon[32617]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T13:57:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2379449510' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:57:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:12 smithi112 ceph-mon[32617]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:57:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 -- 172.21.15.78:0/2882533705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844109790 msgr2=0x7f084410bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 --2- 172.21.15.78:0/2882533705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844109790 0x7f084410bca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f083c009ca0 tx=0x7f083c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 -- 172.21.15.78:0/2882533705 shutdown_connections 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 --2- 172.21.15.78:0/2882533705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844109790 0x7f084410bca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 --2- 172.21.15.78:0/2882533705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0844106c20 0x7f0844109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 --2- 172.21.15.78:0/2882533705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0844101e40 0x7f0844102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 -- 172.21.15.78:0/2882533705 >> 172.21.15.78:0/2882533705 conn(0x7f08440fd3b0 msgr2=0x7f08440ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 -- 172.21.15.78:0/2882533705 shutdown_connections 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.589+0000 7f084a364640 1 -- 172.21.15.78:0/2882533705 wait complete. 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 Processor -- start 2024-04-25T13:57:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 -- start start 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0844101e40 0x7f084419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844106c20 0x7f084419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0844109790 0x7f08441a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f084410f600 con 0x7f0844106c20 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f084410f480 con 0x7f0844101e40 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f084a364640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f084410f780 con 0x7f0844109790 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f08437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844106c20 0x7f084419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f08437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844106c20 0x7f084419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43844/0 (socket says 172.21.15.78:43844) 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f08437fe640 1 -- 172.21.15.78:0/748429498 learned_addr learned my addr 172.21.15.78:0/748429498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f0848901640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0844109790 0x7f08441a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.590+0000 7f0843fff640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0844101e40 0x7f084419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08437fe640 1 -- 172.21.15.78:0/748429498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0844109790 msgr2=0x7f08441a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08437fe640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0844109790 0x7f08441a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08437fe640 1 -- 172.21.15.78:0/748429498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0844101e40 msgr2=0x7f084419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08437fe640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0844101e40 0x7f084419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08437fe640 1 -- 172.21.15.78:0/748429498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08441a1d10 con 0x7f0844106c20 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f0848901640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0844109790 0x7f08441a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f0843fff640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0844101e40 0x7f084419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08437fe640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844106c20 0x7f084419a710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f083400c890 tx=0x7f083400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08417fa640 1 -- 172.21.15.78:0/748429498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834004220 con 0x7f0844106c20 2024-04-25T13:57:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08417fa640 1 -- 172.21.15.78:0/748429498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08340043c0 con 0x7f0844106c20 2024-04-25T13:57:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08441a1fa0 con 0x7f0844106c20 2024-04-25T13:57:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f08417fa640 1 -- 172.21.15.78:0/748429498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834004a00 con 0x7f0844106c20 2024-04-25T13:57:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.591+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08441a2490 con 0x7f0844106c20 2024-04-25T13:57:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.592+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08440feed0 con 0x7f0844106c20 2024-04-25T13:57:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.593+0000 7f08417fa640 1 -- 172.21.15.78:0/748429498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f08340079b0 con 0x7f0844106c20 2024-04-25T13:57:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.594+0000 7f08417fa640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f082c0afaf0 0x7f082c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.594+0000 7f0843fff640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f082c0afaf0 0x7f082c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.594+0000 7f08417fa640 1 -- 172.21.15.78:0/748429498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f08340dec70 con 0x7f0844106c20 2024-04-25T13:57:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.594+0000 7f0843fff640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f082c0afaf0 0x7f082c0b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08380058c0 tx=0x7f0838005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.597+0000 7f08417fa640 1 -- 172.21.15.78:0/748429498 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0834014030 con 0x7f0844106c20 2024-04-25T13:57:12.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.776+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f084402dd20 con 0x7f0844106c20 2024-04-25T13:57:12.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.778+0000 7f08417fa640 1 -- 172.21.15.78:0/748429498 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22256 (secure 0 0 0) 0x7f08340a79a0 con 0x7f0844106c20 2024-04-25T13:57:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:12.778 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":74,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:57:11.484387+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f082c0afaf0 msgr2=0x7f082c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f082c0afaf0 0x7f082c0b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08380058c0 tx=0x7f0838005ec0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844106c20 msgr2=0x7f084419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844106c20 0x7f084419a710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f083400c890 tx=0x7f083400cd50 comp rx=0 tx=0).stop 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 shutdown_connections 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0844109790 0x7f08441a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f082c0afaf0 0x7f082c0b2000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0844106c20 0x7f084419a710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 --2- 172.21.15.78:0/748429498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0844101e40 0x7f084419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 >> 172.21.15.78:0/748429498 conn(0x7f08440fd3b0 msgr2=0x7f0844107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 shutdown_connections 2024-04-25T13:57:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.780+0000 7f084a364640 1 -- 172.21.15.78:0/748429498 wait complete. 2024-04-25T13:57:12.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 -- 172.21.15.78:0/1063108926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c109a40 msgr2=0x7f393c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 --2- 172.21.15.78:0/1063108926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c109a40 0x7f393c10be60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3934009ca0 tx=0x7f393402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 -- 172.21.15.78:0/1063108926 shutdown_connections 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 --2- 172.21.15.78:0/1063108926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c109a40 0x7f393c10be60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 --2- 172.21.15.78:0/1063108926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f393c104d40 0x7f393c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 --2- 172.21.15.78:0/1063108926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c103b00 0x7f393c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 -- 172.21.15.78:0/1063108926 >> 172.21.15.78:0/1063108926 conn(0x7f393c0fecf0 msgr2=0x7f393c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 -- 172.21.15.78:0/1063108926 shutdown_connections 2024-04-25T13:57:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 -- 172.21.15.78:0/1063108926 wait complete. 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 Processor -- start 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 -- start start 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c103b00 0x7f393c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c104d40 0x7f393c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.903+0000 7f3942e69640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f393c109a40 0x7f393c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3942e69640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f393c10f660 con 0x7f393c103b00 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3942e69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f393c10f4e0 con 0x7f393c104d40 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3942e69640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f393c10f7e0 con 0x7f393c109a40 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c103b00 0x7f393c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c103b00 0x7f393c19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43864/0 (socket says 172.21.15.78:43864) 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 -- 172.21.15.78:0/3676911396 learned_addr learned my addr 172.21.15.78:0/3676911396 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3941406640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f393c109a40 0x7f393c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f393bfff640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c104d40 0x7f393c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 -- 172.21.15.78:0/3676911396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f393c109a40 msgr2=0x7f393c1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f393c109a40 0x7f393c1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 -- 172.21.15.78:0/3676911396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c104d40 msgr2=0x7f393c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c104d40 0x7f393c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 -- 172.21.15.78:0/3676911396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c1a1cb0 con 0x7f393c103b00 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f393bfff640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c104d40 0x7f393c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3940c05640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c103b00 0x7f393c19a170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f393000d740 tx=0x7f393000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3939ffb640 1 -- 172.21.15.78:0/3676911396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930017070 con 0x7f393c103b00 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c1a1fa0 con 0x7f393c103b00 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3939ffb640 1 -- 172.21.15.78:0/3676911396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393000de50 con 0x7f393c103b00 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.904+0000 7f3939ffb640 1 -- 172.21.15.78:0/3676911396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930003c30 con 0x7f393c103b00 2024-04-25T13:57:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.905+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c1a2400 con 0x7f393c103b00 2024-04-25T13:57:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.906+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393c06afa0 con 0x7f393c103b00 2024-04-25T13:57:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.907+0000 7f3939ffb640 1 -- 172.21.15.78:0/3676911396 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3930003dd0 con 0x7f393c103b00 2024-04-25T13:57:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.908+0000 7f3939ffb640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f392c0afaf0 0x7f392c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.908+0000 7f393bfff640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f392c0afaf0 0x7f392c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.908+0000 7f3939ffb640 1 -- 172.21.15.78:0/3676911396 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f39300de9c0 con 0x7f393c103b00 2024-04-25T13:57:12.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.908+0000 7f393bfff640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f392c0afaf0 0x7f392c0b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3928005fd0 tx=0x7f3928005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:12.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:12.911+0000 7f3939ffb640 1 -- 172.21.15.78:0/3676911396 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39300a76f0 con 0x7f393c103b00 2024-04-25T13:57:13.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.128+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f393c02dbf0 con 0x7f393c103b00 2024-04-25T13:57:13.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.139+0000 7f3939ffb640 1 -- 172.21.15.78:0/3676911396 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f39300a6ff0 con 0x7f393c103b00 2024-04-25T13:57:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f392c0afaf0 msgr2=0x7f392c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f392c0afaf0 0x7f392c0b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3928005fd0 tx=0x7f3928005ec0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c103b00 msgr2=0x7f393c19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c103b00 0x7f393c19a170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f393000d740 tx=0x7f393000dc00 comp rx=0 tx=0).stop 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 shutdown_connections 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f393c109a40 0x7f393c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f393c104d40 0x7f393c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f392c0afaf0 0x7f392c0b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.141+0000 7f3942e69640 1 --2- 172.21.15.78:0/3676911396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f393c103b00 0x7f393c19a170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.142+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 >> 172.21.15.78:0/3676911396 conn(0x7f393c0fecf0 msgr2=0x7f393c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.142+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 shutdown_connections 2024-04-25T13:57:13.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.142+0000 7f3942e69640 1 -- 172.21.15.78:0/3676911396 wait complete. 2024-04-25T13:57:13.153 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T13:57:13.153 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 -- 172.21.15.78:0/414721648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc104d30 msgr2=0x7f02dc105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/414721648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc104d30 0x7f02dc105140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009900 tx=0x7f02d002a410 comp rx=0 tx=0).stop 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 -- 172.21.15.78:0/414721648 shutdown_connections 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/414721648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc109a30 0x7f02dc10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/414721648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc104d30 0x7f02dc105140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/414721648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f02dc103af0 0x7f02dc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 -- 172.21.15.78:0/414721648 >> 172.21.15.78:0/414721648 conn(0x7f02dc0fece0 msgr2=0x7f02dc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 -- 172.21.15.78:0/414721648 shutdown_connections 2024-04-25T13:57:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.264+0000 7f02e0d4d640 1 -- 172.21.15.78:0/414721648 wait complete. 2024-04-25T13:57:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 Processor -- start 2024-04-25T13:57:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 -- start start 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc103af0 0x7f02dc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc104d30 0x7f02dc19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f02dc109a30 0x7f02dc1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10f670 con 0x7f02dc103af0 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10f4f0 con 0x7f02dc104d30 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02e0d4d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10f7f0 con 0x7f02dc109a30 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc103af0 0x7f02dc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc103af0 0x7f02dc19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43882/0 (socket says 172.21.15.78:43882) 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 -- 172.21.15.78:0/3608349230 learned_addr learned my addr 172.21.15.78:0/3608349230 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02dad9d640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f02dc109a30 0x7f02dc1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 -- 172.21.15.78:0/3608349230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f02dc109a30 msgr2=0x7f02dc1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02d9d9b640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc104d30 0x7f02dc19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f02dc109a30 0x7f02dc1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 -- 172.21.15.78:0/3608349230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc104d30 msgr2=0x7f02dc19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc104d30 0x7f02dc19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 -- 172.21.15.78:0/3608349230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc1a1dd0 con 0x7f02dc103af0 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02d9d9b640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc104d30 0x7f02dc19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02da59c640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc103af0 0x7f02dc19a290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f02cc006fd0 tx=0x7f02cc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.265+0000 7f02cb7fe640 1 -- 172.21.15.78:0/3608349230 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc011830 con 0x7f02dc103af0 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.266+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc1a20c0 con 0x7f02dc103af0 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.266+0000 7f02cb7fe640 1 -- 172.21.15.78:0/3608349230 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc016070 con 0x7f02dc103af0 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.266+0000 7f02cb7fe640 1 -- 172.21.15.78:0/3608349230 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc010e00 con 0x7f02dc103af0 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.266+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc1a26e0 con 0x7f02dc103af0 2024-04-25T13:57:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.267+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02dc06a140 con 0x7f02dc103af0 2024-04-25T13:57:13.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.268+0000 7f02cb7fe640 1 -- 172.21.15.78:0/3608349230 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f02cc0119d0 con 0x7f02dc103af0 2024-04-25T13:57:13.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.269+0000 7f02cb7fe640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02c40afb70 0x7f02c40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:13.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.269+0000 7f02d9d9b640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02c40afb70 0x7f02c40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:13.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.269+0000 7f02cb7fe640 1 -- 172.21.15.78:0/3608349230 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f02cc0de8e0 con 0x7f02dc103af0 2024-04-25T13:57:13.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.269+0000 7f02d9d9b640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02c40afb70 0x7f02c40b2080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02d0006010 tx=0x7f02d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.272+0000 7f02cb7fe640 1 -- 172.21.15.78:0/3608349230 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02cc0a7570 con 0x7f02dc103af0 2024-04-25T13:57:13.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2379449510' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:57:13.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: pgmap v276: 97 pgs: 9 creating+peering, 7 active+clean, 81 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1377952521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/748429498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: Cluster is now healthy 2024-04-25T13:57:13.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3676911396' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:57:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2379449510' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:57:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: pgmap v276: 97 pgs: 9 creating+peering, 7 active+clean, 81 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T13:57:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:57:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:57:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1377952521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/748429498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:13.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:57:13.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: Cluster is now healthy 2024-04-25T13:57:13.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3676911396' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:57:13.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:13.490+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f02dc02db50 con 0x7f02dc103af0 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2379449510' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: pgmap v276: 97 pgs: 9 creating+peering, 7 active+clean, 81 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1377952521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/748429498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T13:57:13.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: Cluster is now healthy 2024-04-25T13:57:13.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3676911396' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:57:14.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.145+0000 7f02cb7fe640 1 -- 172.21.15.78:0/3608349230 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f02cc0a6e70 con 0x7f02dc103af0 2024-04-25T13:57:14.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02c40afb70 msgr2=0x7f02c40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:14.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02c40afb70 0x7f02c40b2080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02d0006010 tx=0x7f02d0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc103af0 msgr2=0x7f02dc19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:14.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc103af0 0x7f02dc19a290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f02cc006fd0 tx=0x7f02cc00b8e0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 shutdown_connections 2024-04-25T13:57:14.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f02dc109a30 0x7f02dc1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02dc104d30 0x7f02dc19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02c40afb70 0x7f02c40b2080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 --2- 172.21.15.78:0/3608349230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02dc103af0 0x7f02dc19a290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 >> 172.21.15.78:0/3608349230 conn(0x7f02dc0fece0 msgr2=0x7f02dc105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:14.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 shutdown_connections 2024-04-25T13:57:14.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.147+0000 7f02e0d4d640 1 -- 172.21.15.78:0/3608349230 wait complete. 2024-04-25T13:57:14.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T13:57:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 -- 172.21.15.78:0/3133078157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230102f40 msgr2=0x7f2230103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 --2- 172.21.15.78:0/3133078157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230102f40 0x7f2230103350 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2224009900 tx=0x7f222402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 -- 172.21.15.78:0/3133078157 shutdown_connections 2024-04-25T13:57:14.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 --2- 172.21.15.78:0/3133078157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2230104460 0x7f223010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 --2- 172.21.15.78:0/3133078157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2230103920 0x7f2230103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 --2- 172.21.15.78:0/3133078157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230102f40 0x7f2230103350 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 -- 172.21.15.78:0/3133078157 >> 172.21.15.78:0/3133078157 conn(0x7f22300fd380 msgr2=0x7f22300ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 -- 172.21.15.78:0/3133078157 shutdown_connections 2024-04-25T13:57:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.272+0000 7f2236eee640 1 -- 172.21.15.78:0/3133078157 wait complete. 2024-04-25T13:57:14.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 Processor -- start 2024-04-25T13:57:14.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 -- start start 2024-04-25T13:57:14.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2230102f40 0x7f223019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:14.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230103920 0x7f223019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:14.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2230104460 0x7f22301a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:14.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f223010f660 con 0x7f2230103920 2024-04-25T13:57:14.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f223010f4e0 con 0x7f2230102f40 2024-04-25T13:57:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2236eee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f223010f7e0 con 0x7f2230104460 2024-04-25T13:57:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230103920 0x7f223019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230103920 0x7f223019c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43912/0 (socket says 172.21.15.78:43912) 2024-04-25T13:57:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 -- 172.21.15.78:0/4231718221 learned_addr learned my addr 172.21.15.78:0/4231718221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f223548b640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2230104460 0x7f22301a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:14.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2234c8a640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2230102f40 0x7f223019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 -- 172.21.15.78:0/4231718221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2230104460 msgr2=0x7f22301a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2230104460 0x7f22301a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 -- 172.21.15.78:0/4231718221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2230102f40 msgr2=0x7f223019c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2230102f40 0x7f223019c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 -- 172.21.15.78:0/4231718221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22301a3ff0 con 0x7f2230103920 2024-04-25T13:57:14.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f2234c8a640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2230102f40 0x7f223019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.273+0000 7f222ffff640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230103920 0x7f223019c9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f221c00d950 tx=0x7f221c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.274+0000 7f222dffb640 1 -- 172.21.15.78:0/4231718221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c017070 con 0x7f2230103920 2024-04-25T13:57:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.274+0000 7f222dffb640 1 -- 172.21.15.78:0/4231718221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c009c30 con 0x7f2230103920 2024-04-25T13:57:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.274+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22301a42e0 con 0x7f2230103920 2024-04-25T13:57:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.274+0000 7f222dffb640 1 -- 172.21.15.78:0/4231718221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c00f620 con 0x7f2230103920 2024-04-25T13:57:14.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.274+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22301a48b0 con 0x7f2230103920 2024-04-25T13:57:14.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.275+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22301a4b10 con 0x7f2230103920 2024-04-25T13:57:14.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.276+0000 7f222dffb640 1 -- 172.21.15.78:0/4231718221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f221c010460 con 0x7f2230103920 2024-04-25T13:57:14.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.277+0000 7f222dffb640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f22200afaf0 0x7f22200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:14.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.277+0000 7f2234c8a640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f22200afaf0 0x7f22200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:14.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.277+0000 7f222dffb640 1 -- 172.21.15.78:0/4231718221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f221c0de7f0 con 0x7f2230103920 2024-04-25T13:57:14.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.277+0000 7f2234c8a640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f22200afaf0 0x7f22200b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2224005f70 tx=0x7f2224000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:14.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.282+0000 7f222dffb640 1 -- 172.21.15.78:0/4231718221 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f221c0a7520 con 0x7f2230103920 2024-04-25T13:57:14.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:14 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3676911396' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:57:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:14 smithi129 ceph-mon[32569]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T13:57:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:14 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] up:active 2024-04-25T13:57:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:14 smithi129 ceph-mon[32569]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:57:14.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:14 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3608349230' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:57:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:14 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3676911396' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:57:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:14 smithi078 ceph-mon[29163]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T13:57:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:14 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] up:active 2024-04-25T13:57:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:14 smithi078 ceph-mon[29163]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:57:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:14 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3608349230' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:57:14.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:14.500+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f22301a4ce0 con 0x7f2230103920 2024-04-25T13:57:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:14 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3676911396' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:57:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:14 smithi112 ceph-mon[32617]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T13:57:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:14 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] up:active 2024-04-25T13:57:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:14 smithi112 ceph-mon[32617]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:57:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:14 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3608349230' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T13:57:15.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.067+0000 7f222dffb640 1 -- 172.21.15.78:0/4231718221 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f221c009de0 con 0x7f2230103920 2024-04-25T13:57:15.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f22200afaf0 msgr2=0x7f22200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:15.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f22200afaf0 0x7f22200b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2224005f70 tx=0x7f2224000bc0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230103920 msgr2=0x7f223019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:15.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230103920 0x7f223019c9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f221c00d950 tx=0x7f221c00de10 comp rx=0 tx=0).stop 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 shutdown_connections 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2230104460 0x7f22301a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f22200afaf0 0x7f22200b2000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2230103920 0x7f223019c9f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 --2- 172.21.15.78:0/4231718221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2230102f40 0x7f223019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 >> 172.21.15.78:0/4231718221 conn(0x7f22300fd380 msgr2=0x7f22300fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 shutdown_connections 2024-04-25T13:57:15.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.069+0000 7f2236eee640 1 -- 172.21.15.78:0/4231718221 wait complete. 2024-04-25T13:57:15.086 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T13:57:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 -- 172.21.15.78:0/2674964210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c1029e0 msgr2=0x7f6e4c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/2674964210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c1029e0 0x7f6e4c1036e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4802a410 comp rx=0 tx=0).stop 2024-04-25T13:57:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 -- 172.21.15.78:0/2674964210 shutdown_connections 2024-04-25T13:57:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/2674964210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e4c108920 0x7f6e4c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/2674964210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4c103c20 0x7f6e4c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/2674964210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c1029e0 0x7f6e4c1036e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 -- 172.21.15.78:0/2674964210 >> 172.21.15.78:0/2674964210 conn(0x7f6e4c0fdbd0 msgr2=0x7f6e4c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 -- 172.21.15.78:0/2674964210 shutdown_connections 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 -- 172.21.15.78:0/2674964210 wait complete. 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 Processor -- start 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.199+0000 7f6e53ff3640 1 -- start start 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4c1029e0 0x7f6e4c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e4c103c20 0x7f6e4c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c108920 0x7f6e4c1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10e4c0 con 0x7f6e4c108920 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10e340 con 0x7f6e4c103c20 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c10e640 con 0x7f6e4c1029e0 2024-04-25T13:57:15.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c108920 0x7f6e4c1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c108920 0x7f6e4c1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43930/0 (socket says 172.21.15.78:43930) 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 -- 172.21.15.78:0/3056197142 learned_addr learned my addr 172.21.15.78:0/3056197142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e5158e640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e4c103c20 0x7f6e4c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e51d8f640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4c1029e0 0x7f6e4c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 -- 172.21.15.78:0/3056197142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4c1029e0 msgr2=0x7f6e4c06a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4c1029e0 0x7f6e4c06a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 -- 172.21.15.78:0/3056197142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e4c103c20 msgr2=0x7f6e4c19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e4c103c20 0x7f6e4c19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 -- 172.21.15.78:0/3056197142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c1a1fb0 con 0x7f6e4c108920 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e52590640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c108920 0x7f6e4c1a1820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400eb10 tx=0x7f6e4400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e42ffd640 1 -- 172.21.15.78:0/3056197142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400ce20 con 0x7f6e4c108920 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c1a22a0 con 0x7f6e4c108920 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e42ffd640 1 -- 172.21.15.78:0/3056197142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e44005020 con 0x7f6e4c108920 2024-04-25T13:57:15.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e42ffd640 1 -- 172.21.15.78:0/3056197142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44010690 con 0x7f6e4c108920 2024-04-25T13:57:15.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.200+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c1a27f0 con 0x7f6e4c108920 2024-04-25T13:57:15.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.202+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4c107730 con 0x7f6e4c108920 2024-04-25T13:57:15.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.203+0000 7f6e42ffd640 1 -- 172.21.15.78:0/3056197142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6e44005510 con 0x7f6e4c108920 2024-04-25T13:57:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.203+0000 7f6e42ffd640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e380afaf0 0x7f6e380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.203+0000 7f6e51d8f640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e380afaf0 0x7f6e380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.203+0000 7f6e42ffd640 1 -- 172.21.15.78:0/3056197142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6e440df0d0 con 0x7f6e4c108920 2024-04-25T13:57:15.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.204+0000 7f6e51d8f640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e380afaf0 0x7f6e380b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009230 tx=0x7f6e48031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.208+0000 7f6e42ffd640 1 -- 172.21.15.78:0/3056197142 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e440a7e00 con 0x7f6e4c108920 2024-04-25T13:57:15.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: pgmap v278: 97 pgs: 9 creating+peering, 60 active+clean, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 766 B/s wr, 6 op/s 2024-04-25T13:57:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:57:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3608349230' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:57:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:57:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:57:15.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:57:15.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:57:15.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4231718221' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:57:15.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4231718221' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:57:15.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] up:active 2024-04-25T13:57:15.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:57:15.415 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:57:15.416 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: Cluster is now healthy 2024-04-25T13:57:15.416 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T13:57:15.416 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T13:57:15.416 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:15 smithi129 ceph-mon[32569]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T13:57:15.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:15.424+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6e4c02dd10 con 0x7f6e4c108920 2024-04-25T13:57:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: pgmap v278: 97 pgs: 9 creating+peering, 60 active+clean, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 766 B/s wr, 6 op/s 2024-04-25T13:57:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:57:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3608349230' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:57:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:57:15.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:57:15.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:57:15.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:57:15.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4231718221' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:57:15.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4231718221' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:57:15.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] up:active 2024-04-25T13:57:15.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:57:15.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:57:15.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: Cluster is now healthy 2024-04-25T13:57:15.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T13:57:15.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T13:57:15.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:15 smithi078 ceph-mon[29163]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: pgmap v278: 97 pgs: 9 creating+peering, 60 active+clean, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 766 B/s wr, 6 op/s 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3608349230' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T13:57:15.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4231718221' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4231718221' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] up:active 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: Cluster is now healthy 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T13:57:15.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:15 smithi112 ceph-mon[32617]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-25T13:57:16.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.074+0000 7f6e42ffd640 1 -- 172.21.15.78:0/3056197142 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f6e440a7700 con 0x7f6e4c108920 2024-04-25T13:57:16.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e380afaf0 msgr2=0x7f6e380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e380afaf0 0x7f6e380b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009230 tx=0x7f6e48031400 comp rx=0 tx=0).stop 2024-04-25T13:57:16.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c108920 msgr2=0x7f6e4c1a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c108920 0x7f6e4c1a1820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400eb10 tx=0x7f6e4400ef10 comp rx=0 tx=0).stop 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 shutdown_connections 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6e380afaf0 0x7f6e380b2000 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4c108920 0x7f6e4c1a1820 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e4c103c20 0x7f6e4c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 --2- 172.21.15.78:0/3056197142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6e4c1029e0 0x7f6e4c06a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 >> 172.21.15.78:0/3056197142 conn(0x7f6e4c0fdbd0 msgr2=0x7f6e4c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 shutdown_connections 2024-04-25T13:57:16.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.078+0000 7f6e53ff3640 1 -- 172.21.15.78:0/3056197142 wait complete. 2024-04-25T13:57:16.094 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T13:57:16.094 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 -- 172.21.15.78:0/1438467780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda34102180 msgr2=0x7fda34102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 --2- 172.21.15.78:0/1438467780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda34102180 0x7fda34102e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda30009900 tx=0x7fda3002a410 comp rx=0 tx=0).stop 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 -- 172.21.15.78:0/1438467780 shutdown_connections 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 --2- 172.21.15.78:0/1438467780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda341080c0 0x7fda3410a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 --2- 172.21.15.78:0/1438467780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda341033c0 0x7fda341037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 --2- 172.21.15.78:0/1438467780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda34102180 0x7fda34102e80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 -- 172.21.15.78:0/1438467780 >> 172.21.15.78:0/1438467780 conn(0x7fda340fd350 msgr2=0x7fda340ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 -- 172.21.15.78:0/1438467780 shutdown_connections 2024-04-25T13:57:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.212+0000 7fda3bd46640 1 -- 172.21.15.78:0/1438467780 wait complete. 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 Processor -- start 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 -- start start 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda34102180 0x7fda34199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda341033c0 0x7fda341996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda341080c0 0x7fda341a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda3410dca0 con 0x7fda341080c0 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda3410db20 con 0x7fda34102180 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3bd46640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda3410de20 con 0x7fda341033c0 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda341080c0 0x7fda341a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda341080c0 0x7fda341a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43936/0 (socket says 172.21.15.78:43936) 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 -- 172.21.15.78:0/2277218681 learned_addr learned my addr 172.21.15.78:0/2277218681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda392e1640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda341033c0 0x7fda341996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda39ae2640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda34102180 0x7fda34199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 -- 172.21.15.78:0/2277218681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda341033c0 msgr2=0x7fda341996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda341033c0 0x7fda341996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 -- 172.21.15.78:0/2277218681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda34102180 msgr2=0x7fda34199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda34102180 0x7fda34199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.213+0000 7fda3a2e3640 1 -- 172.21.15.78:0/2277218681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda341a0cc0 con 0x7fda341080c0 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.214+0000 7fda3a2e3640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda341080c0 0x7fda341a0650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00efd0 tx=0x7fda2c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.214+0000 7fda2affd640 1 -- 172.21.15.78:0/2277218681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c003b70 con 0x7fda341080c0 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.214+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda341a0fb0 con 0x7fda341080c0 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.214+0000 7fda2affd640 1 -- 172.21.15.78:0/2277218681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2c003d10 con 0x7fda341080c0 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.214+0000 7fda2affd640 1 -- 172.21.15.78:0/2277218681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0108a0 con 0x7fda341080c0 2024-04-25T13:57:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.214+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda341a1410 con 0x7fda341080c0 2024-04-25T13:57:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.215+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda341077c0 con 0x7fda341080c0 2024-04-25T13:57:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.216+0000 7fda2affd640 1 -- 172.21.15.78:0/2277218681 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fda2c004b40 con 0x7fda341080c0 2024-04-25T13:57:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.217+0000 7fda2affd640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fda200afaf0 0x7fda200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.217+0000 7fda39ae2640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fda200afaf0 0x7fda200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.217+0000 7fda2affd640 1 -- 172.21.15.78:0/2277218681 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fda2c0df2f0 con 0x7fda341080c0 2024-04-25T13:57:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.217+0000 7fda39ae2640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fda200afaf0 0x7fda200b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda30005ab0 tx=0x7fda30031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.221+0000 7fda2affd640 1 -- 172.21.15.78:0/2277218681 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda2c0a7fc0 con 0x7fda341080c0 2024-04-25T13:57:16.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:16 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3056197142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:57:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:16 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3056197142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:57:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:16 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] up:active 2024-04-25T13:57:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:16 smithi129 ceph-mon[32569]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T13:57:16.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:16 smithi129 ceph-mon[32569]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:57:16.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.426+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fda34062fe0 con 0x7fda200afaf0 2024-04-25T13:57:16.431 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3056197142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:57:16.431 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3056197142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:57:16.431 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] up:active 2024-04-25T13:57:16.431 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-mon[29163]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T13:57:16.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-mon[29163]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:57:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.453+0000 7fda2affd640 1 -- 172.21.15.78:0/2277218681 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda34062fe0 con 0x7fda200afaf0 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fda200afaf0 msgr2=0x7fda200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fda200afaf0 0x7fda200b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda30005ab0 tx=0x7fda30031400 comp rx=0 tx=0).stop 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda341080c0 msgr2=0x7fda341a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda341080c0 0x7fda341a0650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00efd0 tx=0x7fda2c00c6a0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 shutdown_connections 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fda200afaf0 0x7fda200b2000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda341080c0 0x7fda341a0650 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda341033c0 0x7fda341996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 --2- 172.21.15.78:0/2277218681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda34102180 0x7fda34199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 >> 172.21.15.78:0/2277218681 conn(0x7fda340fd350 msgr2=0x7fda340fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 shutdown_connections 2024-04-25T13:57:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.455+0000 7fda3bd46640 1 -- 172.21.15.78:0/2277218681 wait complete. 2024-04-25T13:57:16.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3056197142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3056197142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] up:active 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-mon[32617]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-mon[32617]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: 2024-04-25T13:57:16.131+0000 7f0197d7d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: 2024-04-25T13:57:16.131+0000 7f0197d7d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T13:57:16.516 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 13:57:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: 2024-04-25T13:57:16.131+0000 7f0197d7d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T13:57:16.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/2785391014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec103c30 msgr2=0x7fd9ec104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/2785391014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec103c30 0x7fd9ec104040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009900 tx=0x7fd9dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:16.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/2785391014 shutdown_connections 2024-04-25T13:57:16.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/2785391014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9ec108930 0x7fd9ec10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/2785391014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec103c30 0x7fd9ec104040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/2785391014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9ec1029f0 0x7fd9ec1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/2785391014 >> 172.21.15.78:0/2785391014 conn(0x7fd9ec0fdbc0 msgr2=0x7fd9ec100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/2785391014 shutdown_connections 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/2785391014 wait complete. 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.578+0000 7fd9f3cb1640 1 Processor -- start 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 -- start start 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec1029f0 0x7fd9ec075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9ec103c30 0x7fd9ec19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9ec108930 0x7fd9ec1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec10e4f0 con 0x7fd9ec1029f0 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec10e370 con 0x7fd9ec108930 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec10e670 con 0x7fd9ec103c30 2024-04-25T13:57:16.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec1029f0 0x7fd9ec075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec1029f0 0x7fd9ec075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53326/0 (socket says 172.21.15.78:53326) 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 -- 172.21.15.78:0/83726903 learned_addr learned my addr 172.21.15.78:0/83726903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f224e640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9ec108930 0x7fd9ec1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f124c640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9ec103c30 0x7fd9ec19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 -- 172.21.15.78:0/83726903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9ec103c30 msgr2=0x7fd9ec19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9ec103c30 0x7fd9ec19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 -- 172.21.15.78:0/83726903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9ec108930 msgr2=0x7fd9ec1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9ec108930 0x7fd9ec1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 -- 172.21.15.78:0/83726903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec1a1d20 con 0x7fd9ec1029f0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f1a4d640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec1029f0 0x7fd9ec075860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c970 tx=0x7fd9e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/83726903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8004220 con 0x7fd9ec1029f0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/83726903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e80043c0 con 0x7fd9ec1029f0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec1a1fb0 con 0x7fd9ec1029f0 2024-04-25T13:57:16.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/83726903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800f450 con 0x7fd9ec1029f0 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.579+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec1a2470 con 0x7fd9ec1029f0 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.580+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ec107740 con 0x7fd9ec1029f0 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.583+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/83726903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd9e80079b0 con 0x7fd9ec1029f0 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.583+0000 7fd9e2ffd640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9d80afaf0 0x7fd9d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.583+0000 7fd9f124c640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9d80afaf0 0x7fd9d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.583+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/83726903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fd9e80debd0 con 0x7fd9ec1029f0 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.584+0000 7fd9f124c640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9d80afaf0 0x7fd9d80b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:16.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.587+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/83726903 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9e8014030 con 0x7fd9ec1029f0 2024-04-25T13:57:16.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:57:16.431+0000 7f2785937700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:57:16.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:57:16.431+0000 7f2785937700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:57:16.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:57:16.431+0000 7f2785937700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:57:16.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:57:16.431+0000 7f2785937700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:57:16.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 13:57:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T13:57:16.431+0000 7f2785937700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:57:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.792+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd9ec062fe0 con 0x7fd9d80afaf0 2024-04-25T13:57:16.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.817+0000 7fd9e2ffd640 1 -- 172.21.15.78:0/83726903 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd9ec062fe0 con 0x7fd9d80afaf0 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9d80afaf0 msgr2=0x7fd9d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9d80afaf0 0x7fd9d80b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc031400 comp rx=0 tx=0).stop 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec1029f0 msgr2=0x7fd9ec075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec1029f0 0x7fd9ec075860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c970 tx=0x7fd9e800ce30 comp rx=0 tx=0).stop 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 shutdown_connections 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9ec108930 0x7fd9ec1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9ec103c30 0x7fd9ec19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9d80afaf0 0x7fd9d80b2000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 --2- 172.21.15.78:0/83726903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9ec1029f0 0x7fd9ec075860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.819+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 >> 172.21.15.78:0/83726903 conn(0x7fd9ec0fdbc0 msgr2=0x7fd9ec104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.820+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 shutdown_connections 2024-04-25T13:57:16.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.820+0000 7fd9f3cb1640 1 -- 172.21.15.78:0/83726903 wait complete. 2024-04-25T13:57:16.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T13:57:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 -- 172.21.15.78:0/2756432729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0541050a0 msgr2=0x7ff0541054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 --2- 172.21.15.78:0/2756432729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0541050a0 0x7ff0541054b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff044009900 tx=0x7ff04402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 -- 172.21.15.78:0/2756432729 shutdown_connections 2024-04-25T13:57:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 --2- 172.21.15.78:0/2756432729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff054102250 0x7ff05410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 --2- 172.21.15.78:0/2756432729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff054101750 0x7ff054101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 --2- 172.21.15.78:0/2756432729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0541050a0 0x7ff0541054b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 -- 172.21.15.78:0/2756432729 >> 172.21.15.78:0/2756432729 conn(0x7ff0540fd350 msgr2=0x7ff0540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 -- 172.21.15.78:0/2756432729 shutdown_connections 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.944+0000 7ff059e34640 1 -- 172.21.15.78:0/2756432729 wait complete. 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 Processor -- start 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 -- start start 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff054101750 0x7ff05419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff054102250 0x7ff05419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541050a0 0x7ff0541a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff05410f610 con 0x7ff054101750 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff05410f490 con 0x7ff0541050a0 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff059e34640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff05410f790 con 0x7ff054102250 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff0537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff054101750 0x7ff05419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff0537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff054101750 0x7ff05419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53342/0 (socket says 172.21.15.78:53342) 2024-04-25T13:57:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff0537fe640 1 -- 172.21.15.78:0/1560158042 learned_addr learned my addr 172.21.15.78:0/1560158042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff052ffd640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff054102250 0x7ff05419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff053fff640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541050a0 0x7ff0541a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff0537fe640 1 -- 172.21.15.78:0/1560158042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff054102250 msgr2=0x7ff05419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff0537fe640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff054102250 0x7ff05419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff0537fe640 1 -- 172.21.15.78:0/1560158042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541050a0 msgr2=0x7ff0541a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:16.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.945+0000 7ff0537fe640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541050a0 0x7ff0541a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.946+0000 7ff0537fe640 1 -- 172.21.15.78:0/1560158042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0541a1d50 con 0x7ff054101750 2024-04-25T13:57:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.946+0000 7ff0537fe640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff054101750 0x7ff05419a210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff044006010 tx=0x7ff044004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.946+0000 7ff050ff9640 1 -- 172.21.15.78:0/1560158042 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04403f030 con 0x7ff054101750 2024-04-25T13:57:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.946+0000 7ff050ff9640 1 -- 172.21.15.78:0/1560158042 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff044003aa0 con 0x7ff054101750 2024-04-25T13:57:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.946+0000 7ff050ff9640 1 -- 172.21.15.78:0/1560158042 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044035070 con 0x7ff054101750 2024-04-25T13:57:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.946+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0541a1fe0 con 0x7ff054101750 2024-04-25T13:57:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.946+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0541a23e0 con 0x7ff054101750 2024-04-25T13:57:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.947+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff054108a30 con 0x7ff054101750 2024-04-25T13:57:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.949+0000 7ff050ff9640 1 -- 172.21.15.78:0/1560158042 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff044042030 con 0x7ff054101750 2024-04-25T13:57:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.950+0000 7ff050ff9640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff03c0afaf0 0x7ff03c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.950+0000 7ff052ffd640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff03c0afaf0 0x7ff03c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.950+0000 7ff050ff9640 1 -- 172.21.15.78:0/1560158042 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff0440ff460 con 0x7ff054101750 2024-04-25T13:57:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.950+0000 7ff052ffd640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff03c0afaf0 0x7ff03c0b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff05419b2b0 tx=0x7ff048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:16.953+0000 7ff050ff9640 1 -- 172.21.15.78:0/1560158042 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0440c7b40 con 0x7ff054101750 2024-04-25T13:57:17.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.160+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff054063070 con 0x7ff03c0afaf0 2024-04-25T13:57:17.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.166+0000 7ff050ff9640 1 -- 172.21.15.78:0/1560158042 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff054063070 con 0x7ff03c0afaf0 2024-04-25T13:57:17.166 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_0/e940e0f8-b6e6-4701-94d2-31fbe168d6c5 2024-04-25T13:57:17.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff03c0afaf0 msgr2=0x7ff03c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff03c0afaf0 0x7ff03c0b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff05419b2b0 tx=0x7ff048005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:17.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff054101750 msgr2=0x7ff05419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff054101750 0x7ff05419a210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff044006010 tx=0x7ff044004270 comp rx=0 tx=0).stop 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 shutdown_connections 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541050a0 0x7ff0541a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff054102250 0x7ff05419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff03c0afaf0 0x7ff03c0b2000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 --2- 172.21.15.78:0/1560158042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff054101750 0x7ff05419a210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 >> 172.21.15.78:0/1560158042 conn(0x7ff0540fd350 msgr2=0x7ff054105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 shutdown_connections 2024-04-25T13:57:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.168+0000 7ff059e34640 1 -- 172.21.15.78:0/1560158042 wait complete. 2024-04-25T13:57:17.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T13:57:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1568088084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 msgr2=0x7f62b410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1568088084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b410aba0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009ca0 tx=0x7f62ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1568088084 shutdown_connections 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1568088084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b410aba0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1568088084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4103920 0x7f62b4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1568088084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b4102f40 0x7f62b4103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1568088084 >> 172.21.15.78:0/1568088084 conn(0x7f62b40fd380 msgr2=0x7f62b40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1568088084 shutdown_connections 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.292+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1568088084 wait complete. 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 Processor -- start 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 -- start start 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4102f40 0x7f62b41021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b4103920 0x7f62b419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b41a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62b410e530 con 0x7f62b4104460 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62b410e3b0 con 0x7f62b4102f40 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62bbf8f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62b410e6b0 con 0x7f62b4103920 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62ba52c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b41a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62ba52c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b41a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53366/0 (socket says 172.21.15.78:53366) 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62ba52c640 1 -- 172.21.15.78:0/1303377617 learned_addr learned my addr 172.21.15.78:0/1303377617 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62b952a640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b4103920 0x7f62b419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.293+0000 7f62b9d2b640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4102f40 0x7f62b41021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62ba52c640 1 -- 172.21.15.78:0/1303377617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b4103920 msgr2=0x7f62b419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62ba52c640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b4103920 0x7f62b419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62ba52c640 1 -- 172.21.15.78:0/1303377617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4102f40 msgr2=0x7f62b41021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62ba52c640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4102f40 0x7f62b41021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62ba52c640 1 -- 172.21.15.78:0/1303377617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b41a1cf0 con 0x7f62b4104460 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62b9d2b640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4102f40 0x7f62b41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62ba52c640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b41a14d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f62ac004af0 tx=0x7f62ac031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62a6ffd640 1 -- 172.21.15.78:0/1303377617 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac003b70 con 0x7f62b4104460 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62a6ffd640 1 -- 172.21.15.78:0/1303377617 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ac003d10 con 0x7f62b4104460 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b41a1f80 con 0x7f62b4104460 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62a6ffd640 1 -- 172.21.15.78:0/1303377617 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac02fb20 con 0x7f62b4104460 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.294+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b41a23e0 con 0x7f62b4104460 2024-04-25T13:57:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.295+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b406a190 con 0x7f62b4104460 2024-04-25T13:57:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.296+0000 7f62a6ffd640 1 -- 172.21.15.78:0/1303377617 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f62ac004580 con 0x7f62b4104460 2024-04-25T13:57:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.297+0000 7f62a6ffd640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62a00afaf0 0x7f62a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.297+0000 7f62b9d2b640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62a00afaf0 0x7f62a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.298+0000 7f62a6ffd640 1 -- 172.21.15.78:0/1303377617 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f62ac1002b0 con 0x7f62b4104460 2024-04-25T13:57:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.298+0000 7f62b9d2b640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62a00afaf0 0x7f62a00b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f62b406b540 tx=0x7f62a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:17.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.301+0000 7f62a6ffd640 1 -- 172.21.15.78:0/1303377617 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62b406a190 con 0x7f62b4104460 2024-04-25T13:57:17.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:17 smithi129 ceph-mon[32569]: pgmap v279: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T13:57:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:17 smithi078 ceph-mon[29163]: pgmap v279: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T13:57:17.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.509+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f62b4062fe0 con 0x7f62a00afaf0 2024-04-25T13:57:17.605 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:17 smithi112 ceph-mon[32617]: pgmap v279: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T13:57:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.537+0000 7f62a6ffd640 1 -- 172.21.15.78:0/1303377617 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f62b4062fe0 con 0x7f62a00afaf0 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62a00afaf0 msgr2=0x7f62a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62a00afaf0 0x7f62a00b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f62b406b540 tx=0x7f62a8005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 msgr2=0x7f62b41a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b41a14d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f62ac004af0 tx=0x7f62ac031700 comp rx=0 tx=0).stop 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 shutdown_connections 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f62a00afaf0 0x7f62a00b2000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b4104460 0x7f62b41a14d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b4103920 0x7f62b419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 --2- 172.21.15.78:0/1303377617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b4102f40 0x7f62b41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 >> 172.21.15.78:0/1303377617 conn(0x7f62b40fd380 msgr2=0x7f62b4106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 shutdown_connections 2024-04-25T13:57:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.539+0000 7f62bbf8f640 1 -- 172.21.15.78:0/1303377617 wait complete. 2024-04-25T13:57:17.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T13:57:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/3504228421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 msgr2=0x7fbd0810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/3504228421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd0810be50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009ca0 tx=0x7fbd0002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/3504228421 shutdown_connections 2024-04-25T13:57:17.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/3504228421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd0810be50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/3504228421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd08104d30 0x7fbd08105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/3504228421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd08103af0 0x7fbd081047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/3504228421 >> 172.21.15.78:0/3504228421 conn(0x7fbd080fece0 msgr2=0x7fbd08101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/3504228421 shutdown_connections 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.720+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/3504228421 wait complete. 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 Processor -- start 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 -- start start 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd08103af0 0x7fbd0819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd08104d30 0x7fbd0819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd081a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810f600 con 0x7fbd08109a30 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810f480 con 0x7fbd08103af0 2024-04-25T13:57:17.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0ffc5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810f780 con 0x7fbd08104d30 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd081a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd081a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53380/0 (socket says 172.21.15.78:53380) 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 -- 172.21.15.78:0/1681563440 learned_addr learned my addr 172.21.15.78:0/1681563440 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0dd61640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd08103af0 0x7fbd0819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0d560640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd08104d30 0x7fbd0819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 -- 172.21.15.78:0/1681563440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd08104d30 msgr2=0x7fbd0819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd08104d30 0x7fbd0819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 -- 172.21.15.78:0/1681563440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd08103af0 msgr2=0x7fbd0819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd08103af0 0x7fbd0819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.721+0000 7fbd0e562640 1 -- 172.21.15.78:0/1681563440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd081a1c10 con 0x7fbd08109a30 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.722+0000 7fbd0e562640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd081a1650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd000097b0 tx=0x7fbd00031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.722+0000 7fbcfeffd640 1 -- 172.21.15.78:0/1681563440 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00003bf0 con 0x7fbd08109a30 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.722+0000 7fbcfeffd640 1 -- 172.21.15.78:0/1681563440 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd00003d90 con 0x7fbd08109a30 2024-04-25T13:57:17.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.722+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd081a1ea0 con 0x7fbd08109a30 2024-04-25T13:57:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.722+0000 7fbcfeffd640 1 -- 172.21.15.78:0/1681563440 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0002fb70 con 0x7fbd08109a30 2024-04-25T13:57:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.722+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd081a2300 con 0x7fbd08109a30 2024-04-25T13:57:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.723+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0806a160 con 0x7fbd08109a30 2024-04-25T13:57:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.724+0000 7fbcfeffd640 1 -- 172.21.15.78:0/1681563440 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbd0002fd30 con 0x7fbd08109a30 2024-04-25T13:57:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.725+0000 7fbcfeffd640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcf40afaf0 0x7fbcf40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.725+0000 7fbd0dd61640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcf40afaf0 0x7fbcf40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.725+0000 7fbcfeffd640 1 -- 172.21.15.78:0/1681563440 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fbd00100260 con 0x7fbd08109a30 2024-04-25T13:57:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.725+0000 7fbd0dd61640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcf40afaf0 0x7fbcf40b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd081046f0 tx=0x7fbcf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.728+0000 7fbcfeffd640 1 -- 172.21.15.78:0/1681563440 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd000c8f90 con 0x7fbd08109a30 2024-04-25T13:57:17.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.934+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbd08062fe0 con 0x7fbcf40afaf0 2024-04-25T13:57:17.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.939+0000 7fbcfeffd640 1 -- 172.21.15.78:0/1681563440 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbd08062fe0 con 0x7fbcf40afaf0 2024-04-25T13:57:17.939 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_1/ac7dbde1-25c1-4f0d-b6ec-2d78eebd8cd1 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcf40afaf0 msgr2=0x7fbcf40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcf40afaf0 0x7fbcf40b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd081046f0 tx=0x7fbcf8005f90 comp rx=0 tx=0).stop 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 msgr2=0x7fbd081a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd081a1650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd000097b0 tx=0x7fbd00031840 comp rx=0 tx=0).stop 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 shutdown_connections 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcf40afaf0 0x7fbcf40b2000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd08109a30 0x7fbd081a1650 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd08104d30 0x7fbd0819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 --2- 172.21.15.78:0/1681563440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd08103af0 0x7fbd0819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 >> 172.21.15.78:0/1681563440 conn(0x7fbd080fece0 msgr2=0x7fbd08105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 shutdown_connections 2024-04-25T13:57:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:17.941+0000 7fbd0ffc5640 1 -- 172.21.15.78:0/1681563440 wait complete. 2024-04-25T13:57:17.952 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 -- 172.21.15.78:0/599434938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac109870 msgr2=0x7f10ac10bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 --2- 172.21.15.78:0/599434938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac109870 0x7f10ac10bc90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009900 tx=0x7f10a402a410 comp rx=0 tx=0).stop 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 -- 172.21.15.78:0/599434938 shutdown_connections 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 --2- 172.21.15.78:0/599434938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac109870 0x7f10ac10bc90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 --2- 172.21.15.78:0/599434938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10ac106f10 0x7f10ac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 --2- 172.21.15.78:0/599434938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10ac1045b0 0x7f10ac1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 -- 172.21.15.78:0/599434938 >> 172.21.15.78:0/599434938 conn(0x7f10ac0fdbd0 msgr2=0x7f10ac100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 -- 172.21.15.78:0/599434938 shutdown_connections 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 -- 172.21.15.78:0/599434938 wait complete. 2024-04-25T13:57:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 Processor -- start 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.059+0000 7f10b38ee640 1 -- start start 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac1045b0 0x7f10ac19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10ac106f10 0x7f10ac19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10ac109870 0x7f10ac1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac10f5d0 con 0x7f10ac1045b0 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac10f450 con 0x7f10ac109870 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac10f750 con 0x7f10ac106f10 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac1045b0 0x7f10ac19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac1045b0 0x7f10ac19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53402/0 (socket says 172.21.15.78:53402) 2024-04-25T13:57:18.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 -- 172.21.15.78:0/371971048 learned_addr learned my addr 172.21.15.78:0/371971048 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b0e89640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10ac106f10 0x7f10ac19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 -- 172.21.15.78:0/371971048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10ac106f10 msgr2=0x7f10ac19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10ac106f10 0x7f10ac19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 -- 172.21.15.78:0/371971048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10ac109870 msgr2=0x7f10ac1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10ac109870 0x7f10ac1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 -- 172.21.15.78:0/371971048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac1a1ce0 con 0x7f10ac1045b0 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b0e89640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10ac106f10 0x7f10ac19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b168a640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac1045b0 0x7f10ac19a1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f10a800eb90 tx=0x7f10a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10a27fc640 1 -- 172.21.15.78:0/371971048 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8010630 con 0x7f10ac1045b0 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10a27fc640 1 -- 172.21.15.78:0/371971048 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a800ee90 con 0x7f10ac1045b0 2024-04-25T13:57:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac1a1fd0 con 0x7f10ac1045b0 2024-04-25T13:57:18.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10a27fc640 1 -- 172.21.15.78:0/371971048 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8018ac0 con 0x7f10ac1045b0 2024-04-25T13:57:18.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.060+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac1a2490 con 0x7f10ac1045b0 2024-04-25T13:57:18.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.061+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ac0ff6f0 con 0x7f10ac1045b0 2024-04-25T13:57:18.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.062+0000 7f10a27fc640 1 -- 172.21.15.78:0/371971048 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f10a8018c60 con 0x7f10ac1045b0 2024-04-25T13:57:18.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.063+0000 7f10a27fc640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10980afb70 0x7f10980b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.063+0000 7f10b0e89640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10980afb70 0x7f10980b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.063+0000 7f10a27fc640 1 -- 172.21.15.78:0/371971048 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f10a80df610 con 0x7f10ac1045b0 2024-04-25T13:57:18.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.064+0000 7f10b0e89640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10980afb70 0x7f10980b2080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f109c0058c0 tx=0x7f109c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.067+0000 7f10a27fc640 1 -- 172.21.15.78:0/371971048 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10a80a8340 con 0x7f10ac1045b0 2024-04-25T13:57:18.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:18 smithi078 ceph-mon[29163]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:18 smithi078 ceph-mon[29163]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:18 smithi078 ceph-mon[29163]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.284+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10ac02db50 con 0x7f10ac1045b0 2024-04-25T13:57:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.284+0000 7f10a27fc640 1 -- 172.21.15.78:0/371971048 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f10a80a7c40 con 0x7f10ac1045b0 2024-04-25T13:57:18.284 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:18.284 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:57:16.081259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24512,24484]},"id":1}]} 2024-04-25T13:57:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 21 2024-04-25T13:57:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10980afb70 msgr2=0x7f10980b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10980afb70 0x7f10980b2080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f109c0058c0 tx=0x7f109c005d50 comp rx=0 tx=0).stop 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac1045b0 msgr2=0x7f10ac19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac1045b0 0x7f10ac19a1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f10a800eb90 tx=0x7f10a800c6a0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 shutdown_connections 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10ac109870 0x7f10ac1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10ac106f10 0x7f10ac19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10980afb70 0x7f10980b2080 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 --2- 172.21.15.78:0/371971048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10ac1045b0 0x7f10ac19a1a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 >> 172.21.15.78:0/371971048 conn(0x7f10ac0fdbd0 msgr2=0x7f10ac105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 shutdown_connections 2024-04-25T13:57:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.286+0000 7f10b38ee640 1 -- 172.21.15.78:0/371971048 wait complete. 2024-04-25T13:57:18.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:57:18.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 -- 172.21.15.78:0/812761897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4103c20 msgr2=0x7f06b4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 --2- 172.21.15.78:0/812761897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4103c20 0x7f06b4104030 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009900 tx=0x7f06b002a410 comp rx=0 tx=0).stop 2024-04-25T13:57:18.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 -- 172.21.15.78:0/812761897 shutdown_connections 2024-04-25T13:57:18.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 --2- 172.21.15.78:0/812761897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b4108920 0x7f06b410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 --2- 172.21.15.78:0/812761897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4103c20 0x7f06b4104030 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 --2- 172.21.15.78:0/812761897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b41029e0 0x7f06b41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 -- 172.21.15.78:0/812761897 >> 172.21.15.78:0/812761897 conn(0x7f06b40fdbd0 msgr2=0x7f06b4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 -- 172.21.15.78:0/812761897 shutdown_connections 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.406+0000 7f06bb384640 1 -- 172.21.15.78:0/812761897 wait complete. 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 Processor -- start 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 -- start start 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b41029e0 0x7f06b419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b4103c20 0x7f06b419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4108920 0x7f06b41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06b410e560 con 0x7f06b4108920 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f06b410e3e0 con 0x7f06b41029e0 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06bb384640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06b410e6e0 con 0x7f06b4103c20 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4108920 0x7f06b41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4108920 0x7f06b41a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53420/0 (socket says 172.21.15.78:53420) 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 -- 172.21.15.78:0/585199612 learned_addr learned my addr 172.21.15.78:0/585199612 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:18.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b891f640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b4103c20 0x7f06b419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9120640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b41029e0 0x7f06b419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 -- 172.21.15.78:0/585199612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b4103c20 msgr2=0x7f06b419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b4103c20 0x7f06b419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 -- 172.21.15.78:0/585199612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b41029e0 msgr2=0x7f06b419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b41029e0 0x7f06b419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 -- 172.21.15.78:0/585199612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b41a1df0 con 0x7f06b4108920 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.407+0000 7f06b9921640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4108920 0x7f06b41a1780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00b970 tx=0x7f06ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.408+0000 7f06aa7fc640 1 -- 172.21.15.78:0/585199612 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac019070 con 0x7f06b4108920 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.408+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b41a20e0 con 0x7f06b4108920 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.408+0000 7f06aa7fc640 1 -- 172.21.15.78:0/585199612 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06ac014440 con 0x7f06b4108920 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.408+0000 7f06aa7fc640 1 -- 172.21.15.78:0/585199612 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0075d0 con 0x7f06b4108920 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.408+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b41a26b0 con 0x7f06b4108920 2024-04-25T13:57:18.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.408+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b41a2970 con 0x7f06b4108920 2024-04-25T13:57:18.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.410+0000 7f06aa7fc640 1 -- 172.21.15.78:0/585199612 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f06ac01c070 con 0x7f06b4108920 2024-04-25T13:57:18.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.411+0000 7f06aa7fc640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06a00afaf0 0x7f06a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.411+0000 7f06b9120640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06a00afaf0 0x7f06a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.411+0000 7f06aa7fc640 1 -- 172.21.15.78:0/585199612 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f06ac0de5f0 con 0x7f06b4108920 2024-04-25T13:57:18.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.411+0000 7f06b9120640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06a00afaf0 0x7f06a00b2000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f06b41035e0 tx=0x7f06a4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:18.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:18 smithi129 ceph-mon[32569]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:18 smithi129 ceph-mon[32569]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:18 smithi129 ceph-mon[32569]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.414+0000 7f06aa7fc640 1 -- 172.21.15.78:0/585199612 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06ac0a7320 con 0x7f06b4108920 2024-04-25T13:57:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:18 smithi112 ceph-mon[32617]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:18 smithi112 ceph-mon[32617]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:18 smithi112 ceph-mon[32617]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.592+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06b41a2b40 con 0x7f06b4108920 2024-04-25T13:57:18.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.594+0000 7f06aa7fc640 1 -- 172.21.15.78:0/585199612 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22243 (secure 0 0 0) 0x7f06ac0a6c20 con 0x7f06b4108920 2024-04-25T13:57:18.594 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:18.594 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:57:13.134773+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06a00afaf0 msgr2=0x7f06a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06a00afaf0 0x7f06a00b2000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f06b41035e0 tx=0x7f06a4005c60 comp rx=0 tx=0).stop 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4108920 msgr2=0x7f06b41a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4108920 0x7f06b41a1780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00b970 tx=0x7f06ac00be30 comp rx=0 tx=0).stop 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 shutdown_connections 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06a00afaf0 0x7f06a00b2000 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06b4108920 0x7f06b41a1780 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b4103c20 0x7f06b419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 --2- 172.21.15.78:0/585199612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06b41029e0 0x7f06b419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 >> 172.21.15.78:0/585199612 conn(0x7f06b40fdbd0 msgr2=0x7f06b4109f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:18.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 shutdown_connections 2024-04-25T13:57:18.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.596+0000 7f06bb384640 1 -- 172.21.15.78:0/585199612 wait complete. 2024-04-25T13:57:18.608 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:57:18.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:57:18.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 -- 172.21.15.78:0/1705262684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b81033c0 msgr2=0x7f57b81037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 --2- 172.21.15.78:0/1705262684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b81033c0 0x7f57b81037d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009900 tx=0x7f57ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:18.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 -- 172.21.15.78:0/1705262684 shutdown_connections 2024-04-25T13:57:18.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 --2- 172.21.15.78:0/1705262684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b81080c0 0x7f57b810a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 --2- 172.21.15.78:0/1705262684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b81033c0 0x7f57b81037d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 --2- 172.21.15.78:0/1705262684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8102180 0x7f57b8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 -- 172.21.15.78:0/1705262684 >> 172.21.15.78:0/1705262684 conn(0x7f57b80fd370 msgr2=0x7f57b80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 -- 172.21.15.78:0/1705262684 shutdown_connections 2024-04-25T13:57:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.715+0000 7f57bd249640 1 -- 172.21.15.78:0/1705262684 wait complete. 2024-04-25T13:57:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 Processor -- start 2024-04-25T13:57:18.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 -- start start 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b8102180 0x7f57b810ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b81033c0 0x7f57b819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b81080c0 0x7f57b81a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810e520 con 0x7f57b8102180 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810e3a0 con 0x7f57b81033c0 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57bd249640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810e6a0 con 0x7f57b81080c0 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b8102180 0x7f57b810ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b8102180 0x7f57b810ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53434/0 (socket says 172.21.15.78:53434) 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b6d9d640 1 -- 172.21.15.78:0/2631166418 learned_addr learned my addr 172.21.15.78:0/2631166418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b759e640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b81080c0 0x7f57b81a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b659c640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b81033c0 0x7f57b819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b6d9d640 1 -- 172.21.15.78:0/2631166418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b81080c0 msgr2=0x7f57b81a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b6d9d640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b81080c0 0x7f57b81a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b6d9d640 1 -- 172.21.15.78:0/2631166418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b81033c0 msgr2=0x7f57b819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.716+0000 7f57b6d9d640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b81033c0 0x7f57b819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f57b6d9d640 1 -- 172.21.15.78:0/2631166418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b81a1c50 con 0x7f57b8102180 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f57b659c640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b81033c0 0x7f57b819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f57b6d9d640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b8102180 0x7f57b810ba40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c970 tx=0x7f57a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f579ffff640 1 -- 172.21.15.78:0/2631166418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8007c70 con 0x7f57b8102180 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f579ffff640 1 -- 172.21.15.78:0/2631166418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a8007e10 con 0x7f57b8102180 2024-04-25T13:57:18.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b81a1f40 con 0x7f57b8102180 2024-04-25T13:57:18.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f579ffff640 1 -- 172.21.15.78:0/2631166418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a80187b0 con 0x7f57b8102180 2024-04-25T13:57:18.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.717+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81a23a0 con 0x7f57b8102180 2024-04-25T13:57:18.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.718+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b806a1c0 con 0x7f57b8102180 2024-04-25T13:57:18.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.720+0000 7f579ffff640 1 -- 172.21.15.78:0/2631166418 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f57a800fab0 con 0x7f57b8102180 2024-04-25T13:57:18.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.721+0000 7f579ffff640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f57a00afaf0 0x7f57a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:18.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.721+0000 7f57b659c640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f57a00afaf0 0x7f57a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:18.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.721+0000 7f579ffff640 1 -- 172.21.15.78:0/2631166418 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f57a80ded90 con 0x7f57b8102180 2024-04-25T13:57:18.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.721+0000 7f57b659c640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f57a00afaf0 0x7f57a00b2000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f57ac006010 tx=0x7f57ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:18.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.724+0000 7f579ffff640 1 -- 172.21.15.78:0/2631166418 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57a8053ff0 con 0x7f57b8102180 2024-04-25T13:57:18.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.901+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57b8062fe0 con 0x7f57a00afaf0 2024-04-25T13:57:18.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.908+0000 7f579ffff640 1 -- 172.21.15.78:0/2631166418 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258870 (secure 0 0 0) 0x7f57b8062fe0 con 0x7f57a00afaf0 2024-04-25T13:57:18.912 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:18.913 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.911+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f57a00afaf0 msgr2=0x7f57a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.911+0000 7f57bd249640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f57a00afaf0 0x7f57a00b2000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f57ac006010 tx=0x7f57ac000bc0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.911+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b8102180 msgr2=0x7f57b810ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.911+0000 7f57bd249640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b8102180 0x7f57b810ba40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c970 tx=0x7f57a800ce30 comp rx=0 tx=0).stop 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 shutdown_connections 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b81080c0 0x7f57b81a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b81033c0 0x7f57b819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f57a00afaf0 0x7f57a00b2000 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 --2- 172.21.15.78:0/2631166418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b8102180 0x7f57b810ba40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 >> 172.21.15.78:0/2631166418 conn(0x7f57b80fd370 msgr2=0x7f57b81040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 shutdown_connections 2024-04-25T13:57:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:18.912+0000 7f57bd249640 1 -- 172.21.15.78:0/2631166418 wait complete. 2024-04-25T13:57:18.923 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-25T13:57:18.126957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596148,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":98,"num_read_kb":81,"num_write":216,"num_write_kb":2180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":174,"ondisk_log_size":174,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":250,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371356,"kb_used_data":9676,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124570468,"statfs":{"total":1151940427776,"available":1151560159232,"internally_reserved":0,"allocated":9908224,"data_stored":6398889,"data_compressed":107514,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5780,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.005032"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143214+0000","last_change":"2024-04-25T13:57:12.147384+0000","last_active":"2024-04-25T13:57:13.143214+0000","last_peered":"2024-04-25T13:57:13.143214+0000","last_clean":"2024-04-25T13:57:13.143214+0000","last_became_active":"2024-04-25T13:57:12.146349+0000","last_became_peered":"2024-04-25T13:57:12.146349+0000","last_unstale":"2024-04-25T13:57:13.143214+0000","last_undegraded":"2024-04-25T13:57:13.143214+0000","last_fullsized":"2024-04-25T13:57:13.143214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841462+0000","last_change":"2024-04-25T13:57:12.156698+0000","last_active":"2024-04-25T13:57:13.841462+0000","last_peered":"2024-04-25T13:57:13.841462+0000","last_clean":"2024-04-25T13:57:13.841462+0000","last_became_active":"2024-04-25T13:57:12.155790+0000","last_became_peered":"2024-04-25T13:57:12.155790+0000","last_unstale":"2024-04-25T13:57:13.841462+0000","last_undegraded":"2024-04-25T13:57:13.841462+0000","last_fullsized":"2024-04-25T13:57:13.841462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:25.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841300+0000","last_change":"2024-04-25T13:57:12.149458+0000","last_active":"2024-04-25T13:57:13.841300+0000","last_peered":"2024-04-25T13:57:13.841300+0000","last_clean":"2024-04-25T13:57:13.841300+0000","last_became_active":"2024-04-25T13:57:12.148179+0000","last_became_peered":"2024-04-25T13:57:12.148179+0000","last_unstale":"2024-04-25T13:57:13.841300+0000","last_undegraded":"2024-04-25T13:57:13.841300+0000","last_fullsized":"2024-04-25T13:57:13.841300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:00.502798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.842074+0000","last_change":"2024-04-25T13:57:12.145776+0000","last_active":"2024-04-25T13:57:13.842074+0000","last_peered":"2024-04-25T13:57:13.842074+0000","last_clean":"2024-04-25T13:57:13.842074+0000","last_became_active":"2024-04-25T13:57:12.145038+0000","last_became_peered":"2024-04-25T13:57:12.145038+0000","last_unstale":"2024-04-25T13:57:13.842074+0000","last_undegraded":"2024-04-25T13:57:13.842074+0000","last_fullsized":"2024-04-25T13:57:13.842074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:52.742746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.161955+0000","last_change":"2024-04-25T13:57:12.160273+0000","last_active":"2024-04-25T13:57:12.161955+0000","last_peered":"2024-04-25T13:57:12.161955+0000","last_clean":"2024-04-25T13:57:12.161955+0000","last_became_active":"2024-04-25T13:57:12.159805+0000","last_became_peered":"2024-04-25T13:57:12.159805+0000","last_unstale":"2024-04-25T13:57:12.161955+0000","last_undegraded":"2024-04-25T13:57:12.161955+0000","last_fullsized":"2024-04-25T13:57:12.161955+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.188107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143584+0000","last_change":"2024-04-25T13:57:12.151597+0000","last_active":"2024-04-25T13:57:13.143584+0000","last_peered":"2024-04-25T13:57:13.143584+0000","last_clean":"2024-04-25T13:57:13.143584+0000","last_became_active":"2024-04-25T13:57:12.151203+0000","last_became_peered":"2024-04-25T13:57:12.151203+0000","last_unstale":"2024-04-25T13:57:13.143584+0000","last_undegraded":"2024-04-25T13:57:13.143584+0000","last_fullsized":"2024-04-25T13:57:13.143584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.645264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326406+0000","last_change":"2024-04-25T13:57:12.144959+0000","last_active":"2024-04-25T13:57:13.326406+0000","last_peered":"2024-04-25T13:57:13.326406+0000","last_clean":"2024-04-25T13:57:13.326406+0000","last_became_active":"2024-04-25T13:57:12.144587+0000","last_became_peered":"2024-04-25T13:57:12.144587+0000","last_unstale":"2024-04-25T13:57:13.326406+0000","last_undegraded":"2024-04-25T13:57:13.326406+0000","last_fullsized":"2024-04-25T13:57:13.326406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:45.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213101+0000","last_change":"2024-04-25T13:57:12.157016+0000","last_active":"2024-04-25T13:57:13.213101+0000","last_peered":"2024-04-25T13:57:13.213101+0000","last_clean":"2024-04-25T13:57:13.213101+0000","last_became_active":"2024-04-25T13:57:12.156209+0000","last_became_peered":"2024-04-25T13:57:12.156209+0000","last_unstale":"2024-04-25T13:57:13.213101+0000","last_undegraded":"2024-04-25T13:57:13.213101+0000","last_fullsized":"2024-04-25T13:57:13.213101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841925+0000","last_change":"2024-04-25T13:57:12.146696+0000","last_active":"2024-04-25T13:57:13.841925+0000","last_peered":"2024-04-25T13:57:13.841925+0000","last_clean":"2024-04-25T13:57:13.841925+0000","last_became_active":"2024-04-25T13:57:12.146317+0000","last_became_peered":"2024-04-25T13:57:12.146317+0000","last_unstale":"2024-04-25T13:57:13.841925+0000","last_undegraded":"2024-04-25T13:57:13.841925+0000","last_fullsized":"2024-04-25T13:57:13.841925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.078941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.144011+0000","last_change":"2024-04-25T13:57:12.155100+0000","last_active":"2024-04-25T13:57:13.144011+0000","last_peered":"2024-04-25T13:57:13.144011+0000","last_clean":"2024-04-25T13:57:13.144011+0000","last_became_active":"2024-04-25T13:57:12.154602+0000","last_became_peered":"2024-04-25T13:57:12.154602+0000","last_unstale":"2024-04-25T13:57:13.144011+0000","last_undegraded":"2024-04-25T13:57:13.144011+0000","last_fullsized":"2024-04-25T13:57:13.144011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.837117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326293+0000","last_change":"2024-04-25T13:57:12.149386+0000","last_active":"2024-04-25T13:57:13.326293+0000","last_peered":"2024-04-25T13:57:13.326293+0000","last_clean":"2024-04-25T13:57:13.326293+0000","last_became_active":"2024-04-25T13:57:12.148710+0000","last_became_peered":"2024-04-25T13:57:12.148710+0000","last_unstale":"2024-04-25T13:57:13.326293+0000","last_undegraded":"2024-04-25T13:57:13.326293+0000","last_fullsized":"2024-04-25T13:57:13.326293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:17.655260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.842039+0000","last_change":"2024-04-25T13:57:12.156103+0000","last_active":"2024-04-25T13:57:13.842039+0000","last_peered":"2024-04-25T13:57:13.842039+0000","last_clean":"2024-04-25T13:57:13.842039+0000","last_became_active":"2024-04-25T13:57:12.155738+0000","last_became_peered":"2024-04-25T13:57:12.155738+0000","last_unstale":"2024-04-25T13:57:13.842039+0000","last_undegraded":"2024-04-25T13:57:13.842039+0000","last_fullsized":"2024-04-25T13:57:13.842039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:38.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.155552+0000","last_change":"2024-04-25T13:57:12.154873+0000","last_active":"2024-04-25T13:57:12.155552+0000","last_peered":"2024-04-25T13:57:12.155552+0000","last_clean":"2024-04-25T13:57:12.155552+0000","last_became_active":"2024-04-25T13:57:12.154523+0000","last_became_peered":"2024-04-25T13:57:12.154523+0000","last_unstale":"2024-04-25T13:57:12.155552+0000","last_undegraded":"2024-04-25T13:57:12.155552+0000","last_fullsized":"2024-04-25T13:57:12.155552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:36.943259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841274+0000","last_change":"2024-04-25T13:57:12.156536+0000","last_active":"2024-04-25T13:57:13.841274+0000","last_peered":"2024-04-25T13:57:13.841274+0000","last_clean":"2024-04-25T13:57:13.841274+0000","last_became_active":"2024-04-25T13:57:12.156143+0000","last_became_peered":"2024-04-25T13:57:12.156143+0000","last_unstale":"2024-04-25T13:57:13.841274+0000","last_undegraded":"2024-04-25T13:57:13.841274+0000","last_fullsized":"2024-04-25T13:57:13.841274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:35.210059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841024+0000","last_change":"2024-04-25T13:57:12.144477+0000","last_active":"2024-04-25T13:57:13.841024+0000","last_peered":"2024-04-25T13:57:13.841024+0000","last_clean":"2024-04-25T13:57:13.841024+0000","last_became_active":"2024-04-25T13:57:12.144115+0000","last_became_peered":"2024-04-25T13:57:12.144115+0000","last_unstale":"2024-04-25T13:57:13.841024+0000","last_undegraded":"2024-04-25T13:57:13.841024+0000","last_fullsized":"2024-04-25T13:57:13.841024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:30.733713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841440+0000","last_change":"2024-04-25T13:57:12.148837+0000","last_active":"2024-04-25T13:57:13.841440+0000","last_peered":"2024-04-25T13:57:13.841440+0000","last_clean":"2024-04-25T13:57:13.841440+0000","last_became_active":"2024-04-25T13:57:12.148494+0000","last_became_peered":"2024-04-25T13:57:12.148494+0000","last_unstale":"2024-04-25T13:57:13.841440+0000","last_undegraded":"2024-04-25T13:57:13.841440+0000","last_fullsized":"2024-04-25T13:57:13.841440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:50.414344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143368+0000","last_change":"2024-04-25T13:57:12.148142+0000","last_active":"2024-04-25T13:57:13.143368+0000","last_peered":"2024-04-25T13:57:13.143368+0000","last_clean":"2024-04-25T13:57:13.143368+0000","last_became_active":"2024-04-25T13:57:12.147738+0000","last_became_peered":"2024-04-25T13:57:12.147738+0000","last_unstale":"2024-04-25T13:57:13.143368+0000","last_undegraded":"2024-04-25T13:57:13.143368+0000","last_fullsized":"2024-04-25T13:57:13.143368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:42.014605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142564+0000","last_change":"2024-04-25T13:57:12.160059+0000","last_active":"2024-04-25T13:57:13.142564+0000","last_peered":"2024-04-25T13:57:13.142564+0000","last_clean":"2024-04-25T13:57:13.142564+0000","last_became_active":"2024-04-25T13:57:12.159685+0000","last_became_peered":"2024-04-25T13:57:12.159685+0000","last_unstale":"2024-04-25T13:57:13.142564+0000","last_undegraded":"2024-04-25T13:57:13.142564+0000","last_fullsized":"2024-04-25T13:57:13.142564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:48.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786401+0000","last_change":"2024-04-25T13:57:12.157950+0000","last_active":"2024-04-25T13:57:13.786401+0000","last_peered":"2024-04-25T13:57:13.786401+0000","last_clean":"2024-04-25T13:57:13.786401+0000","last_became_active":"2024-04-25T13:57:12.157612+0000","last_became_peered":"2024-04-25T13:57:12.157612+0000","last_unstale":"2024-04-25T13:57:13.786401+0000","last_undegraded":"2024-04-25T13:57:13.786401+0000","last_fullsized":"2024-04-25T13:57:13.786401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:55.521381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"74'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142680+0000","last_change":"2024-04-25T13:57:11.143834+0000","last_active":"2024-04-25T13:57:13.142680+0000","last_peered":"2024-04-25T13:57:13.142680+0000","last_clean":"2024-04-25T13:57:13.142680+0000","last_became_active":"2024-04-25T13:57:11.143474+0000","last_became_peered":"2024-04-25T13:57:11.143474+0000","last_unstale":"2024-04-25T13:57:13.142680+0000","last_undegraded":"2024-04-25T13:57:13.142680+0000","last_fullsized":"2024-04-25T13:57:13.142680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.137616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213461+0000","last_change":"2024-04-25T13:57:12.151301+0000","last_active":"2024-04-25T13:57:13.213461+0000","last_peered":"2024-04-25T13:57:13.213461+0000","last_clean":"2024-04-25T13:57:13.213461+0000","last_became_active":"2024-04-25T13:57:12.150941+0000","last_became_peered":"2024-04-25T13:57:12.150941+0000","last_unstale":"2024-04-25T13:57:13.213461+0000","last_undegraded":"2024-04-25T13:57:13.213461+0000","last_fullsized":"2024-04-25T13:57:13.213461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:22.259683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213734+0000","last_change":"2024-04-25T13:57:11.147314+0000","last_active":"2024-04-25T13:57:13.213734+0000","last_peered":"2024-04-25T13:57:13.213734+0000","last_clean":"2024-04-25T13:57:13.213734+0000","last_became_active":"2024-04-25T13:57:11.146894+0000","last_became_peered":"2024-04-25T13:57:11.146894+0000","last_unstale":"2024-04-25T13:57:13.213734+0000","last_undegraded":"2024-04-25T13:57:13.213734+0000","last_fullsized":"2024-04-25T13:57:13.213734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:55.927007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841629+0000","last_change":"2024-04-25T13:57:12.149186+0000","last_active":"2024-04-25T13:57:13.841629+0000","last_peered":"2024-04-25T13:57:13.841629+0000","last_clean":"2024-04-25T13:57:13.841629+0000","last_became_active":"2024-04-25T13:57:12.148155+0000","last_became_peered":"2024-04-25T13:57:12.148155+0000","last_unstale":"2024-04-25T13:57:13.841629+0000","last_undegraded":"2024-04-25T13:57:13.841629+0000","last_fullsized":"2024-04-25T13:57:13.841629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162338+0000","last_change":"2024-04-25T13:57:11.140086+0000","last_active":"2024-04-25T13:57:12.162338+0000","last_peered":"2024-04-25T13:57:12.162338+0000","last_clean":"2024-04-25T13:57:12.162338+0000","last_became_active":"2024-04-25T13:57:11.139751+0000","last_became_peered":"2024-04-25T13:57:11.139751+0000","last_unstale":"2024-04-25T13:57:12.162338+0000","last_undegraded":"2024-04-25T13:57:12.162338+0000","last_fullsized":"2024-04-25T13:57:12.162338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.552503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841178+0000","last_change":"2024-04-25T13:57:12.157029+0000","last_active":"2024-04-25T13:57:13.841178+0000","last_peered":"2024-04-25T13:57:13.841178+0000","last_clean":"2024-04-25T13:57:13.841178+0000","last_became_active":"2024-04-25T13:57:12.156668+0000","last_became_peered":"2024-04-25T13:57:12.156668+0000","last_unstale":"2024-04-25T13:57:13.841178+0000","last_undegraded":"2024-04-25T13:57:13.841178+0000","last_fullsized":"2024-04-25T13:57:13.841178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162127+0000","last_change":"2024-04-25T13:57:11.140592+0000","last_active":"2024-04-25T13:57:12.162127+0000","last_peered":"2024-04-25T13:57:12.162127+0000","last_clean":"2024-04-25T13:57:12.162127+0000","last_became_active":"2024-04-25T13:57:11.140244+0000","last_became_peered":"2024-04-25T13:57:11.140244+0000","last_unstale":"2024-04-25T13:57:12.162127+0000","last_undegraded":"2024-04-25T13:57:12.162127+0000","last_fullsized":"2024-04-25T13:57:12.162127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:53.071092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841929+0000","last_change":"2024-04-25T13:57:12.148593+0000","last_active":"2024-04-25T13:57:13.841929+0000","last_peered":"2024-04-25T13:57:13.841929+0000","last_clean":"2024-04-25T13:57:13.841929+0000","last_became_active":"2024-04-25T13:57:12.148003+0000","last_became_peered":"2024-04-25T13:57:12.148003+0000","last_unstale":"2024-04-25T13:57:13.841929+0000","last_undegraded":"2024-04-25T13:57:13.841929+0000","last_fullsized":"2024-04-25T13:57:13.841929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:58.924413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:17.083034+0000","last_change":"2024-04-25T13:57:11.140191+0000","last_active":"2024-04-25T13:57:17.083034+0000","last_peered":"2024-04-25T13:57:17.083034+0000","last_clean":"2024-04-25T13:57:17.083034+0000","last_became_active":"2024-04-25T13:57:11.137935+0000","last_became_peered":"2024-04-25T13:57:11.137935+0000","last_unstale":"2024-04-25T13:57:17.083034+0000","last_undegraded":"2024-04-25T13:57:17.083034+0000","last_fullsized":"2024-04-25T13:57:17.083034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.642996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.149620+0000","last_change":"2024-04-25T13:57:12.148787+0000","last_active":"2024-04-25T13:57:12.149620+0000","last_peered":"2024-04-25T13:57:12.149620+0000","last_clean":"2024-04-25T13:57:12.149620+0000","last_became_active":"2024-04-25T13:57:12.148424+0000","last_became_peered":"2024-04-25T13:57:12.148424+0000","last_unstale":"2024-04-25T13:57:12.149620+0000","last_undegraded":"2024-04-25T13:57:12.149620+0000","last_fullsized":"2024-04-25T13:57:12.149620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.154058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.087067+0000","last_change":"2024-04-25T13:57:11.142424+0000","last_active":"2024-04-25T13:57:15.087067+0000","last_peered":"2024-04-25T13:57:15.087067+0000","last_clean":"2024-04-25T13:57:15.087067+0000","last_became_active":"2024-04-25T13:57:11.142047+0000","last_became_peered":"2024-04-25T13:57:11.142047+0000","last_unstale":"2024-04-25T13:57:15.087067+0000","last_undegraded":"2024-04-25T13:57:15.087067+0000","last_fullsized":"2024-04-25T13:57:15.087067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:13.455149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.842013+0000","last_change":"2024-04-25T13:57:12.155940+0000","last_active":"2024-04-25T13:57:13.842013+0000","last_peered":"2024-04-25T13:57:13.842013+0000","last_clean":"2024-04-25T13:57:13.842013+0000","last_became_active":"2024-04-25T13:57:12.155503+0000","last_became_peered":"2024-04-25T13:57:12.155503+0000","last_unstale":"2024-04-25T13:57:13.842013+0000","last_undegraded":"2024-04-25T13:57:13.842013+0000","last_fullsized":"2024-04-25T13:57:13.842013+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:48.345437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.127248+0000","last_change":"2024-04-25T13:57:11.131160+0000","last_active":"2024-04-25T13:57:12.127248+0000","last_peered":"2024-04-25T13:57:12.127248+0000","last_clean":"2024-04-25T13:57:12.127248+0000","last_became_active":"2024-04-25T13:57:11.130753+0000","last_became_peered":"2024-04-25T13:57:11.130753+0000","last_unstale":"2024-04-25T13:57:12.127248+0000","last_undegraded":"2024-04-25T13:57:12.127248+0000","last_fullsized":"2024-04-25T13:57:12.127248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:48.297546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213163+0000","last_change":"2024-04-25T13:57:12.155779+0000","last_active":"2024-04-25T13:57:13.213163+0000","last_peered":"2024-04-25T13:57:13.213163+0000","last_clean":"2024-04-25T13:57:13.213163+0000","last_became_active":"2024-04-25T13:57:12.154682+0000","last_became_peered":"2024-04-25T13:57:12.154682+0000","last_unstale":"2024-04-25T13:57:13.213163+0000","last_undegraded":"2024-04-25T13:57:13.213163+0000","last_fullsized":"2024-04-25T13:57:13.213163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:20.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841536+0000","last_change":"2024-04-25T13:57:11.140211+0000","last_active":"2024-04-25T13:57:13.841536+0000","last_peered":"2024-04-25T13:57:13.841536+0000","last_clean":"2024-04-25T13:57:13.841536+0000","last_became_active":"2024-04-25T13:57:11.139891+0000","last_became_peered":"2024-04-25T13:57:11.139891+0000","last_unstale":"2024-04-25T13:57:13.841536+0000","last_undegraded":"2024-04-25T13:57:13.841536+0000","last_fullsized":"2024-04-25T13:57:13.841536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:21.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142467+0000","last_change":"2024-04-25T13:57:12.153886+0000","last_active":"2024-04-25T13:57:13.142467+0000","last_peered":"2024-04-25T13:57:13.142467+0000","last_clean":"2024-04-25T13:57:13.142467+0000","last_became_active":"2024-04-25T13:57:12.153500+0000","last_became_peered":"2024-04-25T13:57:12.153500+0000","last_unstale":"2024-04-25T13:57:13.142467+0000","last_undegraded":"2024-04-25T13:57:13.142467+0000","last_fullsized":"2024-04-25T13:57:13.142467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:53.549745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'3","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168548+0000","last_change":"2024-04-25T13:57:11.138829+0000","last_active":"2024-04-25T13:57:14.168548+0000","last_peered":"2024-04-25T13:57:14.168548+0000","last_clean":"2024-04-25T13:57:14.168548+0000","last_became_active":"2024-04-25T13:57:11.138461+0000","last_became_peered":"2024-04-25T13:57:11.138461+0000","last_unstale":"2024-04-25T13:57:14.168548+0000","last_undegraded":"2024-04-25T13:57:14.168548+0000","last_fullsized":"2024-04-25T13:57:14.168548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.134061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841336+0000","last_change":"2024-04-25T13:57:12.157394+0000","last_active":"2024-04-25T13:57:13.841336+0000","last_peered":"2024-04-25T13:57:13.841336+0000","last_clean":"2024-04-25T13:57:13.841336+0000","last_became_active":"2024-04-25T13:57:12.156408+0000","last_became_peered":"2024-04-25T13:57:12.156408+0000","last_unstale":"2024-04-25T13:57:13.841336+0000","last_undegraded":"2024-04-25T13:57:13.841336+0000","last_fullsized":"2024-04-25T13:57:13.841336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:36.741367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168755+0000","last_change":"2024-04-25T13:57:11.142846+0000","last_active":"2024-04-25T13:57:14.168755+0000","last_peered":"2024-04-25T13:57:14.168755+0000","last_clean":"2024-04-25T13:57:14.168755+0000","last_became_active":"2024-04-25T13:57:11.142387+0000","last_became_peered":"2024-04-25T13:57:11.142387+0000","last_unstale":"2024-04-25T13:57:14.168755+0000","last_undegraded":"2024-04-25T13:57:14.168755+0000","last_fullsized":"2024-04-25T13:57:14.168755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.164994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841644+0000","last_change":"2024-04-25T13:57:12.145617+0000","last_active":"2024-04-25T13:57:13.841644+0000","last_peered":"2024-04-25T13:57:13.841644+0000","last_clean":"2024-04-25T13:57:13.841644+0000","last_became_active":"2024-04-25T13:57:12.145263+0000","last_became_peered":"2024-04-25T13:57:12.145263+0000","last_unstale":"2024-04-25T13:57:13.841644+0000","last_undegraded":"2024-04-25T13:57:13.841644+0000","last_fullsized":"2024-04-25T13:57:13.841644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.886538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168886+0000","last_change":"2024-04-25T13:57:11.136972+0000","last_active":"2024-04-25T13:57:14.168886+0000","last_peered":"2024-04-25T13:57:14.168886+0000","last_clean":"2024-04-25T13:57:14.168886+0000","last_became_active":"2024-04-25T13:57:11.136392+0000","last_became_peered":"2024-04-25T13:57:11.136392+0000","last_unstale":"2024-04-25T13:57:14.168886+0000","last_undegraded":"2024-04-25T13:57:14.168886+0000","last_fullsized":"2024-04-25T13:57:14.168886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:45.841670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841761+0000","last_change":"2024-04-25T13:57:12.144905+0000","last_active":"2024-04-25T13:57:13.841761+0000","last_peered":"2024-04-25T13:57:13.841761+0000","last_clean":"2024-04-25T13:57:13.841761+0000","last_became_active":"2024-04-25T13:57:12.144404+0000","last_became_peered":"2024-04-25T13:57:12.144404+0000","last_unstale":"2024-04-25T13:57:13.841761+0000","last_undegraded":"2024-04-25T13:57:13.841761+0000","last_fullsized":"2024-04-25T13:57:13.841761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.186492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.078155+0000","last_change":"2024-04-25T13:57:11.139041+0000","last_active":"2024-04-25T13:57:15.078155+0000","last_peered":"2024-04-25T13:57:15.078155+0000","last_clean":"2024-04-25T13:57:15.078155+0000","last_became_active":"2024-04-25T13:57:11.138629+0000","last_became_peered":"2024-04-25T13:57:11.138629+0000","last_unstale":"2024-04-25T13:57:15.078155+0000","last_undegraded":"2024-04-25T13:57:15.078155+0000","last_fullsized":"2024-04-25T13:57:15.078155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:04.863015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162425+0000","last_change":"2024-04-25T13:57:12.160897+0000","last_active":"2024-04-25T13:57:12.162425+0000","last_peered":"2024-04-25T13:57:12.162425+0000","last_clean":"2024-04-25T13:57:12.162425+0000","last_became_active":"2024-04-25T13:57:12.160322+0000","last_became_peered":"2024-04-25T13:57:12.160322+0000","last_unstale":"2024-04-25T13:57:12.162425+0000","last_undegraded":"2024-04-25T13:57:12.162425+0000","last_fullsized":"2024-04-25T13:57:12.162425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:56.159435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.088199+0000","last_change":"2024-04-25T13:57:11.137104+0000","last_active":"2024-04-25T13:57:15.088199+0000","last_peered":"2024-04-25T13:57:15.088199+0000","last_clean":"2024-04-25T13:57:15.088199+0000","last_became_active":"2024-04-25T13:57:11.136720+0000","last_became_peered":"2024-04-25T13:57:11.136720+0000","last_unstale":"2024-04-25T13:57:15.088199+0000","last_undegraded":"2024-04-25T13:57:15.088199+0000","last_fullsized":"2024-04-25T13:57:15.088199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:41.262776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.148506+0000","last_change":"2024-04-25T13:57:12.147831+0000","last_active":"2024-04-25T13:57:12.148506+0000","last_peered":"2024-04-25T13:57:12.148506+0000","last_clean":"2024-04-25T13:57:12.148506+0000","last_became_active":"2024-04-25T13:57:12.147270+0000","last_became_peered":"2024-04-25T13:57:12.147270+0000","last_unstale":"2024-04-25T13:57:12.148506+0000","last_undegraded":"2024-04-25T13:57:12.148506+0000","last_fullsized":"2024-04-25T13:57:12.148506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:22.786101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168873+0000","last_change":"2024-04-25T13:57:11.138254+0000","last_active":"2024-04-25T13:57:14.168873+0000","last_peered":"2024-04-25T13:57:14.168873+0000","last_clean":"2024-04-25T13:57:14.168873+0000","last_became_active":"2024-04-25T13:57:11.137924+0000","last_became_peered":"2024-04-25T13:57:11.137924+0000","last_unstale":"2024-04-25T13:57:14.168873+0000","last_undegraded":"2024-04-25T13:57:14.168873+0000","last_fullsized":"2024-04-25T13:57:14.168873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:27.746203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326437+0000","last_change":"2024-04-25T13:57:12.145743+0000","last_active":"2024-04-25T13:57:13.326437+0000","last_peered":"2024-04-25T13:57:13.326437+0000","last_clean":"2024-04-25T13:57:13.326437+0000","last_became_active":"2024-04-25T13:57:12.145337+0000","last_became_peered":"2024-04-25T13:57:12.145337+0000","last_unstale":"2024-04-25T13:57:13.326437+0000","last_undegraded":"2024-04-25T13:57:13.326437+0000","last_fullsized":"2024-04-25T13:57:13.326437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.692900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'4","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:16.090779+0000","last_change":"2024-04-25T13:57:11.138059+0000","last_active":"2024-04-25T13:57:16.090779+0000","last_peered":"2024-04-25T13:57:16.090779+0000","last_clean":"2024-04-25T13:57:16.090779+0000","last_became_active":"2024-04-25T13:57:11.137698+0000","last_became_peered":"2024-04-25T13:57:11.137698+0000","last_unstale":"2024-04-25T13:57:16.090779+0000","last_undegraded":"2024-04-25T13:57:16.090779+0000","last_fullsized":"2024-04-25T13:57:16.090779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:03.595126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326123+0000","last_change":"2024-04-25T13:57:12.155507+0000","last_active":"2024-04-25T13:57:13.326123+0000","last_peered":"2024-04-25T13:57:13.326123+0000","last_clean":"2024-04-25T13:57:13.326123+0000","last_became_active":"2024-04-25T13:57:12.155175+0000","last_became_peered":"2024-04-25T13:57:12.155175+0000","last_unstale":"2024-04-25T13:57:13.326123+0000","last_undegraded":"2024-04-25T13:57:13.326123+0000","last_fullsized":"2024-04-25T13:57:13.326123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:21.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.163736+0000","last_change":"2024-04-25T13:57:11.141672+0000","last_active":"2024-04-25T13:57:12.163736+0000","last_peered":"2024-04-25T13:57:12.163736+0000","last_clean":"2024-04-25T13:57:12.163736+0000","last_became_active":"2024-04-25T13:57:11.140957+0000","last_became_peered":"2024-04-25T13:57:11.140957+0000","last_unstale":"2024-04-25T13:57:12.163736+0000","last_undegraded":"2024-04-25T13:57:12.163736+0000","last_fullsized":"2024-04-25T13:57:12.163736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:47.362949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841579+0000","last_change":"2024-04-25T13:57:12.147881+0000","last_active":"2024-04-25T13:57:13.841579+0000","last_peered":"2024-04-25T13:57:13.841579+0000","last_clean":"2024-04-25T13:57:13.841579+0000","last_became_active":"2024-04-25T13:57:12.146200+0000","last_became_peered":"2024-04-25T13:57:12.146200+0000","last_unstale":"2024-04-25T13:57:13.841579+0000","last_undegraded":"2024-04-25T13:57:13.841579+0000","last_fullsized":"2024-04-25T13:57:13.841579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:51.711045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.160439+0000","last_change":"2024-04-25T13:57:11.139574+0000","last_active":"2024-04-25T13:57:12.160439+0000","last_peered":"2024-04-25T13:57:12.160439+0000","last_clean":"2024-04-25T13:57:12.160439+0000","last_became_active":"2024-04-25T13:57:11.139232+0000","last_became_peered":"2024-04-25T13:57:11.139232+0000","last_unstale":"2024-04-25T13:57:12.160439+0000","last_undegraded":"2024-04-25T13:57:12.160439+0000","last_fullsized":"2024-04-25T13:57:12.160439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:08.590270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143130+0000","last_change":"2024-04-25T13:57:12.147122+0000","last_active":"2024-04-25T13:57:13.143130+0000","last_peered":"2024-04-25T13:57:13.143130+0000","last_clean":"2024-04-25T13:57:13.143130+0000","last_became_active":"2024-04-25T13:57:12.146708+0000","last_became_peered":"2024-04-25T13:57:12.146708+0000","last_unstale":"2024-04-25T13:57:13.143130+0000","last_undegraded":"2024-04-25T13:57:13.143130+0000","last_fullsized":"2024-04-25T13:57:13.143130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:47.858290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142519+0000","last_change":"2024-04-25T13:57:11.146288+0000","last_active":"2024-04-25T13:57:13.142519+0000","last_peered":"2024-04-25T13:57:13.142519+0000","last_clean":"2024-04-25T13:57:13.142519+0000","last_became_active":"2024-04-25T13:57:11.145851+0000","last_became_peered":"2024-04-25T13:57:11.145851+0000","last_unstale":"2024-04-25T13:57:13.142519+0000","last_undegraded":"2024-04-25T13:57:13.142519+0000","last_fullsized":"2024-04-25T13:57:13.142519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143855+0000","last_change":"2024-04-25T13:57:12.154576+0000","last_active":"2024-04-25T13:57:13.143855+0000","last_peered":"2024-04-25T13:57:13.143855+0000","last_clean":"2024-04-25T13:57:13.143855+0000","last_became_active":"2024-04-25T13:57:12.154254+0000","last_became_peered":"2024-04-25T13:57:12.154254+0000","last_unstale":"2024-04-25T13:57:13.143855+0000","last_undegraded":"2024-04-25T13:57:13.143855+0000","last_fullsized":"2024-04-25T13:57:13.143855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:33.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326160+0000","last_change":"2024-04-25T13:57:11.139941+0000","last_active":"2024-04-25T13:57:13.326160+0000","last_peered":"2024-04-25T13:57:13.326160+0000","last_clean":"2024-04-25T13:57:13.326160+0000","last_became_active":"2024-04-25T13:57:11.139614+0000","last_became_peered":"2024-04-25T13:57:11.139614+0000","last_unstale":"2024-04-25T13:57:13.326160+0000","last_undegraded":"2024-04-25T13:57:13.326160+0000","last_fullsized":"2024-04-25T13:57:13.326160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:39.656085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143559+0000","last_change":"2024-04-25T13:57:12.155935+0000","last_active":"2024-04-25T13:57:13.143559+0000","last_peered":"2024-04-25T13:57:13.143559+0000","last_clean":"2024-04-25T13:57:13.143559+0000","last_became_active":"2024-04-25T13:57:12.155261+0000","last_became_peered":"2024-04-25T13:57:12.155261+0000","last_unstale":"2024-04-25T13:57:13.143559+0000","last_undegraded":"2024-04-25T13:57:13.143559+0000","last_fullsized":"2024-04-25T13:57:13.143559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:54.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841903+0000","last_change":"2024-04-25T13:57:12.148525+0000","last_active":"2024-04-25T13:57:13.841903+0000","last_peered":"2024-04-25T13:57:13.841903+0000","last_clean":"2024-04-25T13:57:13.841903+0000","last_became_active":"2024-04-25T13:57:12.148160+0000","last_became_peered":"2024-04-25T13:57:12.148160+0000","last_unstale":"2024-04-25T13:57:13.841903+0000","last_undegraded":"2024-04-25T13:57:13.841903+0000","last_fullsized":"2024-04-25T13:57:13.841903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:05.928439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.127234+0000","last_change":"2024-04-25T13:57:11.135226+0000","last_active":"2024-04-25T13:57:12.127234+0000","last_peered":"2024-04-25T13:57:12.127234+0000","last_clean":"2024-04-25T13:57:12.127234+0000","last_became_active":"2024-04-25T13:57:11.134886+0000","last_became_peered":"2024-04-25T13:57:11.134886+0000","last_unstale":"2024-04-25T13:57:12.127234+0000","last_undegraded":"2024-04-25T13:57:12.127234+0000","last_fullsized":"2024-04-25T13:57:12.127234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:00.153597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213255+0000","last_change":"2024-04-25T13:57:12.149758+0000","last_active":"2024-04-25T13:57:13.213255+0000","last_peered":"2024-04-25T13:57:13.213255+0000","last_clean":"2024-04-25T13:57:13.213255+0000","last_became_active":"2024-04-25T13:57:12.149428+0000","last_became_peered":"2024-04-25T13:57:12.149428+0000","last_unstale":"2024-04-25T13:57:13.213255+0000","last_undegraded":"2024-04-25T13:57:13.213255+0000","last_fullsized":"2024-04-25T13:57:13.213255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.739074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.088819+0000","last_change":"2024-04-25T13:57:11.135968+0000","last_active":"2024-04-25T13:57:15.088819+0000","last_peered":"2024-04-25T13:57:15.088819+0000","last_clean":"2024-04-25T13:57:15.088819+0000","last_became_active":"2024-04-25T13:57:11.135577+0000","last_became_peered":"2024-04-25T13:57:11.135577+0000","last_unstale":"2024-04-25T13:57:15.088819+0000","last_undegraded":"2024-04-25T13:57:15.088819+0000","last_fullsized":"2024-04-25T13:57:15.088819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:00.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.325913+0000","last_change":"2024-04-25T13:55:57.666399+0000","last_active":"2024-04-25T13:57:13.325913+0000","last_peered":"2024-04-25T13:57:13.325913+0000","last_clean":"2024-04-25T13:57:13.325913+0000","last_became_active":"2024-04-25T13:55:57.660470+0000","last_became_peered":"2024-04-25T13:55:57.660470+0000","last_unstale":"2024-04-25T13:57:13.325913+0000","last_undegraded":"2024-04-25T13:57:13.325913+0000","last_fullsized":"2024-04-25T13:57:13.325913+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_clean_scrub_stamp":"2024-04-25T13:52:33.081441+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:22.788632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.143869+0000","last_change":"2024-04-25T13:57:12.141923+0000","last_active":"2024-04-25T13:57:12.143869+0000","last_peered":"2024-04-25T13:57:12.143869+0000","last_clean":"2024-04-25T13:57:12.143869+0000","last_became_active":"2024-04-25T13:57:12.140946+0000","last_became_peered":"2024-04-25T13:57:12.140946+0000","last_unstale":"2024-04-25T13:57:12.143869+0000","last_undegraded":"2024-04-25T13:57:12.143869+0000","last_fullsized":"2024-04-25T13:57:12.143869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:28.362262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.167676+0000","last_change":"2024-04-25T13:57:11.139851+0000","last_active":"2024-04-25T13:57:14.167676+0000","last_peered":"2024-04-25T13:57:14.167676+0000","last_clean":"2024-04-25T13:57:14.167676+0000","last_became_active":"2024-04-25T13:57:11.139476+0000","last_became_peered":"2024-04-25T13:57:11.139476+0000","last_unstale":"2024-04-25T13:57:14.167676+0000","last_undegraded":"2024-04-25T13:57:14.167676+0000","last_fullsized":"2024-04-25T13:57:14.167676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:29.435917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.136341+0000","last_change":"2024-04-25T13:57:11.146054+0000","last_active":"2024-04-25T13:57:12.136341+0000","last_peered":"2024-04-25T13:57:12.136341+0000","last_clean":"2024-04-25T13:57:12.136341+0000","last_became_active":"2024-04-25T13:57:11.145616+0000","last_became_peered":"2024-04-25T13:57:11.145616+0000","last_unstale":"2024-04-25T13:57:12.136341+0000","last_undegraded":"2024-04-25T13:57:12.136341+0000","last_fullsized":"2024-04-25T13:57:12.136341+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:09.825563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.136798+0000","last_change":"2024-04-25T13:57:12.136204+0000","last_active":"2024-04-25T13:57:12.136798+0000","last_peered":"2024-04-25T13:57:12.136798+0000","last_clean":"2024-04-25T13:57:12.136798+0000","last_became_active":"2024-04-25T13:57:12.135798+0000","last_became_peered":"2024-04-25T13:57:12.135798+0000","last_unstale":"2024-04-25T13:57:12.136798+0000","last_undegraded":"2024-04-25T13:57:12.136798+0000","last_fullsized":"2024-04-25T13:57:12.136798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.471835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.130923+0000","last_change":"2024-04-25T13:57:11.138699+0000","last_active":"2024-04-25T13:57:14.130923+0000","last_peered":"2024-04-25T13:57:14.130923+0000","last_clean":"2024-04-25T13:57:14.130923+0000","last_became_active":"2024-04-25T13:57:11.138335+0000","last_became_peered":"2024-04-25T13:57:11.138335+0000","last_unstale":"2024-04-25T13:57:14.130923+0000","last_undegraded":"2024-04-25T13:57:14.130923+0000","last_fullsized":"2024-04-25T13:57:14.130923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:44.413228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213623+0000","last_change":"2024-04-25T13:57:12.155851+0000","last_active":"2024-04-25T13:57:13.213623+0000","last_peered":"2024-04-25T13:57:13.213623+0000","last_clean":"2024-04-25T13:57:13.213623+0000","last_became_active":"2024-04-25T13:57:12.155416+0000","last_became_peered":"2024-04-25T13:57:12.155416+0000","last_unstale":"2024-04-25T13:57:13.213623+0000","last_undegraded":"2024-04-25T13:57:13.213623+0000","last_fullsized":"2024-04-25T13:57:13.213623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:22.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.166686+0000","last_change":"2024-04-25T13:57:11.147465+0000","last_active":"2024-04-25T13:57:14.166686+0000","last_peered":"2024-04-25T13:57:14.166686+0000","last_clean":"2024-04-25T13:57:14.166686+0000","last_became_active":"2024-04-25T13:57:11.147096+0000","last_became_peered":"2024-04-25T13:57:11.147096+0000","last_unstale":"2024-04-25T13:57:14.166686+0000","last_undegraded":"2024-04-25T13:57:14.166686+0000","last_fullsized":"2024-04-25T13:57:14.166686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.229356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841536+0000","last_change":"2024-04-25T13:57:12.146446+0000","last_active":"2024-04-25T13:57:13.841536+0000","last_peered":"2024-04-25T13:57:13.841536+0000","last_clean":"2024-04-25T13:57:13.841536+0000","last_became_active":"2024-04-25T13:57:12.146129+0000","last_became_peered":"2024-04-25T13:57:12.146129+0000","last_unstale":"2024-04-25T13:57:13.841536+0000","last_undegraded":"2024-04-25T13:57:13.841536+0000","last_fullsized":"2024-04-25T13:57:13.841536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.977663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168778+0000","last_change":"2024-04-25T13:57:11.145127+0000","last_active":"2024-04-25T13:57:14.168778+0000","last_peered":"2024-04-25T13:57:14.168778+0000","last_clean":"2024-04-25T13:57:14.168778+0000","last_became_active":"2024-04-25T13:57:11.144822+0000","last_became_peered":"2024-04-25T13:57:11.144822+0000","last_unstale":"2024-04-25T13:57:14.168778+0000","last_undegraded":"2024-04-25T13:57:14.168778+0000","last_fullsized":"2024-04-25T13:57:14.168778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:00.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.160222+0000","last_change":"2024-04-25T13:57:12.156184+0000","last_active":"2024-04-25T13:57:12.160222+0000","last_peered":"2024-04-25T13:57:12.160222+0000","last_clean":"2024-04-25T13:57:12.160222+0000","last_became_active":"2024-04-25T13:57:12.155855+0000","last_became_peered":"2024-04-25T13:57:12.155855+0000","last_unstale":"2024-04-25T13:57:12.160222+0000","last_undegraded":"2024-04-25T13:57:12.160222+0000","last_fullsized":"2024-04-25T13:57:12.160222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:09.573860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168413+0000","last_change":"2024-04-25T13:57:11.137616+0000","last_active":"2024-04-25T13:57:14.168413+0000","last_peered":"2024-04-25T13:57:14.168413+0000","last_clean":"2024-04-25T13:57:14.168413+0000","last_became_active":"2024-04-25T13:57:11.137041+0000","last_became_peered":"2024-04-25T13:57:11.137041+0000","last_unstale":"2024-04-25T13:57:14.168413+0000","last_undegraded":"2024-04-25T13:57:14.168413+0000","last_fullsized":"2024-04-25T13:57:14.168413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:54.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142868+0000","last_change":"2024-04-25T13:57:12.151223+0000","last_active":"2024-04-25T13:57:13.142868+0000","last_peered":"2024-04-25T13:57:13.142868+0000","last_clean":"2024-04-25T13:57:13.142868+0000","last_became_active":"2024-04-25T13:57:12.150871+0000","last_became_peered":"2024-04-25T13:57:12.150871+0000","last_unstale":"2024-04-25T13:57:13.142868+0000","last_undegraded":"2024-04-25T13:57:13.142868+0000","last_fullsized":"2024-04-25T13:57:13.142868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:17.957728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786569+0000","last_change":"2024-04-25T13:57:11.136524+0000","last_active":"2024-04-25T13:57:13.786569+0000","last_peered":"2024-04-25T13:57:13.786569+0000","last_clean":"2024-04-25T13:57:13.786569+0000","last_became_active":"2024-04-25T13:57:11.136152+0000","last_became_peered":"2024-04-25T13:57:11.136152+0000","last_unstale":"2024-04-25T13:57:13.786569+0000","last_undegraded":"2024-04-25T13:57:13.786569+0000","last_fullsized":"2024-04-25T13:57:13.786569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:23.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213401+0000","last_change":"2024-04-25T13:57:12.157641+0000","last_active":"2024-04-25T13:57:13.213401+0000","last_peered":"2024-04-25T13:57:13.213401+0000","last_clean":"2024-04-25T13:57:13.213401+0000","last_became_active":"2024-04-25T13:57:12.157281+0000","last_became_peered":"2024-04-25T13:57:12.157281+0000","last_unstale":"2024-04-25T13:57:13.213401+0000","last_undegraded":"2024-04-25T13:57:13.213401+0000","last_fullsized":"2024-04-25T13:57:13.213401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:28.245071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162634+0000","last_change":"2024-04-25T13:57:11.139036+0000","last_active":"2024-04-25T13:57:12.162634+0000","last_peered":"2024-04-25T13:57:12.162634+0000","last_clean":"2024-04-25T13:57:12.162634+0000","last_became_active":"2024-04-25T13:57:11.138681+0000","last_became_peered":"2024-04-25T13:57:11.138681+0000","last_unstale":"2024-04-25T13:57:12.162634+0000","last_undegraded":"2024-04-25T13:57:12.162634+0000","last_fullsized":"2024-04-25T13:57:12.162634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:48.584313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326529+0000","last_change":"2024-04-25T13:57:12.153072+0000","last_active":"2024-04-25T13:57:13.326529+0000","last_peered":"2024-04-25T13:57:13.326529+0000","last_clean":"2024-04-25T13:57:13.326529+0000","last_became_active":"2024-04-25T13:57:12.151556+0000","last_became_peered":"2024-04-25T13:57:12.151556+0000","last_unstale":"2024-04-25T13:57:13.326529+0000","last_undegraded":"2024-04-25T13:57:13.326529+0000","last_fullsized":"2024-04-25T13:57:13.326529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:20.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.089122+0000","last_change":"2024-04-25T13:57:11.148823+0000","last_active":"2024-04-25T13:57:15.089122+0000","last_peered":"2024-04-25T13:57:15.089122+0000","last_clean":"2024-04-25T13:57:15.089122+0000","last_became_active":"2024-04-25T13:57:11.148436+0000","last_became_peered":"2024-04-25T13:57:11.148436+0000","last_unstale":"2024-04-25T13:57:15.089122+0000","last_undegraded":"2024-04-25T13:57:15.089122+0000","last_fullsized":"2024-04-25T13:57:15.089122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:28.496184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326536+0000","last_change":"2024-04-25T13:57:12.149082+0000","last_active":"2024-04-25T13:57:13.326536+0000","last_peered":"2024-04-25T13:57:13.326536+0000","last_clean":"2024-04-25T13:57:13.326536+0000","last_became_active":"2024-04-25T13:57:12.148260+0000","last_became_peered":"2024-04-25T13:57:12.148260+0000","last_unstale":"2024-04-25T13:57:13.326536+0000","last_undegraded":"2024-04-25T13:57:13.326536+0000","last_fullsized":"2024-04-25T13:57:13.326536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:49.733114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841903+0000","last_change":"2024-04-25T13:57:11.136736+0000","last_active":"2024-04-25T13:57:13.841903+0000","last_peered":"2024-04-25T13:57:13.841903+0000","last_clean":"2024-04-25T13:57:13.841903+0000","last_became_active":"2024-04-25T13:57:11.135831+0000","last_became_peered":"2024-04-25T13:57:11.135831+0000","last_unstale":"2024-04-25T13:57:13.841903+0000","last_undegraded":"2024-04-25T13:57:13.841903+0000","last_fullsized":"2024-04-25T13:57:13.841903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:49.654642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326296+0000","last_change":"2024-04-25T13:57:12.149797+0000","last_active":"2024-04-25T13:57:13.326296+0000","last_peered":"2024-04-25T13:57:13.326296+0000","last_clean":"2024-04-25T13:57:13.326296+0000","last_became_active":"2024-04-25T13:57:12.149413+0000","last_became_peered":"2024-04-25T13:57:12.149413+0000","last_unstale":"2024-04-25T13:57:13.326296+0000","last_undegraded":"2024-04-25T13:57:13.326296+0000","last_fullsized":"2024-04-25T13:57:13.326296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:58.517208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.154403+0000","last_change":"2024-04-25T13:57:11.138419+0000","last_active":"2024-04-25T13:57:12.154403+0000","last_peered":"2024-04-25T13:57:12.154403+0000","last_clean":"2024-04-25T13:57:12.154403+0000","last_became_active":"2024-04-25T13:57:11.138058+0000","last_became_peered":"2024-04-25T13:57:11.138058+0000","last_unstale":"2024-04-25T13:57:12.154403+0000","last_undegraded":"2024-04-25T13:57:12.154403+0000","last_fullsized":"2024-04-25T13:57:12.154403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:24.231004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326547+0000","last_change":"2024-04-25T13:57:12.150247+0000","last_active":"2024-04-25T13:57:13.326547+0000","last_peered":"2024-04-25T13:57:13.326547+0000","last_clean":"2024-04-25T13:57:13.326547+0000","last_became_active":"2024-04-25T13:57:12.149882+0000","last_became_peered":"2024-04-25T13:57:12.149882+0000","last_unstale":"2024-04-25T13:57:13.326547+0000","last_undegraded":"2024-04-25T13:57:13.326547+0000","last_fullsized":"2024-04-25T13:57:13.326547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.624928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143138+0000","last_change":"2024-04-25T13:57:12.154618+0000","last_active":"2024-04-25T13:57:13.143138+0000","last_peered":"2024-04-25T13:57:13.143138+0000","last_clean":"2024-04-25T13:57:13.143138+0000","last_became_active":"2024-04-25T13:57:12.153648+0000","last_became_peered":"2024-04-25T13:57:12.153648+0000","last_unstale":"2024-04-25T13:57:13.143138+0000","last_undegraded":"2024-04-25T13:57:13.143138+0000","last_fullsized":"2024-04-25T13:57:13.143138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:24.010980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786093+0000","last_change":"2024-04-25T13:57:12.143097+0000","last_active":"2024-04-25T13:57:13.786093+0000","last_peered":"2024-04-25T13:57:13.786093+0000","last_clean":"2024-04-25T13:57:13.786093+0000","last_became_active":"2024-04-25T13:57:12.142295+0000","last_became_peered":"2024-04-25T13:57:12.142295+0000","last_unstale":"2024-04-25T13:57:13.786093+0000","last_undegraded":"2024-04-25T13:57:13.786093+0000","last_fullsized":"2024-04-25T13:57:13.786093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.907643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786373+0000","last_change":"2024-04-25T13:57:12.155659+0000","last_active":"2024-04-25T13:57:13.786373+0000","last_peered":"2024-04-25T13:57:13.786373+0000","last_clean":"2024-04-25T13:57:13.786373+0000","last_became_active":"2024-04-25T13:57:12.155308+0000","last_became_peered":"2024-04-25T13:57:12.155308+0000","last_unstale":"2024-04-25T13:57:13.786373+0000","last_undegraded":"2024-04-25T13:57:13.786373+0000","last_fullsized":"2024-04-25T13:57:13.786373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:12.361660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841274+0000","last_change":"2024-04-25T13:57:12.145750+0000","last_active":"2024-04-25T13:57:13.841274+0000","last_peered":"2024-04-25T13:57:13.841274+0000","last_clean":"2024-04-25T13:57:13.841274+0000","last_became_active":"2024-04-25T13:57:12.145339+0000","last_became_peered":"2024-04-25T13:57:12.145339+0000","last_unstale":"2024-04-25T13:57:13.841274+0000","last_undegraded":"2024-04-25T13:57:13.841274+0000","last_fullsized":"2024-04-25T13:57:13.841274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:18.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.144452+0000","last_change":"2024-04-25T13:57:12.142134+0000","last_active":"2024-04-25T13:57:12.144452+0000","last_peered":"2024-04-25T13:57:12.144452+0000","last_clean":"2024-04-25T13:57:12.144452+0000","last_became_active":"2024-04-25T13:57:12.141776+0000","last_became_peered":"2024-04-25T13:57:12.141776+0000","last_unstale":"2024-04-25T13:57:12.144452+0000","last_undegraded":"2024-04-25T13:57:12.144452+0000","last_fullsized":"2024-04-25T13:57:12.144452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:09.817661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143137+0000","last_change":"2024-04-25T13:57:12.146269+0000","last_active":"2024-04-25T13:57:13.143137+0000","last_peered":"2024-04-25T13:57:13.143137+0000","last_clean":"2024-04-25T13:57:13.143137+0000","last_became_active":"2024-04-25T13:57:12.145880+0000","last_became_peered":"2024-04-25T13:57:12.145880+0000","last_unstale":"2024-04-25T13:57:13.143137+0000","last_undegraded":"2024-04-25T13:57:13.143137+0000","last_fullsized":"2024-04-25T13:57:13.143137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:08.928272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786752+0000","last_change":"2024-04-25T13:57:12.147818+0000","last_active":"2024-04-25T13:57:13.786752+0000","last_peered":"2024-04-25T13:57:13.786752+0000","last_clean":"2024-04-25T13:57:13.786752+0000","last_became_active":"2024-04-25T13:57:12.146764+0000","last_became_peered":"2024-04-25T13:57:12.146764+0000","last_unstale":"2024-04-25T13:57:13.786752+0000","last_undegraded":"2024-04-25T13:57:13.786752+0000","last_fullsized":"2024-04-25T13:57:13.786752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.750115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213475+0000","last_change":"2024-04-25T13:57:12.152591+0000","last_active":"2024-04-25T13:57:13.213475+0000","last_peered":"2024-04-25T13:57:13.213475+0000","last_clean":"2024-04-25T13:57:13.213475+0000","last_became_active":"2024-04-25T13:57:12.152198+0000","last_became_peered":"2024-04-25T13:57:12.152198+0000","last_unstale":"2024-04-25T13:57:13.213475+0000","last_undegraded":"2024-04-25T13:57:13.213475+0000","last_fullsized":"2024-04-25T13:57:13.213475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:05.868774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841628+0000","last_change":"2024-04-25T13:57:12.154924+0000","last_active":"2024-04-25T13:57:13.841628+0000","last_peered":"2024-04-25T13:57:13.841628+0000","last_clean":"2024-04-25T13:57:13.841628+0000","last_became_active":"2024-04-25T13:57:12.154549+0000","last_became_peered":"2024-04-25T13:57:12.154549+0000","last_unstale":"2024-04-25T13:57:13.841628+0000","last_undegraded":"2024-04-25T13:57:13.841628+0000","last_fullsized":"2024-04-25T13:57:13.841628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:49.418436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213103+0000","last_change":"2024-04-25T13:57:12.155774+0000","last_active":"2024-04-25T13:57:13.213103+0000","last_peered":"2024-04-25T13:57:13.213103+0000","last_clean":"2024-04-25T13:57:13.213103+0000","last_became_active":"2024-04-25T13:57:12.155352+0000","last_became_peered":"2024-04-25T13:57:12.155352+0000","last_unstale":"2024-04-25T13:57:13.213103+0000","last_undegraded":"2024-04-25T13:57:13.213103+0000","last_fullsized":"2024-04-25T13:57:13.213103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:46.854558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841775+0000","last_change":"2024-04-25T13:57:12.158672+0000","last_active":"2024-04-25T13:57:13.841775+0000","last_peered":"2024-04-25T13:57:13.841775+0000","last_clean":"2024-04-25T13:57:13.841775+0000","last_became_active":"2024-04-25T13:57:12.158391+0000","last_became_peered":"2024-04-25T13:57:12.158391+0000","last_unstale":"2024-04-25T13:57:13.841775+0000","last_undegraded":"2024-04-25T13:57:13.841775+0000","last_fullsized":"2024-04-25T13:57:13.841775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.369907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.140017+0000","last_change":"2024-04-25T13:57:12.139310+0000","last_active":"2024-04-25T13:57:12.140017+0000","last_peered":"2024-04-25T13:57:12.140017+0000","last_clean":"2024-04-25T13:57:12.140017+0000","last_became_active":"2024-04-25T13:57:12.138942+0000","last_became_peered":"2024-04-25T13:57:12.138942+0000","last_unstale":"2024-04-25T13:57:12.140017+0000","last_undegraded":"2024-04-25T13:57:12.140017+0000","last_fullsized":"2024-04-25T13:57:12.140017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:55.906480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142870+0000","last_change":"2024-04-25T13:57:12.147451+0000","last_active":"2024-04-25T13:57:13.142870+0000","last_peered":"2024-04-25T13:57:13.142870+0000","last_clean":"2024-04-25T13:57:13.142870+0000","last_became_active":"2024-04-25T13:57:12.146746+0000","last_became_peered":"2024-04-25T13:57:12.146746+0000","last_unstale":"2024-04-25T13:57:13.142870+0000","last_undegraded":"2024-04-25T13:57:13.142870+0000","last_fullsized":"2024-04-25T13:57:13.142870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247568,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5780,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":71073,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710734,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":802816,"data_stored":384344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906963,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":977320,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070488,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":802816,"data_stored":384344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266717,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":794624,"data_stored":375796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":974712,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626471,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":786432,"data_stored":375192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":404657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":802816,"data_stored":383852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":839680,"data_stored":393715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":819200,"data_stored":384163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":811008,"data_stored":384797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":975997,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38193,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:57:18.928 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 -- 172.21.15.78:0/3453673127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 msgr2=0x7f9fac104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3453673127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac104060 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009900 tx=0x7f9f9c02a410 comp rx=0 tx=0).stop 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 -- 172.21.15.78:0/3453673127 shutdown_connections 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3453673127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fac108950 0x7f9fac10ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3453673127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac104060 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3453673127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fac102a10 0x7f9fac103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 -- 172.21.15.78:0/3453673127 >> 172.21.15.78:0/3453673127 conn(0x7f9fac0fdc00 msgr2=0x7f9fac1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 -- 172.21.15.78:0/3453673127 shutdown_connections 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 -- 172.21.15.78:0/3453673127 wait complete. 2024-04-25T13:57:19.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 Processor -- start 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.071+0000 7f9fb2996640 1 -- start start 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fb2996640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fac102a10 0x7f9fac19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fb2996640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fb2996640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fac108950 0x7f9fac1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fb2996640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10e510 con 0x7f9fac103c50 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fb2996640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10e390 con 0x7f9fac102a10 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fb2996640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac10e690 con 0x7f9fac108950 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53446/0 (socket says 172.21.15.78:53446) 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 -- 172.21.15.78:0/3876056160 learned_addr learned my addr 172.21.15.78:0/3876056160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fabfff640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fac102a10 0x7f9fac19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:19.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fb0f33640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fac108950 0x7f9fac1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 -- 172.21.15.78:0/3876056160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fac108950 msgr2=0x7f9fac1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fac108950 0x7f9fac1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 -- 172.21.15.78:0/3876056160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fac102a10 msgr2=0x7f9fac19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fac102a10 0x7f9fac19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 -- 172.21.15.78:0/3876056160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac1a1df0 con 0x7f9fac103c50 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.072+0000 7f9fab7fe640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac19a860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006010 tx=0x7f9f9c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.073+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c03f030 con 0x7f9fac103c50 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.073+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c003aa0 con 0x7f9fac103c50 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.073+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac1a2080 con 0x7f9fac103c50 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.073+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c035070 con 0x7f9fac103c50 2024-04-25T13:57:19.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.073+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac1a2570 con 0x7f9fac103c50 2024-04-25T13:57:19.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.074+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac1a27d0 con 0x7f9fac103c50 2024-04-25T13:57:19.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.076+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9f9c042030 con 0x7f9fac103c50 2024-04-25T13:57:19.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.076+0000 7f9fa97fa640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f940afaf0 0x7f9f940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:19.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.076+0000 7f9fabfff640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f940afaf0 0x7f9f940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.077+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f9f9c0ff310 con 0x7f9fac103c50 2024-04-25T13:57:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.077+0000 7f9fabfff640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f940afaf0 0x7f9f940b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005fd0 tx=0x7f9fa000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.080+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f9c0c8040 con 0x7f9fac103c50 2024-04-25T13:57:19.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.174+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9f9c0c3bd0 con 0x7f9fac103c50 2024-04-25T13:57:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:19 smithi078 ceph-mon[29163]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:19 smithi078 ceph-mon[29163]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:19 smithi078 ceph-mon[29163]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-04-25T13:57:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:19 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/371971048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:19 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/585199612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:19.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.262+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fac10a5b0 con 0x7f9f940afaf0 2024-04-25T13:57:19.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.269+0000 7f9fa97fa640 1 -- 172.21.15.78:0/3876056160 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258870 (secure 0 0 0) 0x7f9fac10a5b0 con 0x7f9f940afaf0 2024-04-25T13:57:19.271 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:19.272 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f940afaf0 msgr2=0x7f9f940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f940afaf0 0x7f9f940b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005fd0 tx=0x7f9fa000b410 comp rx=0 tx=0).stop 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 msgr2=0x7f9fac19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac19a860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006010 tx=0x7f9f9c031a50 comp rx=0 tx=0).stop 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 shutdown_connections 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fac108950 0x7f9fac1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f940afaf0 0x7f9f940b2000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fac103c50 0x7f9fac19a860 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 --2- 172.21.15.78:0/3876056160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fac102a10 0x7f9fac19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 >> 172.21.15.78:0/3876056160 conn(0x7f9fac0fdc00 msgr2=0x7f9fac109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:19.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 shutdown_connections 2024-04-25T13:57:19.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:19.272+0000 7f9fb2996640 1 -- 172.21.15.78:0/3876056160 wait complete. 2024-04-25T13:57:19.283 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-25T13:57:18.126957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596148,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":98,"num_read_kb":81,"num_write":216,"num_write_kb":2180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":174,"ondisk_log_size":174,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":250,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371356,"kb_used_data":9676,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124570468,"statfs":{"total":1151940427776,"available":1151560159232,"internally_reserved":0,"allocated":9908224,"data_stored":6398889,"data_compressed":107514,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5780,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.005032"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143214+0000","last_change":"2024-04-25T13:57:12.147384+0000","last_active":"2024-04-25T13:57:13.143214+0000","last_peered":"2024-04-25T13:57:13.143214+0000","last_clean":"2024-04-25T13:57:13.143214+0000","last_became_active":"2024-04-25T13:57:12.146349+0000","last_became_peered":"2024-04-25T13:57:12.146349+0000","last_unstale":"2024-04-25T13:57:13.143214+0000","last_undegraded":"2024-04-25T13:57:13.143214+0000","last_fullsized":"2024-04-25T13:57:13.143214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841462+0000","last_change":"2024-04-25T13:57:12.156698+0000","last_active":"2024-04-25T13:57:13.841462+0000","last_peered":"2024-04-25T13:57:13.841462+0000","last_clean":"2024-04-25T13:57:13.841462+0000","last_became_active":"2024-04-25T13:57:12.155790+0000","last_became_peered":"2024-04-25T13:57:12.155790+0000","last_unstale":"2024-04-25T13:57:13.841462+0000","last_undegraded":"2024-04-25T13:57:13.841462+0000","last_fullsized":"2024-04-25T13:57:13.841462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:25.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841300+0000","last_change":"2024-04-25T13:57:12.149458+0000","last_active":"2024-04-25T13:57:13.841300+0000","last_peered":"2024-04-25T13:57:13.841300+0000","last_clean":"2024-04-25T13:57:13.841300+0000","last_became_active":"2024-04-25T13:57:12.148179+0000","last_became_peered":"2024-04-25T13:57:12.148179+0000","last_unstale":"2024-04-25T13:57:13.841300+0000","last_undegraded":"2024-04-25T13:57:13.841300+0000","last_fullsized":"2024-04-25T13:57:13.841300+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:00.502798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.842074+0000","last_change":"2024-04-25T13:57:12.145776+0000","last_active":"2024-04-25T13:57:13.842074+0000","last_peered":"2024-04-25T13:57:13.842074+0000","last_clean":"2024-04-25T13:57:13.842074+0000","last_became_active":"2024-04-25T13:57:12.145038+0000","last_became_peered":"2024-04-25T13:57:12.145038+0000","last_unstale":"2024-04-25T13:57:13.842074+0000","last_undegraded":"2024-04-25T13:57:13.842074+0000","last_fullsized":"2024-04-25T13:57:13.842074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:52.742746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.161955+0000","last_change":"2024-04-25T13:57:12.160273+0000","last_active":"2024-04-25T13:57:12.161955+0000","last_peered":"2024-04-25T13:57:12.161955+0000","last_clean":"2024-04-25T13:57:12.161955+0000","last_became_active":"2024-04-25T13:57:12.159805+0000","last_became_peered":"2024-04-25T13:57:12.159805+0000","last_unstale":"2024-04-25T13:57:12.161955+0000","last_undegraded":"2024-04-25T13:57:12.161955+0000","last_fullsized":"2024-04-25T13:57:12.161955+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:18.188107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143584+0000","last_change":"2024-04-25T13:57:12.151597+0000","last_active":"2024-04-25T13:57:13.143584+0000","last_peered":"2024-04-25T13:57:13.143584+0000","last_clean":"2024-04-25T13:57:13.143584+0000","last_became_active":"2024-04-25T13:57:12.151203+0000","last_became_peered":"2024-04-25T13:57:12.151203+0000","last_unstale":"2024-04-25T13:57:13.143584+0000","last_undegraded":"2024-04-25T13:57:13.143584+0000","last_fullsized":"2024-04-25T13:57:13.143584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.645264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326406+0000","last_change":"2024-04-25T13:57:12.144959+0000","last_active":"2024-04-25T13:57:13.326406+0000","last_peered":"2024-04-25T13:57:13.326406+0000","last_clean":"2024-04-25T13:57:13.326406+0000","last_became_active":"2024-04-25T13:57:12.144587+0000","last_became_peered":"2024-04-25T13:57:12.144587+0000","last_unstale":"2024-04-25T13:57:13.326406+0000","last_undegraded":"2024-04-25T13:57:13.326406+0000","last_fullsized":"2024-04-25T13:57:13.326406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:45.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213101+0000","last_change":"2024-04-25T13:57:12.157016+0000","last_active":"2024-04-25T13:57:13.213101+0000","last_peered":"2024-04-25T13:57:13.213101+0000","last_clean":"2024-04-25T13:57:13.213101+0000","last_became_active":"2024-04-25T13:57:12.156209+0000","last_became_peered":"2024-04-25T13:57:12.156209+0000","last_unstale":"2024-04-25T13:57:13.213101+0000","last_undegraded":"2024-04-25T13:57:13.213101+0000","last_fullsized":"2024-04-25T13:57:13.213101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841925+0000","last_change":"2024-04-25T13:57:12.146696+0000","last_active":"2024-04-25T13:57:13.841925+0000","last_peered":"2024-04-25T13:57:13.841925+0000","last_clean":"2024-04-25T13:57:13.841925+0000","last_became_active":"2024-04-25T13:57:12.146317+0000","last_became_peered":"2024-04-25T13:57:12.146317+0000","last_unstale":"2024-04-25T13:57:13.841925+0000","last_undegraded":"2024-04-25T13:57:13.841925+0000","last_fullsized":"2024-04-25T13:57:13.841925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.078941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.144011+0000","last_change":"2024-04-25T13:57:12.155100+0000","last_active":"2024-04-25T13:57:13.144011+0000","last_peered":"2024-04-25T13:57:13.144011+0000","last_clean":"2024-04-25T13:57:13.144011+0000","last_became_active":"2024-04-25T13:57:12.154602+0000","last_became_peered":"2024-04-25T13:57:12.154602+0000","last_unstale":"2024-04-25T13:57:13.144011+0000","last_undegraded":"2024-04-25T13:57:13.144011+0000","last_fullsized":"2024-04-25T13:57:13.144011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.837117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326293+0000","last_change":"2024-04-25T13:57:12.149386+0000","last_active":"2024-04-25T13:57:13.326293+0000","last_peered":"2024-04-25T13:57:13.326293+0000","last_clean":"2024-04-25T13:57:13.326293+0000","last_became_active":"2024-04-25T13:57:12.148710+0000","last_became_peered":"2024-04-25T13:57:12.148710+0000","last_unstale":"2024-04-25T13:57:13.326293+0000","last_undegraded":"2024-04-25T13:57:13.326293+0000","last_fullsized":"2024-04-25T13:57:13.326293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:17.655260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.842039+0000","last_change":"2024-04-25T13:57:12.156103+0000","last_active":"2024-04-25T13:57:13.842039+0000","last_peered":"2024-04-25T13:57:13.842039+0000","last_clean":"2024-04-25T13:57:13.842039+0000","last_became_active":"2024-04-25T13:57:12.155738+0000","last_became_peered":"2024-04-25T13:57:12.155738+0000","last_unstale":"2024-04-25T13:57:13.842039+0000","last_undegraded":"2024-04-25T13:57:13.842039+0000","last_fullsized":"2024-04-25T13:57:13.842039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:38.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.155552+0000","last_change":"2024-04-25T13:57:12.154873+0000","last_active":"2024-04-25T13:57:12.155552+0000","last_peered":"2024-04-25T13:57:12.155552+0000","last_clean":"2024-04-25T13:57:12.155552+0000","last_became_active":"2024-04-25T13:57:12.154523+0000","last_became_peered":"2024-04-25T13:57:12.154523+0000","last_unstale":"2024-04-25T13:57:12.155552+0000","last_undegraded":"2024-04-25T13:57:12.155552+0000","last_fullsized":"2024-04-25T13:57:12.155552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:36.943259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841274+0000","last_change":"2024-04-25T13:57:12.156536+0000","last_active":"2024-04-25T13:57:13.841274+0000","last_peered":"2024-04-25T13:57:13.841274+0000","last_clean":"2024-04-25T13:57:13.841274+0000","last_became_active":"2024-04-25T13:57:12.156143+0000","last_became_peered":"2024-04-25T13:57:12.156143+0000","last_unstale":"2024-04-25T13:57:13.841274+0000","last_undegraded":"2024-04-25T13:57:13.841274+0000","last_fullsized":"2024-04-25T13:57:13.841274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:35.210059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841024+0000","last_change":"2024-04-25T13:57:12.144477+0000","last_active":"2024-04-25T13:57:13.841024+0000","last_peered":"2024-04-25T13:57:13.841024+0000","last_clean":"2024-04-25T13:57:13.841024+0000","last_became_active":"2024-04-25T13:57:12.144115+0000","last_became_peered":"2024-04-25T13:57:12.144115+0000","last_unstale":"2024-04-25T13:57:13.841024+0000","last_undegraded":"2024-04-25T13:57:13.841024+0000","last_fullsized":"2024-04-25T13:57:13.841024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:30.733713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841440+0000","last_change":"2024-04-25T13:57:12.148837+0000","last_active":"2024-04-25T13:57:13.841440+0000","last_peered":"2024-04-25T13:57:13.841440+0000","last_clean":"2024-04-25T13:57:13.841440+0000","last_became_active":"2024-04-25T13:57:12.148494+0000","last_became_peered":"2024-04-25T13:57:12.148494+0000","last_unstale":"2024-04-25T13:57:13.841440+0000","last_undegraded":"2024-04-25T13:57:13.841440+0000","last_fullsized":"2024-04-25T13:57:13.841440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:50.414344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143368+0000","last_change":"2024-04-25T13:57:12.148142+0000","last_active":"2024-04-25T13:57:13.143368+0000","last_peered":"2024-04-25T13:57:13.143368+0000","last_clean":"2024-04-25T13:57:13.143368+0000","last_became_active":"2024-04-25T13:57:12.147738+0000","last_became_peered":"2024-04-25T13:57:12.147738+0000","last_unstale":"2024-04-25T13:57:13.143368+0000","last_undegraded":"2024-04-25T13:57:13.143368+0000","last_fullsized":"2024-04-25T13:57:13.143368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:42.014605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142564+0000","last_change":"2024-04-25T13:57:12.160059+0000","last_active":"2024-04-25T13:57:13.142564+0000","last_peered":"2024-04-25T13:57:13.142564+0000","last_clean":"2024-04-25T13:57:13.142564+0000","last_became_active":"2024-04-25T13:57:12.159685+0000","last_became_peered":"2024-04-25T13:57:12.159685+0000","last_unstale":"2024-04-25T13:57:13.142564+0000","last_undegraded":"2024-04-25T13:57:13.142564+0000","last_fullsized":"2024-04-25T13:57:13.142564+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:48.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786401+0000","last_change":"2024-04-25T13:57:12.157950+0000","last_active":"2024-04-25T13:57:13.786401+0000","last_peered":"2024-04-25T13:57:13.786401+0000","last_clean":"2024-04-25T13:57:13.786401+0000","last_became_active":"2024-04-25T13:57:12.157612+0000","last_became_peered":"2024-04-25T13:57:12.157612+0000","last_unstale":"2024-04-25T13:57:13.786401+0000","last_undegraded":"2024-04-25T13:57:13.786401+0000","last_fullsized":"2024-04-25T13:57:13.786401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:55.521381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"74'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142680+0000","last_change":"2024-04-25T13:57:11.143834+0000","last_active":"2024-04-25T13:57:13.142680+0000","last_peered":"2024-04-25T13:57:13.142680+0000","last_clean":"2024-04-25T13:57:13.142680+0000","last_became_active":"2024-04-25T13:57:11.143474+0000","last_became_peered":"2024-04-25T13:57:11.143474+0000","last_unstale":"2024-04-25T13:57:13.142680+0000","last_undegraded":"2024-04-25T13:57:13.142680+0000","last_fullsized":"2024-04-25T13:57:13.142680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:30.137616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213461+0000","last_change":"2024-04-25T13:57:12.151301+0000","last_active":"2024-04-25T13:57:13.213461+0000","last_peered":"2024-04-25T13:57:13.213461+0000","last_clean":"2024-04-25T13:57:13.213461+0000","last_became_active":"2024-04-25T13:57:12.150941+0000","last_became_peered":"2024-04-25T13:57:12.150941+0000","last_unstale":"2024-04-25T13:57:13.213461+0000","last_undegraded":"2024-04-25T13:57:13.213461+0000","last_fullsized":"2024-04-25T13:57:13.213461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:22.259683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213734+0000","last_change":"2024-04-25T13:57:11.147314+0000","last_active":"2024-04-25T13:57:13.213734+0000","last_peered":"2024-04-25T13:57:13.213734+0000","last_clean":"2024-04-25T13:57:13.213734+0000","last_became_active":"2024-04-25T13:57:11.146894+0000","last_became_peered":"2024-04-25T13:57:11.146894+0000","last_unstale":"2024-04-25T13:57:13.213734+0000","last_undegraded":"2024-04-25T13:57:13.213734+0000","last_fullsized":"2024-04-25T13:57:13.213734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:55.927007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841629+0000","last_change":"2024-04-25T13:57:12.149186+0000","last_active":"2024-04-25T13:57:13.841629+0000","last_peered":"2024-04-25T13:57:13.841629+0000","last_clean":"2024-04-25T13:57:13.841629+0000","last_became_active":"2024-04-25T13:57:12.148155+0000","last_became_peered":"2024-04-25T13:57:12.148155+0000","last_unstale":"2024-04-25T13:57:13.841629+0000","last_undegraded":"2024-04-25T13:57:13.841629+0000","last_fullsized":"2024-04-25T13:57:13.841629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162338+0000","last_change":"2024-04-25T13:57:11.140086+0000","last_active":"2024-04-25T13:57:12.162338+0000","last_peered":"2024-04-25T13:57:12.162338+0000","last_clean":"2024-04-25T13:57:12.162338+0000","last_became_active":"2024-04-25T13:57:11.139751+0000","last_became_peered":"2024-04-25T13:57:11.139751+0000","last_unstale":"2024-04-25T13:57:12.162338+0000","last_undegraded":"2024-04-25T13:57:12.162338+0000","last_fullsized":"2024-04-25T13:57:12.162338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.552503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841178+0000","last_change":"2024-04-25T13:57:12.157029+0000","last_active":"2024-04-25T13:57:13.841178+0000","last_peered":"2024-04-25T13:57:13.841178+0000","last_clean":"2024-04-25T13:57:13.841178+0000","last_became_active":"2024-04-25T13:57:12.156668+0000","last_became_peered":"2024-04-25T13:57:12.156668+0000","last_unstale":"2024-04-25T13:57:13.841178+0000","last_undegraded":"2024-04-25T13:57:13.841178+0000","last_fullsized":"2024-04-25T13:57:13.841178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162127+0000","last_change":"2024-04-25T13:57:11.140592+0000","last_active":"2024-04-25T13:57:12.162127+0000","last_peered":"2024-04-25T13:57:12.162127+0000","last_clean":"2024-04-25T13:57:12.162127+0000","last_became_active":"2024-04-25T13:57:11.140244+0000","last_became_peered":"2024-04-25T13:57:11.140244+0000","last_unstale":"2024-04-25T13:57:12.162127+0000","last_undegraded":"2024-04-25T13:57:12.162127+0000","last_fullsized":"2024-04-25T13:57:12.162127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:53.071092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841929+0000","last_change":"2024-04-25T13:57:12.148593+0000","last_active":"2024-04-25T13:57:13.841929+0000","last_peered":"2024-04-25T13:57:13.841929+0000","last_clean":"2024-04-25T13:57:13.841929+0000","last_became_active":"2024-04-25T13:57:12.148003+0000","last_became_peered":"2024-04-25T13:57:12.148003+0000","last_unstale":"2024-04-25T13:57:13.841929+0000","last_undegraded":"2024-04-25T13:57:13.841929+0000","last_fullsized":"2024-04-25T13:57:13.841929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:58.924413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:17.083034+0000","last_change":"2024-04-25T13:57:11.140191+0000","last_active":"2024-04-25T13:57:17.083034+0000","last_peered":"2024-04-25T13:57:17.083034+0000","last_clean":"2024-04-25T13:57:17.083034+0000","last_became_active":"2024-04-25T13:57:11.137935+0000","last_became_peered":"2024-04-25T13:57:11.137935+0000","last_unstale":"2024-04-25T13:57:17.083034+0000","last_undegraded":"2024-04-25T13:57:17.083034+0000","last_fullsized":"2024-04-25T13:57:17.083034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:05.642996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.149620+0000","last_change":"2024-04-25T13:57:12.148787+0000","last_active":"2024-04-25T13:57:12.149620+0000","last_peered":"2024-04-25T13:57:12.149620+0000","last_clean":"2024-04-25T13:57:12.149620+0000","last_became_active":"2024-04-25T13:57:12.148424+0000","last_became_peered":"2024-04-25T13:57:12.148424+0000","last_unstale":"2024-04-25T13:57:12.149620+0000","last_undegraded":"2024-04-25T13:57:12.149620+0000","last_fullsized":"2024-04-25T13:57:12.149620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:38.154058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.087067+0000","last_change":"2024-04-25T13:57:11.142424+0000","last_active":"2024-04-25T13:57:15.087067+0000","last_peered":"2024-04-25T13:57:15.087067+0000","last_clean":"2024-04-25T13:57:15.087067+0000","last_became_active":"2024-04-25T13:57:11.142047+0000","last_became_peered":"2024-04-25T13:57:11.142047+0000","last_unstale":"2024-04-25T13:57:15.087067+0000","last_undegraded":"2024-04-25T13:57:15.087067+0000","last_fullsized":"2024-04-25T13:57:15.087067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:13.455149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.842013+0000","last_change":"2024-04-25T13:57:12.155940+0000","last_active":"2024-04-25T13:57:13.842013+0000","last_peered":"2024-04-25T13:57:13.842013+0000","last_clean":"2024-04-25T13:57:13.842013+0000","last_became_active":"2024-04-25T13:57:12.155503+0000","last_became_peered":"2024-04-25T13:57:12.155503+0000","last_unstale":"2024-04-25T13:57:13.842013+0000","last_undegraded":"2024-04-25T13:57:13.842013+0000","last_fullsized":"2024-04-25T13:57:13.842013+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:48.345437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.127248+0000","last_change":"2024-04-25T13:57:11.131160+0000","last_active":"2024-04-25T13:57:12.127248+0000","last_peered":"2024-04-25T13:57:12.127248+0000","last_clean":"2024-04-25T13:57:12.127248+0000","last_became_active":"2024-04-25T13:57:11.130753+0000","last_became_peered":"2024-04-25T13:57:11.130753+0000","last_unstale":"2024-04-25T13:57:12.127248+0000","last_undegraded":"2024-04-25T13:57:12.127248+0000","last_fullsized":"2024-04-25T13:57:12.127248+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:48.297546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213163+0000","last_change":"2024-04-25T13:57:12.155779+0000","last_active":"2024-04-25T13:57:13.213163+0000","last_peered":"2024-04-25T13:57:13.213163+0000","last_clean":"2024-04-25T13:57:13.213163+0000","last_became_active":"2024-04-25T13:57:12.154682+0000","last_became_peered":"2024-04-25T13:57:12.154682+0000","last_unstale":"2024-04-25T13:57:13.213163+0000","last_undegraded":"2024-04-25T13:57:13.213163+0000","last_fullsized":"2024-04-25T13:57:13.213163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:20.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841536+0000","last_change":"2024-04-25T13:57:11.140211+0000","last_active":"2024-04-25T13:57:13.841536+0000","last_peered":"2024-04-25T13:57:13.841536+0000","last_clean":"2024-04-25T13:57:13.841536+0000","last_became_active":"2024-04-25T13:57:11.139891+0000","last_became_peered":"2024-04-25T13:57:11.139891+0000","last_unstale":"2024-04-25T13:57:13.841536+0000","last_undegraded":"2024-04-25T13:57:13.841536+0000","last_fullsized":"2024-04-25T13:57:13.841536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:21.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142467+0000","last_change":"2024-04-25T13:57:12.153886+0000","last_active":"2024-04-25T13:57:13.142467+0000","last_peered":"2024-04-25T13:57:13.142467+0000","last_clean":"2024-04-25T13:57:13.142467+0000","last_became_active":"2024-04-25T13:57:12.153500+0000","last_became_peered":"2024-04-25T13:57:12.153500+0000","last_unstale":"2024-04-25T13:57:13.142467+0000","last_undegraded":"2024-04-25T13:57:13.142467+0000","last_fullsized":"2024-04-25T13:57:13.142467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:53.549745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'3","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168548+0000","last_change":"2024-04-25T13:57:11.138829+0000","last_active":"2024-04-25T13:57:14.168548+0000","last_peered":"2024-04-25T13:57:14.168548+0000","last_clean":"2024-04-25T13:57:14.168548+0000","last_became_active":"2024-04-25T13:57:11.138461+0000","last_became_peered":"2024-04-25T13:57:11.138461+0000","last_unstale":"2024-04-25T13:57:14.168548+0000","last_undegraded":"2024-04-25T13:57:14.168548+0000","last_fullsized":"2024-04-25T13:57:14.168548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.134061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841336+0000","last_change":"2024-04-25T13:57:12.157394+0000","last_active":"2024-04-25T13:57:13.841336+0000","last_peered":"2024-04-25T13:57:13.841336+0000","last_clean":"2024-04-25T13:57:13.841336+0000","last_became_active":"2024-04-25T13:57:12.156408+0000","last_became_peered":"2024-04-25T13:57:12.156408+0000","last_unstale":"2024-04-25T13:57:13.841336+0000","last_undegraded":"2024-04-25T13:57:13.841336+0000","last_fullsized":"2024-04-25T13:57:13.841336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:36.741367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168755+0000","last_change":"2024-04-25T13:57:11.142846+0000","last_active":"2024-04-25T13:57:14.168755+0000","last_peered":"2024-04-25T13:57:14.168755+0000","last_clean":"2024-04-25T13:57:14.168755+0000","last_became_active":"2024-04-25T13:57:11.142387+0000","last_became_peered":"2024-04-25T13:57:11.142387+0000","last_unstale":"2024-04-25T13:57:14.168755+0000","last_undegraded":"2024-04-25T13:57:14.168755+0000","last_fullsized":"2024-04-25T13:57:14.168755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.164994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841644+0000","last_change":"2024-04-25T13:57:12.145617+0000","last_active":"2024-04-25T13:57:13.841644+0000","last_peered":"2024-04-25T13:57:13.841644+0000","last_clean":"2024-04-25T13:57:13.841644+0000","last_became_active":"2024-04-25T13:57:12.145263+0000","last_became_peered":"2024-04-25T13:57:12.145263+0000","last_unstale":"2024-04-25T13:57:13.841644+0000","last_undegraded":"2024-04-25T13:57:13.841644+0000","last_fullsized":"2024-04-25T13:57:13.841644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.886538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168886+0000","last_change":"2024-04-25T13:57:11.136972+0000","last_active":"2024-04-25T13:57:14.168886+0000","last_peered":"2024-04-25T13:57:14.168886+0000","last_clean":"2024-04-25T13:57:14.168886+0000","last_became_active":"2024-04-25T13:57:11.136392+0000","last_became_peered":"2024-04-25T13:57:11.136392+0000","last_unstale":"2024-04-25T13:57:14.168886+0000","last_undegraded":"2024-04-25T13:57:14.168886+0000","last_fullsized":"2024-04-25T13:57:14.168886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:45.841670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841761+0000","last_change":"2024-04-25T13:57:12.144905+0000","last_active":"2024-04-25T13:57:13.841761+0000","last_peered":"2024-04-25T13:57:13.841761+0000","last_clean":"2024-04-25T13:57:13.841761+0000","last_became_active":"2024-04-25T13:57:12.144404+0000","last_became_peered":"2024-04-25T13:57:12.144404+0000","last_unstale":"2024-04-25T13:57:13.841761+0000","last_undegraded":"2024-04-25T13:57:13.841761+0000","last_fullsized":"2024-04-25T13:57:13.841761+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.186492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.078155+0000","last_change":"2024-04-25T13:57:11.139041+0000","last_active":"2024-04-25T13:57:15.078155+0000","last_peered":"2024-04-25T13:57:15.078155+0000","last_clean":"2024-04-25T13:57:15.078155+0000","last_became_active":"2024-04-25T13:57:11.138629+0000","last_became_peered":"2024-04-25T13:57:11.138629+0000","last_unstale":"2024-04-25T13:57:15.078155+0000","last_undegraded":"2024-04-25T13:57:15.078155+0000","last_fullsized":"2024-04-25T13:57:15.078155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:04.863015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162425+0000","last_change":"2024-04-25T13:57:12.160897+0000","last_active":"2024-04-25T13:57:12.162425+0000","last_peered":"2024-04-25T13:57:12.162425+0000","last_clean":"2024-04-25T13:57:12.162425+0000","last_became_active":"2024-04-25T13:57:12.160322+0000","last_became_peered":"2024-04-25T13:57:12.160322+0000","last_unstale":"2024-04-25T13:57:12.162425+0000","last_undegraded":"2024-04-25T13:57:12.162425+0000","last_fullsized":"2024-04-25T13:57:12.162425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:56.159435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.088199+0000","last_change":"2024-04-25T13:57:11.137104+0000","last_active":"2024-04-25T13:57:15.088199+0000","last_peered":"2024-04-25T13:57:15.088199+0000","last_clean":"2024-04-25T13:57:15.088199+0000","last_became_active":"2024-04-25T13:57:11.136720+0000","last_became_peered":"2024-04-25T13:57:11.136720+0000","last_unstale":"2024-04-25T13:57:15.088199+0000","last_undegraded":"2024-04-25T13:57:15.088199+0000","last_fullsized":"2024-04-25T13:57:15.088199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:41.262776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.148506+0000","last_change":"2024-04-25T13:57:12.147831+0000","last_active":"2024-04-25T13:57:12.148506+0000","last_peered":"2024-04-25T13:57:12.148506+0000","last_clean":"2024-04-25T13:57:12.148506+0000","last_became_active":"2024-04-25T13:57:12.147270+0000","last_became_peered":"2024-04-25T13:57:12.147270+0000","last_unstale":"2024-04-25T13:57:12.148506+0000","last_undegraded":"2024-04-25T13:57:12.148506+0000","last_fullsized":"2024-04-25T13:57:12.148506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:22.786101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168873+0000","last_change":"2024-04-25T13:57:11.138254+0000","last_active":"2024-04-25T13:57:14.168873+0000","last_peered":"2024-04-25T13:57:14.168873+0000","last_clean":"2024-04-25T13:57:14.168873+0000","last_became_active":"2024-04-25T13:57:11.137924+0000","last_became_peered":"2024-04-25T13:57:11.137924+0000","last_unstale":"2024-04-25T13:57:14.168873+0000","last_undegraded":"2024-04-25T13:57:14.168873+0000","last_fullsized":"2024-04-25T13:57:14.168873+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:27.746203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326437+0000","last_change":"2024-04-25T13:57:12.145743+0000","last_active":"2024-04-25T13:57:13.326437+0000","last_peered":"2024-04-25T13:57:13.326437+0000","last_clean":"2024-04-25T13:57:13.326437+0000","last_became_active":"2024-04-25T13:57:12.145337+0000","last_became_peered":"2024-04-25T13:57:12.145337+0000","last_unstale":"2024-04-25T13:57:13.326437+0000","last_undegraded":"2024-04-25T13:57:13.326437+0000","last_fullsized":"2024-04-25T13:57:13.326437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.692900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'4","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:16.090779+0000","last_change":"2024-04-25T13:57:11.138059+0000","last_active":"2024-04-25T13:57:16.090779+0000","last_peered":"2024-04-25T13:57:16.090779+0000","last_clean":"2024-04-25T13:57:16.090779+0000","last_became_active":"2024-04-25T13:57:11.137698+0000","last_became_peered":"2024-04-25T13:57:11.137698+0000","last_unstale":"2024-04-25T13:57:16.090779+0000","last_undegraded":"2024-04-25T13:57:16.090779+0000","last_fullsized":"2024-04-25T13:57:16.090779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:03.595126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326123+0000","last_change":"2024-04-25T13:57:12.155507+0000","last_active":"2024-04-25T13:57:13.326123+0000","last_peered":"2024-04-25T13:57:13.326123+0000","last_clean":"2024-04-25T13:57:13.326123+0000","last_became_active":"2024-04-25T13:57:12.155175+0000","last_became_peered":"2024-04-25T13:57:12.155175+0000","last_unstale":"2024-04-25T13:57:13.326123+0000","last_undegraded":"2024-04-25T13:57:13.326123+0000","last_fullsized":"2024-04-25T13:57:13.326123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:21.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'8","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.163736+0000","last_change":"2024-04-25T13:57:11.141672+0000","last_active":"2024-04-25T13:57:12.163736+0000","last_peered":"2024-04-25T13:57:12.163736+0000","last_clean":"2024-04-25T13:57:12.163736+0000","last_became_active":"2024-04-25T13:57:11.140957+0000","last_became_peered":"2024-04-25T13:57:11.140957+0000","last_unstale":"2024-04-25T13:57:12.163736+0000","last_undegraded":"2024-04-25T13:57:12.163736+0000","last_fullsized":"2024-04-25T13:57:12.163736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:47.362949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841579+0000","last_change":"2024-04-25T13:57:12.147881+0000","last_active":"2024-04-25T13:57:13.841579+0000","last_peered":"2024-04-25T13:57:13.841579+0000","last_clean":"2024-04-25T13:57:13.841579+0000","last_became_active":"2024-04-25T13:57:12.146200+0000","last_became_peered":"2024-04-25T13:57:12.146200+0000","last_unstale":"2024-04-25T13:57:13.841579+0000","last_undegraded":"2024-04-25T13:57:13.841579+0000","last_fullsized":"2024-04-25T13:57:13.841579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:51.711045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.160439+0000","last_change":"2024-04-25T13:57:11.139574+0000","last_active":"2024-04-25T13:57:12.160439+0000","last_peered":"2024-04-25T13:57:12.160439+0000","last_clean":"2024-04-25T13:57:12.160439+0000","last_became_active":"2024-04-25T13:57:11.139232+0000","last_became_peered":"2024-04-25T13:57:11.139232+0000","last_unstale":"2024-04-25T13:57:12.160439+0000","last_undegraded":"2024-04-25T13:57:12.160439+0000","last_fullsized":"2024-04-25T13:57:12.160439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:08.590270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143130+0000","last_change":"2024-04-25T13:57:12.147122+0000","last_active":"2024-04-25T13:57:13.143130+0000","last_peered":"2024-04-25T13:57:13.143130+0000","last_clean":"2024-04-25T13:57:13.143130+0000","last_became_active":"2024-04-25T13:57:12.146708+0000","last_became_peered":"2024-04-25T13:57:12.146708+0000","last_unstale":"2024-04-25T13:57:13.143130+0000","last_undegraded":"2024-04-25T13:57:13.143130+0000","last_fullsized":"2024-04-25T13:57:13.143130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:47.858290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142519+0000","last_change":"2024-04-25T13:57:11.146288+0000","last_active":"2024-04-25T13:57:13.142519+0000","last_peered":"2024-04-25T13:57:13.142519+0000","last_clean":"2024-04-25T13:57:13.142519+0000","last_became_active":"2024-04-25T13:57:11.145851+0000","last_became_peered":"2024-04-25T13:57:11.145851+0000","last_unstale":"2024-04-25T13:57:13.142519+0000","last_undegraded":"2024-04-25T13:57:13.142519+0000","last_fullsized":"2024-04-25T13:57:13.142519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143855+0000","last_change":"2024-04-25T13:57:12.154576+0000","last_active":"2024-04-25T13:57:13.143855+0000","last_peered":"2024-04-25T13:57:13.143855+0000","last_clean":"2024-04-25T13:57:13.143855+0000","last_became_active":"2024-04-25T13:57:12.154254+0000","last_became_peered":"2024-04-25T13:57:12.154254+0000","last_unstale":"2024-04-25T13:57:13.143855+0000","last_undegraded":"2024-04-25T13:57:13.143855+0000","last_fullsized":"2024-04-25T13:57:13.143855+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:33.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326160+0000","last_change":"2024-04-25T13:57:11.139941+0000","last_active":"2024-04-25T13:57:13.326160+0000","last_peered":"2024-04-25T13:57:13.326160+0000","last_clean":"2024-04-25T13:57:13.326160+0000","last_became_active":"2024-04-25T13:57:11.139614+0000","last_became_peered":"2024-04-25T13:57:11.139614+0000","last_unstale":"2024-04-25T13:57:13.326160+0000","last_undegraded":"2024-04-25T13:57:13.326160+0000","last_fullsized":"2024-04-25T13:57:13.326160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:39.656085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143559+0000","last_change":"2024-04-25T13:57:12.155935+0000","last_active":"2024-04-25T13:57:13.143559+0000","last_peered":"2024-04-25T13:57:13.143559+0000","last_clean":"2024-04-25T13:57:13.143559+0000","last_became_active":"2024-04-25T13:57:12.155261+0000","last_became_peered":"2024-04-25T13:57:12.155261+0000","last_unstale":"2024-04-25T13:57:13.143559+0000","last_undegraded":"2024-04-25T13:57:13.143559+0000","last_fullsized":"2024-04-25T13:57:13.143559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:54.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841903+0000","last_change":"2024-04-25T13:57:12.148525+0000","last_active":"2024-04-25T13:57:13.841903+0000","last_peered":"2024-04-25T13:57:13.841903+0000","last_clean":"2024-04-25T13:57:13.841903+0000","last_became_active":"2024-04-25T13:57:12.148160+0000","last_became_peered":"2024-04-25T13:57:12.148160+0000","last_unstale":"2024-04-25T13:57:13.841903+0000","last_undegraded":"2024-04-25T13:57:13.841903+0000","last_fullsized":"2024-04-25T13:57:13.841903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:05.928439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.127234+0000","last_change":"2024-04-25T13:57:11.135226+0000","last_active":"2024-04-25T13:57:12.127234+0000","last_peered":"2024-04-25T13:57:12.127234+0000","last_clean":"2024-04-25T13:57:12.127234+0000","last_became_active":"2024-04-25T13:57:11.134886+0000","last_became_peered":"2024-04-25T13:57:11.134886+0000","last_unstale":"2024-04-25T13:57:12.127234+0000","last_undegraded":"2024-04-25T13:57:12.127234+0000","last_fullsized":"2024-04-25T13:57:12.127234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:00.153597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213255+0000","last_change":"2024-04-25T13:57:12.149758+0000","last_active":"2024-04-25T13:57:13.213255+0000","last_peered":"2024-04-25T13:57:13.213255+0000","last_clean":"2024-04-25T13:57:13.213255+0000","last_became_active":"2024-04-25T13:57:12.149428+0000","last_became_peered":"2024-04-25T13:57:12.149428+0000","last_unstale":"2024-04-25T13:57:13.213255+0000","last_undegraded":"2024-04-25T13:57:13.213255+0000","last_fullsized":"2024-04-25T13:57:13.213255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.739074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.088819+0000","last_change":"2024-04-25T13:57:11.135968+0000","last_active":"2024-04-25T13:57:15.088819+0000","last_peered":"2024-04-25T13:57:15.088819+0000","last_clean":"2024-04-25T13:57:15.088819+0000","last_became_active":"2024-04-25T13:57:11.135577+0000","last_became_peered":"2024-04-25T13:57:11.135577+0000","last_unstale":"2024-04-25T13:57:15.088819+0000","last_undegraded":"2024-04-25T13:57:15.088819+0000","last_fullsized":"2024-04-25T13:57:15.088819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:00.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.325913+0000","last_change":"2024-04-25T13:55:57.666399+0000","last_active":"2024-04-25T13:57:13.325913+0000","last_peered":"2024-04-25T13:57:13.325913+0000","last_clean":"2024-04-25T13:57:13.325913+0000","last_became_active":"2024-04-25T13:55:57.660470+0000","last_became_peered":"2024-04-25T13:55:57.660470+0000","last_unstale":"2024-04-25T13:57:13.325913+0000","last_undegraded":"2024-04-25T13:57:13.325913+0000","last_fullsized":"2024-04-25T13:57:13.325913+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_clean_scrub_stamp":"2024-04-25T13:52:33.081441+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:22.788632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.143869+0000","last_change":"2024-04-25T13:57:12.141923+0000","last_active":"2024-04-25T13:57:12.143869+0000","last_peered":"2024-04-25T13:57:12.143869+0000","last_clean":"2024-04-25T13:57:12.143869+0000","last_became_active":"2024-04-25T13:57:12.140946+0000","last_became_peered":"2024-04-25T13:57:12.140946+0000","last_unstale":"2024-04-25T13:57:12.143869+0000","last_undegraded":"2024-04-25T13:57:12.143869+0000","last_fullsized":"2024-04-25T13:57:12.143869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:28.362262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.167676+0000","last_change":"2024-04-25T13:57:11.139851+0000","last_active":"2024-04-25T13:57:14.167676+0000","last_peered":"2024-04-25T13:57:14.167676+0000","last_clean":"2024-04-25T13:57:14.167676+0000","last_became_active":"2024-04-25T13:57:11.139476+0000","last_became_peered":"2024-04-25T13:57:11.139476+0000","last_unstale":"2024-04-25T13:57:14.167676+0000","last_undegraded":"2024-04-25T13:57:14.167676+0000","last_fullsized":"2024-04-25T13:57:14.167676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:29.435917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.136341+0000","last_change":"2024-04-25T13:57:11.146054+0000","last_active":"2024-04-25T13:57:12.136341+0000","last_peered":"2024-04-25T13:57:12.136341+0000","last_clean":"2024-04-25T13:57:12.136341+0000","last_became_active":"2024-04-25T13:57:11.145616+0000","last_became_peered":"2024-04-25T13:57:11.145616+0000","last_unstale":"2024-04-25T13:57:12.136341+0000","last_undegraded":"2024-04-25T13:57:12.136341+0000","last_fullsized":"2024-04-25T13:57:12.136341+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:09.825563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.136798+0000","last_change":"2024-04-25T13:57:12.136204+0000","last_active":"2024-04-25T13:57:12.136798+0000","last_peered":"2024-04-25T13:57:12.136798+0000","last_clean":"2024-04-25T13:57:12.136798+0000","last_became_active":"2024-04-25T13:57:12.135798+0000","last_became_peered":"2024-04-25T13:57:12.135798+0000","last_unstale":"2024-04-25T13:57:12.136798+0000","last_undegraded":"2024-04-25T13:57:12.136798+0000","last_fullsized":"2024-04-25T13:57:12.136798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.471835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.130923+0000","last_change":"2024-04-25T13:57:11.138699+0000","last_active":"2024-04-25T13:57:14.130923+0000","last_peered":"2024-04-25T13:57:14.130923+0000","last_clean":"2024-04-25T13:57:14.130923+0000","last_became_active":"2024-04-25T13:57:11.138335+0000","last_became_peered":"2024-04-25T13:57:11.138335+0000","last_unstale":"2024-04-25T13:57:14.130923+0000","last_undegraded":"2024-04-25T13:57:14.130923+0000","last_fullsized":"2024-04-25T13:57:14.130923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:44.413228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213623+0000","last_change":"2024-04-25T13:57:12.155851+0000","last_active":"2024-04-25T13:57:13.213623+0000","last_peered":"2024-04-25T13:57:13.213623+0000","last_clean":"2024-04-25T13:57:13.213623+0000","last_became_active":"2024-04-25T13:57:12.155416+0000","last_became_peered":"2024-04-25T13:57:12.155416+0000","last_unstale":"2024-04-25T13:57:13.213623+0000","last_undegraded":"2024-04-25T13:57:13.213623+0000","last_fullsized":"2024-04-25T13:57:13.213623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:22.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.166686+0000","last_change":"2024-04-25T13:57:11.147465+0000","last_active":"2024-04-25T13:57:14.166686+0000","last_peered":"2024-04-25T13:57:14.166686+0000","last_clean":"2024-04-25T13:57:14.166686+0000","last_became_active":"2024-04-25T13:57:11.147096+0000","last_became_peered":"2024-04-25T13:57:11.147096+0000","last_unstale":"2024-04-25T13:57:14.166686+0000","last_undegraded":"2024-04-25T13:57:14.166686+0000","last_fullsized":"2024-04-25T13:57:14.166686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.229356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841536+0000","last_change":"2024-04-25T13:57:12.146446+0000","last_active":"2024-04-25T13:57:13.841536+0000","last_peered":"2024-04-25T13:57:13.841536+0000","last_clean":"2024-04-25T13:57:13.841536+0000","last_became_active":"2024-04-25T13:57:12.146129+0000","last_became_peered":"2024-04-25T13:57:12.146129+0000","last_unstale":"2024-04-25T13:57:13.841536+0000","last_undegraded":"2024-04-25T13:57:13.841536+0000","last_fullsized":"2024-04-25T13:57:13.841536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.977663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168778+0000","last_change":"2024-04-25T13:57:11.145127+0000","last_active":"2024-04-25T13:57:14.168778+0000","last_peered":"2024-04-25T13:57:14.168778+0000","last_clean":"2024-04-25T13:57:14.168778+0000","last_became_active":"2024-04-25T13:57:11.144822+0000","last_became_peered":"2024-04-25T13:57:11.144822+0000","last_unstale":"2024-04-25T13:57:14.168778+0000","last_undegraded":"2024-04-25T13:57:14.168778+0000","last_fullsized":"2024-04-25T13:57:14.168778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:00.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.160222+0000","last_change":"2024-04-25T13:57:12.156184+0000","last_active":"2024-04-25T13:57:12.160222+0000","last_peered":"2024-04-25T13:57:12.160222+0000","last_clean":"2024-04-25T13:57:12.160222+0000","last_became_active":"2024-04-25T13:57:12.155855+0000","last_became_peered":"2024-04-25T13:57:12.155855+0000","last_unstale":"2024-04-25T13:57:12.160222+0000","last_undegraded":"2024-04-25T13:57:12.160222+0000","last_fullsized":"2024-04-25T13:57:12.160222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:09.573860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:14.168413+0000","last_change":"2024-04-25T13:57:11.137616+0000","last_active":"2024-04-25T13:57:14.168413+0000","last_peered":"2024-04-25T13:57:14.168413+0000","last_clean":"2024-04-25T13:57:14.168413+0000","last_became_active":"2024-04-25T13:57:11.137041+0000","last_became_peered":"2024-04-25T13:57:11.137041+0000","last_unstale":"2024-04-25T13:57:14.168413+0000","last_undegraded":"2024-04-25T13:57:14.168413+0000","last_fullsized":"2024-04-25T13:57:14.168413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:54.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142868+0000","last_change":"2024-04-25T13:57:12.151223+0000","last_active":"2024-04-25T13:57:13.142868+0000","last_peered":"2024-04-25T13:57:13.142868+0000","last_clean":"2024-04-25T13:57:13.142868+0000","last_became_active":"2024-04-25T13:57:12.150871+0000","last_became_peered":"2024-04-25T13:57:12.150871+0000","last_unstale":"2024-04-25T13:57:13.142868+0000","last_undegraded":"2024-04-25T13:57:13.142868+0000","last_fullsized":"2024-04-25T13:57:13.142868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:17.957728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786569+0000","last_change":"2024-04-25T13:57:11.136524+0000","last_active":"2024-04-25T13:57:13.786569+0000","last_peered":"2024-04-25T13:57:13.786569+0000","last_clean":"2024-04-25T13:57:13.786569+0000","last_became_active":"2024-04-25T13:57:11.136152+0000","last_became_peered":"2024-04-25T13:57:11.136152+0000","last_unstale":"2024-04-25T13:57:13.786569+0000","last_undegraded":"2024-04-25T13:57:13.786569+0000","last_fullsized":"2024-04-25T13:57:13.786569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:23.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213401+0000","last_change":"2024-04-25T13:57:12.157641+0000","last_active":"2024-04-25T13:57:13.213401+0000","last_peered":"2024-04-25T13:57:13.213401+0000","last_clean":"2024-04-25T13:57:13.213401+0000","last_became_active":"2024-04-25T13:57:12.157281+0000","last_became_peered":"2024-04-25T13:57:12.157281+0000","last_unstale":"2024-04-25T13:57:13.213401+0000","last_undegraded":"2024-04-25T13:57:13.213401+0000","last_fullsized":"2024-04-25T13:57:13.213401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:28.245071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.162634+0000","last_change":"2024-04-25T13:57:11.139036+0000","last_active":"2024-04-25T13:57:12.162634+0000","last_peered":"2024-04-25T13:57:12.162634+0000","last_clean":"2024-04-25T13:57:12.162634+0000","last_became_active":"2024-04-25T13:57:11.138681+0000","last_became_peered":"2024-04-25T13:57:11.138681+0000","last_unstale":"2024-04-25T13:57:12.162634+0000","last_undegraded":"2024-04-25T13:57:12.162634+0000","last_fullsized":"2024-04-25T13:57:12.162634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:48.584313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326529+0000","last_change":"2024-04-25T13:57:12.153072+0000","last_active":"2024-04-25T13:57:13.326529+0000","last_peered":"2024-04-25T13:57:13.326529+0000","last_clean":"2024-04-25T13:57:13.326529+0000","last_became_active":"2024-04-25T13:57:12.151556+0000","last_became_peered":"2024-04-25T13:57:12.151556+0000","last_unstale":"2024-04-25T13:57:13.326529+0000","last_undegraded":"2024-04-25T13:57:13.326529+0000","last_fullsized":"2024-04-25T13:57:13.326529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:20.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:15.089122+0000","last_change":"2024-04-25T13:57:11.148823+0000","last_active":"2024-04-25T13:57:15.089122+0000","last_peered":"2024-04-25T13:57:15.089122+0000","last_clean":"2024-04-25T13:57:15.089122+0000","last_became_active":"2024-04-25T13:57:11.148436+0000","last_became_peered":"2024-04-25T13:57:11.148436+0000","last_unstale":"2024-04-25T13:57:15.089122+0000","last_undegraded":"2024-04-25T13:57:15.089122+0000","last_fullsized":"2024-04-25T13:57:15.089122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:28.496184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326536+0000","last_change":"2024-04-25T13:57:12.149082+0000","last_active":"2024-04-25T13:57:13.326536+0000","last_peered":"2024-04-25T13:57:13.326536+0000","last_clean":"2024-04-25T13:57:13.326536+0000","last_became_active":"2024-04-25T13:57:12.148260+0000","last_became_peered":"2024-04-25T13:57:12.148260+0000","last_unstale":"2024-04-25T13:57:13.326536+0000","last_undegraded":"2024-04-25T13:57:13.326536+0000","last_fullsized":"2024-04-25T13:57:13.326536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:49.733114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841903+0000","last_change":"2024-04-25T13:57:11.136736+0000","last_active":"2024-04-25T13:57:13.841903+0000","last_peered":"2024-04-25T13:57:13.841903+0000","last_clean":"2024-04-25T13:57:13.841903+0000","last_became_active":"2024-04-25T13:57:11.135831+0000","last_became_peered":"2024-04-25T13:57:11.135831+0000","last_unstale":"2024-04-25T13:57:13.841903+0000","last_undegraded":"2024-04-25T13:57:13.841903+0000","last_fullsized":"2024-04-25T13:57:13.841903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:49.654642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326296+0000","last_change":"2024-04-25T13:57:12.149797+0000","last_active":"2024-04-25T13:57:13.326296+0000","last_peered":"2024-04-25T13:57:13.326296+0000","last_clean":"2024-04-25T13:57:13.326296+0000","last_became_active":"2024-04-25T13:57:12.149413+0000","last_became_peered":"2024-04-25T13:57:12.149413+0000","last_unstale":"2024-04-25T13:57:13.326296+0000","last_undegraded":"2024-04-25T13:57:13.326296+0000","last_fullsized":"2024-04-25T13:57:13.326296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:58.517208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.154403+0000","last_change":"2024-04-25T13:57:11.138419+0000","last_active":"2024-04-25T13:57:12.154403+0000","last_peered":"2024-04-25T13:57:12.154403+0000","last_clean":"2024-04-25T13:57:12.154403+0000","last_became_active":"2024-04-25T13:57:11.138058+0000","last_became_peered":"2024-04-25T13:57:11.138058+0000","last_unstale":"2024-04-25T13:57:12.154403+0000","last_undegraded":"2024-04-25T13:57:12.154403+0000","last_fullsized":"2024-04-25T13:57:12.154403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:24.231004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.326547+0000","last_change":"2024-04-25T13:57:12.150247+0000","last_active":"2024-04-25T13:57:13.326547+0000","last_peered":"2024-04-25T13:57:13.326547+0000","last_clean":"2024-04-25T13:57:13.326547+0000","last_became_active":"2024-04-25T13:57:12.149882+0000","last_became_peered":"2024-04-25T13:57:12.149882+0000","last_unstale":"2024-04-25T13:57:13.326547+0000","last_undegraded":"2024-04-25T13:57:13.326547+0000","last_fullsized":"2024-04-25T13:57:13.326547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.624928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143138+0000","last_change":"2024-04-25T13:57:12.154618+0000","last_active":"2024-04-25T13:57:13.143138+0000","last_peered":"2024-04-25T13:57:13.143138+0000","last_clean":"2024-04-25T13:57:13.143138+0000","last_became_active":"2024-04-25T13:57:12.153648+0000","last_became_peered":"2024-04-25T13:57:12.153648+0000","last_unstale":"2024-04-25T13:57:13.143138+0000","last_undegraded":"2024-04-25T13:57:13.143138+0000","last_fullsized":"2024-04-25T13:57:13.143138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:24.010980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786093+0000","last_change":"2024-04-25T13:57:12.143097+0000","last_active":"2024-04-25T13:57:13.786093+0000","last_peered":"2024-04-25T13:57:13.786093+0000","last_clean":"2024-04-25T13:57:13.786093+0000","last_became_active":"2024-04-25T13:57:12.142295+0000","last_became_peered":"2024-04-25T13:57:12.142295+0000","last_unstale":"2024-04-25T13:57:13.786093+0000","last_undegraded":"2024-04-25T13:57:13.786093+0000","last_fullsized":"2024-04-25T13:57:13.786093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.907643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786373+0000","last_change":"2024-04-25T13:57:12.155659+0000","last_active":"2024-04-25T13:57:13.786373+0000","last_peered":"2024-04-25T13:57:13.786373+0000","last_clean":"2024-04-25T13:57:13.786373+0000","last_became_active":"2024-04-25T13:57:12.155308+0000","last_became_peered":"2024-04-25T13:57:12.155308+0000","last_unstale":"2024-04-25T13:57:13.786373+0000","last_undegraded":"2024-04-25T13:57:13.786373+0000","last_fullsized":"2024-04-25T13:57:13.786373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:12.361660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841274+0000","last_change":"2024-04-25T13:57:12.145750+0000","last_active":"2024-04-25T13:57:13.841274+0000","last_peered":"2024-04-25T13:57:13.841274+0000","last_clean":"2024-04-25T13:57:13.841274+0000","last_became_active":"2024-04-25T13:57:12.145339+0000","last_became_peered":"2024-04-25T13:57:12.145339+0000","last_unstale":"2024-04-25T13:57:13.841274+0000","last_undegraded":"2024-04-25T13:57:13.841274+0000","last_fullsized":"2024-04-25T13:57:13.841274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:18.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.144452+0000","last_change":"2024-04-25T13:57:12.142134+0000","last_active":"2024-04-25T13:57:12.144452+0000","last_peered":"2024-04-25T13:57:12.144452+0000","last_clean":"2024-04-25T13:57:12.144452+0000","last_became_active":"2024-04-25T13:57:12.141776+0000","last_became_peered":"2024-04-25T13:57:12.141776+0000","last_unstale":"2024-04-25T13:57:12.144452+0000","last_undegraded":"2024-04-25T13:57:12.144452+0000","last_fullsized":"2024-04-25T13:57:12.144452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:09.817661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.143137+0000","last_change":"2024-04-25T13:57:12.146269+0000","last_active":"2024-04-25T13:57:13.143137+0000","last_peered":"2024-04-25T13:57:13.143137+0000","last_clean":"2024-04-25T13:57:13.143137+0000","last_became_active":"2024-04-25T13:57:12.145880+0000","last_became_peered":"2024-04-25T13:57:12.145880+0000","last_unstale":"2024-04-25T13:57:13.143137+0000","last_undegraded":"2024-04-25T13:57:13.143137+0000","last_fullsized":"2024-04-25T13:57:13.143137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:08.928272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.786752+0000","last_change":"2024-04-25T13:57:12.147818+0000","last_active":"2024-04-25T13:57:13.786752+0000","last_peered":"2024-04-25T13:57:13.786752+0000","last_clean":"2024-04-25T13:57:13.786752+0000","last_became_active":"2024-04-25T13:57:12.146764+0000","last_became_peered":"2024-04-25T13:57:12.146764+0000","last_unstale":"2024-04-25T13:57:13.786752+0000","last_undegraded":"2024-04-25T13:57:13.786752+0000","last_fullsized":"2024-04-25T13:57:13.786752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.750115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213475+0000","last_change":"2024-04-25T13:57:12.152591+0000","last_active":"2024-04-25T13:57:13.213475+0000","last_peered":"2024-04-25T13:57:13.213475+0000","last_clean":"2024-04-25T13:57:13.213475+0000","last_became_active":"2024-04-25T13:57:12.152198+0000","last_became_peered":"2024-04-25T13:57:12.152198+0000","last_unstale":"2024-04-25T13:57:13.213475+0000","last_undegraded":"2024-04-25T13:57:13.213475+0000","last_fullsized":"2024-04-25T13:57:13.213475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:05.868774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841628+0000","last_change":"2024-04-25T13:57:12.154924+0000","last_active":"2024-04-25T13:57:13.841628+0000","last_peered":"2024-04-25T13:57:13.841628+0000","last_clean":"2024-04-25T13:57:13.841628+0000","last_became_active":"2024-04-25T13:57:12.154549+0000","last_became_peered":"2024-04-25T13:57:12.154549+0000","last_unstale":"2024-04-25T13:57:13.841628+0000","last_undegraded":"2024-04-25T13:57:13.841628+0000","last_fullsized":"2024-04-25T13:57:13.841628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:49.418436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.213103+0000","last_change":"2024-04-25T13:57:12.155774+0000","last_active":"2024-04-25T13:57:13.213103+0000","last_peered":"2024-04-25T13:57:13.213103+0000","last_clean":"2024-04-25T13:57:13.213103+0000","last_became_active":"2024-04-25T13:57:12.155352+0000","last_became_peered":"2024-04-25T13:57:12.155352+0000","last_unstale":"2024-04-25T13:57:13.213103+0000","last_undegraded":"2024-04-25T13:57:13.213103+0000","last_fullsized":"2024-04-25T13:57:13.213103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:46.854558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.841775+0000","last_change":"2024-04-25T13:57:12.158672+0000","last_active":"2024-04-25T13:57:13.841775+0000","last_peered":"2024-04-25T13:57:13.841775+0000","last_clean":"2024-04-25T13:57:13.841775+0000","last_became_active":"2024-04-25T13:57:12.158391+0000","last_became_peered":"2024-04-25T13:57:12.158391+0000","last_unstale":"2024-04-25T13:57:13.841775+0000","last_undegraded":"2024-04-25T13:57:13.841775+0000","last_fullsized":"2024-04-25T13:57:13.841775+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.369907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T13:57:12.140017+0000","last_change":"2024-04-25T13:57:12.139310+0000","last_active":"2024-04-25T13:57:12.140017+0000","last_peered":"2024-04-25T13:57:12.140017+0000","last_clean":"2024-04-25T13:57:12.140017+0000","last_became_active":"2024-04-25T13:57:12.138942+0000","last_became_peered":"2024-04-25T13:57:12.138942+0000","last_unstale":"2024-04-25T13:57:12.140017+0000","last_undegraded":"2024-04-25T13:57:12.140017+0000","last_fullsized":"2024-04-25T13:57:12.140017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:55.906480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T13:57:13.142870+0000","last_change":"2024-04-25T13:57:12.147451+0000","last_active":"2024-04-25T13:57:13.142870+0000","last_peered":"2024-04-25T13:57:13.142870+0000","last_clean":"2024-04-25T13:57:13.142870+0000","last_became_active":"2024-04-25T13:57:12.146746+0000","last_became_peered":"2024-04-25T13:57:12.146746+0000","last_unstale":"2024-04-25T13:57:13.142870+0000","last_undegraded":"2024-04-25T13:57:13.142870+0000","last_fullsized":"2024-04-25T13:57:13.142870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247568,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5780,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":71073,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710734,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":802816,"data_stored":384344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906963,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":892928,"data_stored":977320,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070488,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":802816,"data_stored":384344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266717,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":794624,"data_stored":375796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":974712,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626471,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":786432,"data_stored":375192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":827392,"data_stored":404657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":802816,"data_stored":383852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":839680,"data_stored":393715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":819200,"data_stored":384163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":811008,"data_stored":384797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":975997,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38193,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:57:19.287 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:57:19.287 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T13:57:19.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T13:57:19.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:19 smithi129 ceph-mon[32569]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:19.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:19 smithi129 ceph-mon[32569]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:19 smithi129 ceph-mon[32569]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-04-25T13:57:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:19 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/371971048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:19.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:19 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/585199612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:19.417 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:19 smithi112 ceph-mon[32617]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:19.417 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:19 smithi112 ceph-mon[32617]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:57:19.418 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:19 smithi112 ceph-mon[32617]: pgmap v280: 97 pgs: 97 active+clean; 582 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.3 KiB/s wr, 16 op/s 2024-04-25T13:57:19.418 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:19 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/371971048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:57:19.418 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:19 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/585199612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:19.555 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:20.437 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:20 smithi112 ceph-mon[32617]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:57:20.438 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:20 smithi112 ceph-mon[32617]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:57:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:20 smithi078 ceph-mon[29163]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:57:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:20 smithi078 ceph-mon[29163]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:57:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:20 smithi129 ceph-mon[32569]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:57:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:20 smithi129 ceph-mon[32569]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:57:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 -- 172.21.15.78:0/280777602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 msgr2=0x7f4094104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 --2- 172.21.15.78:0/280777602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f4094104660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4080009990 tx=0x7f4080009ca0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 -- 172.21.15.78:0/280777602 shutdown_connections 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 --2- 172.21.15.78:0/280777602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4094104bf0 0x7f409410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 --2- 172.21.15.78:0/280777602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4094103120 0x7f4094103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 --2- 172.21.15.78:0/280777602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f4094104660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 -- 172.21.15.78:0/280777602 >> 172.21.15.78:0/280777602 conn(0x7f40940fe760 msgr2=0x7f4094100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 -- 172.21.15.78:0/280777602 shutdown_connections 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.259+0000 7f409ac78700 1 -- 172.21.15.78:0/280777602 wait complete. 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 Processor -- start 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 -- start start 2024-04-25T13:57:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4094103120 0x7f409419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f409419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4094104bf0 0x7f40941a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f409410ec10 con 0x7f4094104250 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f409410ea90 con 0x7f4094103120 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f409ac78700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f409410ed90 con 0x7f4094104bf0 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f4093fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f409419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f4098a14700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4094103120 0x7f409419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f4098a14700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4094103120 0x7f409419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:48112/0 (socket says 172.21.15.78:48112) 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f4093fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f409419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53458/0 (socket says 172.21.15.78:53458) 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f4098a14700 1 -- 172.21.15.78:0/193872631 learned_addr learned my addr 172.21.15.78:0/193872631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.260+0000 7f4099215700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4094104bf0 0x7f40941a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4093fff700 1 -- 172.21.15.78:0/193872631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4094104bf0 msgr2=0x7f40941a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4093fff700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4094104bf0 0x7f40941a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4093fff700 1 -- 172.21.15.78:0/193872631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4094103120 msgr2=0x7f409419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4093fff700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4094103120 0x7f409419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4093fff700 1 -- 172.21.15.78:0/193872631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40941a23b0 con 0x7f4094104250 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4093fff700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f409419ad90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4080000c00 tx=0x7f40800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4091ffb700 1 -- 172.21.15.78:0/193872631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080027070 con 0x7f4094104250 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4091ffb700 1 -- 172.21.15.78:0/193872631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4080004000 con 0x7f4094104250 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40941a2640 con 0x7f4094104250 2024-04-25T13:57:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f4091ffb700 1 -- 172.21.15.78:0/193872631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080022cd0 con 0x7f4094104250 2024-04-25T13:57:21.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.261+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40941a2a90 con 0x7f4094104250 2024-04-25T13:57:21.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.262+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4094051190 con 0x7f4094104250 2024-04-25T13:57:21.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.264+0000 7f4091ffb700 1 -- 172.21.15.78:0/193872631 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f408002a070 con 0x7f4094104250 2024-04-25T13:57:21.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.264+0000 7f4091ffb700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40840afc10 0x7f40840b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.264+0000 7f4098a14700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40840afc10 0x7f40840b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.264+0000 7f4091ffb700 1 -- 172.21.15.78:0/193872631 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f40800dfdd0 con 0x7f4094104250 2024-04-25T13:57:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.265+0000 7f4098a14700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40840afc10 0x7f40840b2120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4088005950 tx=0x7f4088007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:21.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.268+0000 7f4091ffb700 1 -- 172.21.15.78:0/193872631 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4080013070 con 0x7f4094104250 2024-04-25T13:57:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:21 smithi078 ceph-mon[29163]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:57:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:21 smithi078 ceph-mon[29163]: pgmap v281: 97 pgs: 97 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 KiB/s wr, 20 op/s 2024-04-25T13:57:21.477 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:21 smithi129 ceph-mon[32569]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:57:21.477 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:21 smithi129 ceph-mon[32569]: pgmap v281: 97 pgs: 97 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 KiB/s wr, 20 op/s 2024-04-25T13:57:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:21 smithi112 ceph-mon[32617]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:57:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:21 smithi112 ceph-mon[32617]: pgmap v281: 97 pgs: 97 active+clean; 583 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 KiB/s wr, 20 op/s 2024-04-25T13:57:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.549+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f40941a2ff0 con 0x7f4094104250 2024-04-25T13:57:21.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.551+0000 7f4091ffb700 1 -- 172.21.15.78:0/193872631 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f40800a8a90 con 0x7f4094104250 2024-04-25T13:57:21.551 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-04-25T13:57:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBBYSpmYOnQIBAANNLr40ilvseZLu17+zlfcQ== 2024-04-25T13:57:21.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.554+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40840afc10 msgr2=0x7f40840b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:21.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.554+0000 7f409ac78700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40840afc10 0x7f40840b2120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4088005950 tx=0x7f4088007480 comp rx=0 tx=0).stop 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.554+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 msgr2=0x7f409419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.554+0000 7f409ac78700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f409419ad90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4080000c00 tx=0x7f40800047b0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 shutdown_connections 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40840afc10 0x7f40840b2120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4094104bf0 0x7f40941a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4094103120 0x7f409419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 --2- 172.21.15.78:0/193872631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4094104250 0x7f409419ad90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 >> 172.21.15.78:0/193872631 conn(0x7f40940fe760 msgr2=0x7f4094100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 shutdown_connections 2024-04-25T13:57:21.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:21.555+0000 7f409ac78700 1 -- 172.21.15.78:0/193872631 wait complete. 2024-04-25T13:57:22.244 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:57:22.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T13:57:22.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:57:22.280 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:57:22.280 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:57:22.280 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T13:57:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:22 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/193872631' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:57:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:22 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/193872631' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:57:22.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:22 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/193872631' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:57:22.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:22 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/193872631' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:57:22.556 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:22 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/193872631' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:57:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:22 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/193872631' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:57:23.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:23 smithi078 ceph-mon[29163]: pgmap v282: 97 pgs: 97 active+clean; 618 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-04-25T13:57:23.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:23 smithi112 ceph-mon[32617]: pgmap v282: 97 pgs: 97 active+clean; 618 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-04-25T13:57:23.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:23 smithi129 ceph-mon[32569]: pgmap v282: 97 pgs: 97 active+clean; 618 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 -- 172.21.15.78:0/581623796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4103390 msgr2=0x7f5ed41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/581623796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4103390 0x7f5ed41037a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0099c0 tx=0x7f5ebc009cd0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 -- 172.21.15.78:0/581623796 shutdown_connections 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/581623796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed4108610 0x7f5ed410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/581623796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4103390 0x7f5ed41037a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/581623796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4107cc0 0x7f5ed41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 -- 172.21.15.78:0/581623796 >> 172.21.15.78:0/581623796 conn(0x7f5ed40fec90 msgr2=0x7f5ed41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 -- 172.21.15.78:0/581623796 shutdown_connections 2024-04-25T13:57:24.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.285+0000 7f5ed9b18700 1 -- 172.21.15.78:0/581623796 wait complete. 2024-04-25T13:57:24.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.286+0000 7f5ed9b18700 1 Processor -- start 2024-04-25T13:57:24.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.286+0000 7f5ed9b18700 1 -- start start 2024-04-25T13:57:24.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed9b18700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed4103390 0x7f5ed40ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:24.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed9b18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4107cc0 0x7f5ed40ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:24.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed9b18700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4108610 0x7f5ed40ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:24.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed9b18700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41132c0 con 0x7f5ed4108610 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed9b18700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4113140 con 0x7f5ed4107cc0 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed9b18700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4113440 con 0x7f5ed4103390 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4108610 0x7f5ed40ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4108610 0x7f5ed40ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53472/0 (socket says 172.21.15.78:53472) 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 -- 172.21.15.78:0/4187612635 learned_addr learned my addr 172.21.15.78:0/4187612635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed2ffd700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4107cc0 0x7f5ed40ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed37fe700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed4103390 0x7f5ed40ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 -- 172.21.15.78:0/4187612635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed4103390 msgr2=0x7f5ed40ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed4103390 0x7f5ed40ff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 -- 172.21.15.78:0/4187612635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4107cc0 msgr2=0x7f5ed40ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4107cc0 0x7f5ed40ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 -- 172.21.15.78:0/4187612635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41a9010 con 0x7f5ed4108610 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed3fff700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4108610 0x7f5ed40ffbf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00b970 tx=0x7f5ecc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed0ff9700 1 -- 172.21.15.78:0/4187612635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc00d860 con 0x7f5ed4108610 2024-04-25T13:57:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41a9300 con 0x7f5ed4108610 2024-04-25T13:57:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed0ff9700 1 -- 172.21.15.78:0/4187612635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ecc00e430 con 0x7f5ed4108610 2024-04-25T13:57:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.287+0000 7f5ed0ff9700 1 -- 172.21.15.78:0/4187612635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc017660 con 0x7f5ed4108610 2024-04-25T13:57:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.288+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41a9840 con 0x7f5ed4108610 2024-04-25T13:57:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.288+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed4051190 con 0x7f5ed4108610 2024-04-25T13:57:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.290+0000 7f5ed0ff9700 1 -- 172.21.15.78:0/4187612635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5ecc00da00 con 0x7f5ed4108610 2024-04-25T13:57:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.291+0000 7f5ed0ff9700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ec00afc10 0x7f5ec00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.291+0000 7f5ed37fe700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ec00afc10 0x7f5ec00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.291+0000 7f5ed0ff9700 1 -- 172.21.15.78:0/4187612635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5ecc0deb20 con 0x7f5ed4108610 2024-04-25T13:57:24.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.291+0000 7f5ed37fe700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ec00afc10 0x7f5ec00b2120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009990 tx=0x7f5ebc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.294+0000 7f5ed0ff9700 1 -- 172.21.15.78:0/4187612635 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ecc052ff0 con 0x7f5ed4108610 2024-04-25T13:57:24.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.550+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5ed41006a0 con 0x7f5ed4108610 2024-04-25T13:57:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.555+0000 7f5ed0ff9700 1 -- 172.21.15.78:0/4187612635 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284476 (secure 0 0 0) 0x7f5ecc013030 con 0x7f5ed4108610 2024-04-25T13:57:24.559 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ec00afc10 msgr2=0x7f5ec00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ec00afc10 0x7f5ec00b2120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009990 tx=0x7f5ebc000bc0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4108610 msgr2=0x7f5ed40ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4108610 0x7f5ed40ffbf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00b970 tx=0x7f5ecc00bc80 comp rx=0 tx=0).stop 2024-04-25T13:57:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 shutdown_connections 2024-04-25T13:57:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ec00afc10 0x7f5ec00b2120 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ed4108610 0x7f5ed40ffbf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed4103390 0x7f5ed40ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 --2- 172.21.15.78:0/4187612635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed4107cc0 0x7f5ed40ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.558+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 >> 172.21.15.78:0/4187612635 conn(0x7f5ed40fec90 msgr2=0x7f5ed406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.559+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 shutdown_connections 2024-04-25T13:57:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:24.559+0000 7f5ed9b18700 1 -- 172.21.15.78:0/4187612635 wait complete. 2024-04-25T13:57:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":3547239980},{"type":"v1","addr":"172.21.15.78:6801","nonce":3547239980}]},"active_addr":"172.21.15.78:6801/3547239980","active_change":"2024-04-25T13:50:08.038043+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.78:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":615323434}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":489464145}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":220839409}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2596351593}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":734292637}]}]} 2024-04-25T13:57:25.236 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T13:57:25.236 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T13:57:25.236 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:57:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:25 smithi078 ceph-mon[29163]: pgmap v283: 97 pgs: 97 active+clean; 618 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.3 KiB/s wr, 18 op/s 2024-04-25T13:57:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:25 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4187612635' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:57:25.472 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:25.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:25 smithi112 ceph-mon[32617]: pgmap v283: 97 pgs: 97 active+clean; 618 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.3 KiB/s wr, 18 op/s 2024-04-25T13:57:25.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:25 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4187612635' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:57:25.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:25 smithi129 ceph-mon[32569]: pgmap v283: 97 pgs: 97 active+clean; 618 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.3 KiB/s wr, 18 op/s 2024-04-25T13:57:25.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:25 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4187612635' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:57:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 -- 172.21.15.78:0/3739792660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c1033c0 msgr2=0x7f2a0c1037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 --2- 172.21.15.78:0/3739792660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c1033c0 0x7f2a0c1037d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00b0d0 tx=0x7f29fc00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 -- 172.21.15.78:0/3739792660 shutdown_connections 2024-04-25T13:57:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 --2- 172.21.15.78:0/3739792660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c108640 0x7f2a0c10ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 --2- 172.21.15.78:0/3739792660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c1033c0 0x7f2a0c1037d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 --2- 172.21.15.78:0/3739792660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c107cf0 0x7f2a0c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 -- 172.21.15.78:0/3739792660 >> 172.21.15.78:0/3739792660 conn(0x7f2a0c0fedb0 msgr2=0x7f2a0c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 -- 172.21.15.78:0/3739792660 shutdown_connections 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.201+0000 7f2a132a9700 1 -- 172.21.15.78:0/3739792660 wait complete. 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 Processor -- start 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 -- start start 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c1033c0 0x7f2a0c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c107cf0 0x7f2a0c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c108640 0x7f2a0c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c113270 con 0x7f2a0c108640 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1130f0 con 0x7f2a0c107cf0 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.202+0000 7f2a132a9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1133f0 con 0x7f2a0c1033c0 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c108640 0x7f2a0c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c108640 0x7f2a0c19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40936/0 (socket says 172.21.15.78:40936) 2024-04-25T13:57:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 -- 172.21.15.78:0/2929610369 learned_addr learned my addr 172.21.15.78:0/2929610369 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a10844700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c107cf0 0x7f2a0c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11045700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c1033c0 0x7f2a0c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 -- 172.21.15.78:0/2929610369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c1033c0 msgr2=0x7f2a0c1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c1033c0 0x7f2a0c1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 -- 172.21.15.78:0/2929610369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c107cf0 msgr2=0x7f2a0c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c107cf0 0x7f2a0c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 -- 172.21.15.78:0/2929610369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c19b100 con 0x7f2a0c108640 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a10844700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c107cf0 0x7f2a0c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a11846700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c108640 0x7f2a0c19aac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400c900 tx=0x7f2a0400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a0a7fc700 1 -- 172.21.15.78:0/2929610369 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04007b00 con 0x7f2a0c108640 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a0a7fc700 1 -- 172.21.15.78:0/2929610369 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a04007ca0 con 0x7f2a0c108640 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c19b3f0 con 0x7f2a0c108640 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a0a7fc700 1 -- 172.21.15.78:0/2929610369 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04018660 con 0x7f2a0c108640 2024-04-25T13:57:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.203+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c19b8e0 con 0x7f2a0c108640 2024-04-25T13:57:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.204+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c051190 con 0x7f2a0c108640 2024-04-25T13:57:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.206+0000 7f2a0a7fc700 1 -- 172.21.15.78:0/2929610369 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2a0400f450 con 0x7f2a0c108640 2024-04-25T13:57:27.206 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:27 smithi078 ceph-mon[29163]: pgmap v284: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-25T13:57:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.206+0000 7f2a0a7fc700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc10 0x7f29f40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.206+0000 7f2a11045700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc10 0x7f29f40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.207+0000 7f2a0a7fc700 1 -- 172.21.15.78:0/2929610369 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f2a040dec50 con 0x7f2a0c108640 2024-04-25T13:57:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.207+0000 7f2a11045700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc10 0x7f29f40b2120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009900 tx=0x7f29fc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:27.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.210+0000 7f2a0a7fc700 1 -- 172.21.15.78:0/2929610369 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a04014030 con 0x7f2a0c108640 2024-04-25T13:57:27.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.409+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a0c19bff0 con 0x7f2a0c108640 2024-04-25T13:57:27.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.410+0000 7f2a0a7fc700 1 -- 172.21.15.78:0/2929610369 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22243 (secure 0 0 0) 0x7f2a040a7910 con 0x7f2a0c108640 2024-04-25T13:57:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:27.410 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:57:13.134773+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:57:27.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc10 msgr2=0x7f29f40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:27.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc10 0x7f29f40b2120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009900 tx=0x7f29fc00b040 comp rx=0 tx=0).stop 2024-04-25T13:57:27.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c108640 msgr2=0x7f2a0c19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c108640 0x7f2a0c19aac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400c900 tx=0x7f2a0400ccc0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 shutdown_connections 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc10 0x7f29f40b2120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c108640 0x7f2a0c19aac0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c1033c0 0x7f2a0c1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 --2- 172.21.15.78:0/2929610369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c107cf0 0x7f2a0c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 >> 172.21.15.78:0/2929610369 conn(0x7f2a0c0fedb0 msgr2=0x7f2a0c075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.412+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 shutdown_connections 2024-04-25T13:57:27.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:27.413+0000 7f2a132a9700 1 -- 172.21.15.78:0/2929610369 wait complete. 2024-04-25T13:57:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:27 smithi112 ceph-mon[32617]: pgmap v284: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-25T13:57:27.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:27 smithi129 ceph-mon[32569]: pgmap v284: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-25T13:57:28.097 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T13:57:28.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:57:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:28.462 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:28 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2929610369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:28.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:28 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2929610369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:28 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2929610369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:29 smithi078 ceph-mon[29163]: pgmap v285: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-25T13:57:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:29 smithi112 ceph-mon[32617]: pgmap v285: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-25T13:57:29.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:29 smithi129 ceph-mon[32569]: pgmap v285: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-25T13:57:30.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 -- 172.21.15.78:0/963383300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 msgr2=0x7f51301084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:30.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 --2- 172.21.15.78:0/963383300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f51301084c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5120009990 tx=0x7f5120009ca0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 -- 172.21.15.78:0/963383300 shutdown_connections 2024-04-25T13:57:30.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 --2- 172.21.15.78:0/963383300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51301041a0 0x7f513010ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 --2- 172.21.15.78:0/963383300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f51301084c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 --2- 172.21.15.78:0/963383300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5130103850 0x7f5130103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 -- 172.21.15.78:0/963383300 >> 172.21.15.78:0/963383300 conn(0x7f51300fe370 msgr2=0x7f51301007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:30.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 -- 172.21.15.78:0/963383300 shutdown_connections 2024-04-25T13:57:30.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.213+0000 7f5135911700 1 -- 172.21.15.78:0/963383300 wait complete. 2024-04-25T13:57:30.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.214+0000 7f5135911700 1 Processor -- start 2024-04-25T13:57:30.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.214+0000 7f5135911700 1 -- start start 2024-04-25T13:57:30.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5130103850 0x7f513019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:30.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51301041a0 0x7f513019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:30.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:30.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5130113290 con 0x7f51301080b0 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5130113110 con 0x7f51301041a0 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5130113410 con 0x7f5130103850 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40950/0 (socket says 172.21.15.78:40950) 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 -- 172.21.15.78:0/987272376 learned_addr learned my addr 172.21.15.78:0/987272376 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512effd700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5130103850 0x7f513019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512e7fc700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51301041a0 0x7f513019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 -- 172.21.15.78:0/987272376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5130103850 msgr2=0x7f513019bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5130103850 0x7f513019bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 -- 172.21.15.78:0/987272376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51301041a0 msgr2=0x7f513019c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51301041a0 0x7f513019c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 -- 172.21.15.78:0/987272376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51301a6e30 con 0x7f51301080b0 2024-04-25T13:57:30.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512effd700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5130103850 0x7f513019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f512f7fe700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019c610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5128007d70 tx=0x7f512800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f513490f700 1 -- 172.21.15.78:0/987272376 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128016020 con 0x7f51301080b0 2024-04-25T13:57:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51301a7120 con 0x7f51301080b0 2024-04-25T13:57:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f513490f700 1 -- 172.21.15.78:0/987272376 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512800ad40 con 0x7f51301080b0 2024-04-25T13:57:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f513490f700 1 -- 172.21.15.78:0/987272376 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51280115e0 con 0x7f51301080b0 2024-04-25T13:57:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.215+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51301a7610 con 0x7f51301080b0 2024-04-25T13:57:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.216+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5130051190 con 0x7f51301080b0 2024-04-25T13:57:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.219+0000 7f513490f700 1 -- 172.21.15.78:0/987272376 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5128019020 con 0x7f51301080b0 2024-04-25T13:57:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.219+0000 7f513490f700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f51180afc10 0x7f51180b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.219+0000 7f512effd700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f51180afc10 0x7f51180b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.219+0000 7f513490f700 1 -- 172.21.15.78:0/987272376 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f51280def10 con 0x7f51301080b0 2024-04-25T13:57:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.220+0000 7f512effd700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f51180afc10 0x7f51180b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5120000c00 tx=0x7f5120005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:30.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.223+0000 7f513490f700 1 -- 172.21.15.78:0/987272376 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5128020020 con 0x7f51301080b0 2024-04-25T13:57:30.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.420+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f513010b650 con 0x7f51301080b0 2024-04-25T13:57:30.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.422+0000 7f513490f700 1 -- 172.21.15.78:0/987272376 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22243 (secure 0 0 0) 0x7f51280a7c50 con 0x7f51301080b0 2024-04-25T13:57:30.422 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:57:30.422 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:57:13.134773+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f51180afc10 msgr2=0x7f51180b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f51180afc10 0x7f51180b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5120000c00 tx=0x7f5120005fb0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 msgr2=0x7f513019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019c610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5128007d70 tx=0x7f512800a340 comp rx=0 tx=0).stop 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 shutdown_connections 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f51180afc10 0x7f51180b2120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019c610 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5130103850 0x7f513019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 --2- 172.21.15.78:0/987272376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51301041a0 0x7f513019c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:30.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 >> 172.21.15.78:0/987272376 conn(0x7f51300fe370 msgr2=0x7f51300fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:30.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 shutdown_connections 2024-04-25T13:57:30.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:30.424+0000 7f5135911700 1 -- 172.21.15.78:0/987272376 wait complete. 2024-04-25T13:57:31.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T13:57:31.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T13:57:31.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T13:57:31.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T13:57:31.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T13:57:31.136 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T13:57:31.136 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T13:57:31.137 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T13:57:31.137 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T13:57:31.138 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T13:57:31.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T13:57:31.142 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T13:57:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:31 smithi078 ceph-mon[29163]: pgmap v286: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T13:57:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:31 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/987272376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:31 smithi112 ceph-mon[32617]: pgmap v286: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T13:57:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:31 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/987272376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:31 smithi129 ceph-mon[32569]: pgmap v286: 97 pgs: 97 active+clean; 623 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T13:57:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:31 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/987272376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:57:31.830 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:31.837 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:32.177 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:33.074 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:33.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:33 smithi078 ceph-mon[29163]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-25T13:57:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:33 smithi112 ceph-mon[32617]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-25T13:57:33.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:33 smithi129 ceph-mon[32569]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-25T13:57:35.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:35 smithi078 ceph-mon[29163]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T13:57:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:35 smithi112 ceph-mon[32617]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T13:57:35.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:35 smithi129 ceph-mon[32569]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T13:57:36.280 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:37.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:37 smithi078 ceph-mon[29163]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T13:57:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:37 smithi112 ceph-mon[32617]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T13:57:37.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:37 smithi129 ceph-mon[32569]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-25T13:57:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:38 smithi078 ceph-mon[29163]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:57:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:38 smithi112 ceph-mon[32617]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:57:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:38 smithi129 ceph-mon[32569]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T13:57:40.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 -- 172.21.15.78:0/391206393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8108610 msgr2=0x7f87e810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 --2- 172.21.15.78:0/391206393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8108610 0x7f87e810ff70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f87e000b0d0 tx=0x7f87e000b3e0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 -- 172.21.15.78:0/391206393 shutdown_connections 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 --2- 172.21.15.78:0/391206393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8108610 0x7f87e810ff70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 --2- 172.21.15.78:0/391206393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8103390 0x7f87e81037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 --2- 172.21.15.78:0/391206393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e8107cc0 0x7f87e81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 -- 172.21.15.78:0/391206393 >> 172.21.15.78:0/391206393 conn(0x7f87e80fec90 msgr2=0x7f87e81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 -- 172.21.15.78:0/391206393 shutdown_connections 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.623+0000 7f87ed93f700 1 -- 172.21.15.78:0/391206393 wait complete. 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 Processor -- start 2024-04-25T13:57:40.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 -- start start 2024-04-25T13:57:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8103390 0x7f87e81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e8107cc0 0x7f87e81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8108610 0x7f87e819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8113240 con 0x7f87e8103390 2024-04-25T13:57:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81130c0 con 0x7f87e8108610 2024-04-25T13:57:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87ed93f700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81133c0 con 0x7f87e8107cc0 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87e6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8103390 0x7f87e81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87e6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8103390 0x7f87e81a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41986/0 (socket says 172.21.15.78:41986) 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87e6ffd700 1 -- 172.21.15.78:0/1727978312 learned_addr learned my addr 172.21.15.78:0/1727978312 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87e67fc700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e8107cc0 0x7f87e81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.624+0000 7f87e77fe700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8108610 0x7f87e819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87e6ffd700 1 -- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e8107cc0 msgr2=0x7f87e81a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87e6ffd700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e8107cc0 0x7f87e81a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87e6ffd700 1 -- 172.21.15.78:0/1727978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8108610 msgr2=0x7f87e819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87e6ffd700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87e8108610 0x7f87e819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87e6ffd700 1 -- 172.21.15.78:0/1727978312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e819b120 con 0x7f87e8103390 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87e6ffd700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8103390 0x7f87e81a0850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87d800d950 tx=0x7f87d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8016070 con 0x7f87e8103390 2024-04-25T13:57:40.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e819b410 con 0x7f87e8103390 2024-04-25T13:57:40.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d800fd40 con 0x7f87e8103390 2024-04-25T13:57:40.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d800e3b0 con 0x7f87e8103390 2024-04-25T13:57:40.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.625+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81a76e0 con 0x7f87e8103390 2024-04-25T13:57:40.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.626+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87e8063e20 con 0x7f87e8103390 2024-04-25T13:57:40.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.627+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f87d8019070 con 0x7f87e8103390 2024-04-25T13:57:40.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.628+0000 7f87ec93d700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87d00afc10 0x7f87d00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.628+0000 7f87e67fc700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87d00afc10 0x7f87d00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.628+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f87d80deca0 con 0x7f87e8103390 2024-04-25T13:57:40.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.628+0000 7f87ec93d700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] conn(0x7f87d00b9440 0x7f87d00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:40.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.628+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 --> [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87d00bbf60 con 0x7f87d00b9440 2024-04-25T13:57:40.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.628+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f87d80df180 con 0x7f87e8103390 2024-04-25T13:57:40.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.628+0000 7f87e67fc700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87d00afc10 0x7f87d00b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f87e81097d0 tx=0x7f87dc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.629+0000 7f87e77fe700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] conn(0x7f87d00b9440 0x7f87d00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:40.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.629+0000 7f87e77fe700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] conn(0x7f87d00b9440 0x7f87d00bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:40.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.630+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== osd.2 v2:172.21.15.129:6800/1698371012 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87d00bbf60 con 0x7f87d00b9440 2024-04-25T13:57:40.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.647+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 --> [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f87e80642a0 con 0x7f87d00b9440 2024-04-25T13:57:40.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.647+0000 7f87ec93d700 1 -- 172.21.15.78:0/1727978312 <== osd.2 v2:172.21.15.129:6800/1698371012 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f87e80642a0 con 0x7f87d00b9440 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] conn(0x7f87d00b9440 msgr2=0x7f87d00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] conn(0x7f87d00b9440 0x7f87d00bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87d00afc10 msgr2=0x7f87d00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87d00afc10 0x7f87d00b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f87e81097d0 tx=0x7f87dc00a300 comp rx=0 tx=0).stop 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8103390 msgr2=0x7f87e81a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 --2- 172.21.15.78:0/1727978312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87e8103390 0x7f87e81a0850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87d800d950 tx=0x7f87d800dc60 comp rx=0 tx=0).stop 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87e77fe700 1 -- 172.21.15.78:0/1727978312 reap_dead start 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 shutdown_connections 2024-04-25T13:57:40.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 >> 172.21.15.78:0/1727978312 conn(0x7f87e80fec90 msgr2=0x7f87e806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:40.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 shutdown_connections 2024-04-25T13:57:40.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:40.648+0000 7f87ed93f700 1 -- 172.21.15.78:0/1727978312 wait complete. 2024-04-25T13:57:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:41 smithi078 ceph-mon[29163]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-25T13:57:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:41 smithi112 ceph-mon[32617]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-25T13:57:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:41 smithi129 ceph-mon[32569]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-25T13:57:43.237 INFO:teuthology.orchestra.run.smithi078.stdout:77309411393 2024-04-25T13:57:43.238 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T13:57:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:43 smithi078 ceph-mon[29163]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:57:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:43 smithi112 ceph-mon[32617]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:57:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:43 smithi129 ceph-mon[32569]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T13:57:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/2830082339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 msgr2=0x7fcf1410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 --2- 172.21.15.78:0/2830082339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 0x7fcf1410ffb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b0d0 tx=0x7fcf0c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/2830082339 shutdown_connections 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 --2- 172.21.15.78:0/2830082339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 0x7fcf1410ffb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 --2- 172.21.15.78:0/2830082339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf14103420 0x7fcf14103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 --2- 172.21.15.78:0/2830082339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf14107d00 0x7fcf14108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/2830082339 >> 172.21.15.78:0/2830082339 conn(0x7fcf14078cd0 msgr2=0x7fcf14079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/2830082339 shutdown_connections 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.330+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/2830082339 wait complete. 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 Processor -- start 2024-04-25T13:57:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 -- start start 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf14103420 0x7fcf14199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf14107d00 0x7fcf14199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 0x7fcf1419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14113280 con 0x7fcf14108650 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14113100 con 0x7fcf14103420 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1bc1c700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14113400 con 0x7fcf14107d00 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1a1b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 0x7fcf1419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1a1b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 0x7fcf1419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42028/0 (socket says 172.21.15.78:42028) 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1a1b9700 1 -- 172.21.15.78:0/567984564 learned_addr learned my addr 172.21.15.78:0/567984564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf191b7700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf14107d00 0x7fcf14199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf199b8700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf14103420 0x7fcf14199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1a1b9700 1 -- 172.21.15.78:0/567984564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf14107d00 msgr2=0x7fcf14199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.331+0000 7fcf1a1b9700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf14107d00 0x7fcf14199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf1a1b9700 1 -- 172.21.15.78:0/567984564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf14103420 msgr2=0x7fcf14199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf1a1b9700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf14103420 0x7fcf14199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf1a1b9700 1 -- 172.21.15.78:0/567984564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf141a8fb0 con 0x7fcf14108650 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf199b8700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf14103420 0x7fcf14199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf1a1b9700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 0x7fcf1419a3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00d010 tx=0x7fcf0c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c01f070 con 0x7fcf14108650 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c003c00 con 0x7fcf14108650 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf141a9240 con 0x7fcf14108650 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c01b6a0 con 0x7fcf14108650 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.332+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a96f0 con 0x7fcf14108650 2024-04-25T13:57:44.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.333+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf14102760 con 0x7fcf14108650 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.335+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcf0c022070 con 0x7fcf14108650 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.335+0000 7fcf0affd700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf000afc90 0x7fcf000b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.335+0000 7fcf199b8700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf000afc90 0x7fcf000b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.335+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fcf0c00f020 con 0x7fcf14108650 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.336+0000 7fcf0affd700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] conn(0x7fcf000b94c0 0x7fcf000bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.336+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 --> [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf0c01cec0 con 0x7fcf000b94c0 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.336+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c0e0700 con 0x7fcf14108650 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.336+0000 7fcf191b7700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] conn(0x7fcf000b94c0 0x7fcf000bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.336+0000 7fcf199b8700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf000afc90 0x7fcf000b21a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf10005fd0 tx=0x7fcf10005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:44.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.336+0000 7fcf191b7700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] conn(0x7fcf000b94c0 0x7fcf000bb910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:44.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.337+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== osd.6 v2:172.21.15.78:6818/3216701822 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf0c01cec0 con 0x7fcf000b94c0 2024-04-25T13:57:44.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.353+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 --> [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf14104880 con 0x7fcf000b94c0 2024-04-25T13:57:44.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.353+0000 7fcf0affd700 1 -- 172.21.15.78:0/567984564 <== osd.6 v2:172.21.15.78:6818/3216701822 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf14104880 con 0x7fcf000b94c0 2024-04-25T13:57:44.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] conn(0x7fcf000b94c0 msgr2=0x7fcf000bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] conn(0x7fcf000b94c0 0x7fcf000bb910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf000afc90 msgr2=0x7fcf000b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcf000afc90 0x7fcf000b21a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf10005fd0 tx=0x7fcf10005e30 comp rx=0 tx=0).stop 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 msgr2=0x7fcf1419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 --2- 172.21.15.78:0/567984564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf14108650 0x7fcf1419a3b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00d010 tx=0x7fcf0c007a20 comp rx=0 tx=0).stop 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1a1b9700 1 -- 172.21.15.78:0/567984564 reap_dead start 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 shutdown_connections 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 >> 172.21.15.78:0/567984564 conn(0x7fcf14078cd0 msgr2=0x7fcf14101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 shutdown_connections 2024-04-25T13:57:44.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.354+0000 7fcf1bc1c700 1 -- 172.21.15.78:0/567984564 wait complete. 2024-04-25T13:57:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 -- 172.21.15.78:0/632284682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0108db0 msgr2=0x7fbce01091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 --2- 172.21.15.78:0/632284682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0108db0 0x7fbce01091c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40099c0 tx=0x7fbcd4009cd0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 -- 172.21.15.78:0/632284682 shutdown_connections 2024-04-25T13:57:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 --2- 172.21.15.78:0/632284682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce0109700 0x7fbce0111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 --2- 172.21.15.78:0/632284682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0104480 0x7fbce0104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 --2- 172.21.15.78:0/632284682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0108db0 0x7fbce01091c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 -- 172.21.15.78:0/632284682 >> 172.21.15.78:0/632284682 conn(0x7fbce00ffd70 msgr2=0x7fbce01021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 -- 172.21.15.78:0/632284682 shutdown_connections 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 -- 172.21.15.78:0/632284682 wait complete. 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.579+0000 7fbce51a0700 1 Processor -- start 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbce51a0700 1 -- start start 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbce51a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0104480 0x7fbce01a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbce51a0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce0108db0 0x7fbce01a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbce51a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0109700 0x7fbce019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbce51a0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0114310 con 0x7fbce0104480 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbce51a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0114190 con 0x7fbce0109700 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbce51a0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0114490 con 0x7fbce0108db0 2024-04-25T13:57:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0104480 0x7fbce01a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0104480 0x7fbce01a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42036/0 (socket says 172.21.15.78:42036) 2024-04-25T13:57:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 -- 172.21.15.78:0/2381342851 learned_addr learned my addr 172.21.15.78:0/2381342851 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcde59c700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce0108db0 0x7fbce01a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcdf59e700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0109700 0x7fbce019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 -- 172.21.15.78:0/2381342851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce0108db0 msgr2=0x7fbce01a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce0108db0 0x7fbce01a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 -- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0109700 msgr2=0x7fbce019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0109700 0x7fbce019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcded9d700 1 -- 172.21.15.78:0/2381342851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce019b280 con 0x7fbce0104480 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.580+0000 7fbcdf59e700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce0109700 0x7fbce019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.581+0000 7fbcded9d700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0104480 0x7fbce01a09c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000e9a0 tx=0x7fbcd000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.581+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000cbd0 con 0x7fbce0104480 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.581+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd000cd70 con 0x7fbce0104480 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.581+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0018680 con 0x7fbce0104480 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.581+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce019b570 con 0x7fbce0104480 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.581+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01a7770 con 0x7fbce0104480 2024-04-25T13:57:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.582+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbce0063de0 con 0x7fbce0104480 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.583+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fbcd0010970 con 0x7fbce0104480 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcc7fff700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcc80afc10 0x7fbcc80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcde59c700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcc80afc10 0x7fbcc80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fbcd00dfaf0 con 0x7fbce0104480 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcc7fff700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] conn(0x7fbcc80b9440 0x7fbcc80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 --> [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbcd001de50 con 0x7fbcc80b9440 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fbcd00dffd0 con 0x7fbce0104480 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcde59c700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcc80afc10 0x7fbcc80b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009990 tx=0x7fbcd401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.584+0000 7fbcdf59e700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] conn(0x7fbcc80b9440 0x7fbcc80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.585+0000 7fbcdf59e700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] conn(0x7fbcc80b9440 0x7fbcc80bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.586+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== osd.1 v2:172.21.15.112:6800/1864123277 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcd001de50 con 0x7fbcc80b9440 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.602+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 --> [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbce0064290 con 0x7fbcc80b9440 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbcc7fff700 1 -- 172.21.15.78:0/2381342851 <== osd.1 v2:172.21.15.112:6800/1864123277 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbce0064290 con 0x7fbcc80b9440 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] conn(0x7fbcc80b9440 msgr2=0x7fbcc80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbce51a0700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] conn(0x7fbcc80b9440 0x7fbcc80bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcc80afc10 msgr2=0x7fbcc80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbce51a0700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcc80afc10 0x7fbcc80b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009990 tx=0x7fbcd401a040 comp rx=0 tx=0).stop 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0104480 msgr2=0x7fbce01a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbce51a0700 1 --2- 172.21.15.78:0/2381342851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce0104480 0x7fbce01a09c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000e9a0 tx=0x7fbcd000ed60 comp rx=0 tx=0).stop 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.603+0000 7fbcdf59e700 1 -- 172.21.15.78:0/2381342851 reap_dead start 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.604+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 shutdown_connections 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.604+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 >> 172.21.15.78:0/2381342851 conn(0x7fbce00ffd70 msgr2=0x7fbce01085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.604+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 shutdown_connections 2024-04-25T13:57:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:44.604+0000 7fbce51a0700 1 -- 172.21.15.78:0/2381342851 wait complete. 2024-04-25T13:57:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:45 smithi078 ceph-mon[29163]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:57:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:45 smithi112 ceph-mon[32617]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:57:45.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:45 smithi129 ceph-mon[32569]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/819801219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4103390 msgr2=0x7f6bc41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 --2- 172.21.15.78:0/819801219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4103390 0x7f6bc41037a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00099c0 tx=0x7f6bc0009cd0 comp rx=0 tx=0).stop 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/819801219 shutdown_connections 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 --2- 172.21.15.78:0/819801219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4108610 0x7f6bc410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 --2- 172.21.15.78:0/819801219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4103390 0x7f6bc41037a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 --2- 172.21.15.78:0/819801219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc4107cc0 0x7f6bc41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/819801219 >> 172.21.15.78:0/819801219 conn(0x7f6bc40fec90 msgr2=0x7f6bc41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/819801219 shutdown_connections 2024-04-25T13:57:46.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/819801219 wait complete. 2024-04-25T13:57:46.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 Processor -- start 2024-04-25T13:57:46.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.137+0000 7f6bcc7fb700 1 -- start start 2024-04-25T13:57:46.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcc7fb700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc4103390 0x7f6bc419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:46.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcc7fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4107cc0 0x7f6bc419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:46.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcc7fb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4108610 0x7f6bc419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcc7fb700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4113240 con 0x7f6bc4108610 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcc7fb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41130c0 con 0x7f6bc4107cc0 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcc7fb700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41133c0 con 0x7f6bc4103390 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4108610 0x7f6bc419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4108610 0x7f6bc419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42048/0 (socket says 172.21.15.78:42048) 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 -- 172.21.15.78:0/945643117 learned_addr learned my addr 172.21.15.78:0/945643117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bc9d96700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4107cc0 0x7f6bc419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bca597700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc4103390 0x7f6bc419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 -- 172.21.15.78:0/945643117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc4103390 msgr2=0x7f6bc419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc4103390 0x7f6bc419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 -- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4107cc0 msgr2=0x7f6bc419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bc4107cc0 0x7f6bc419bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 -- 172.21.15.78:0/945643117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc41a6d60 con 0x7f6bc4108610 2024-04-25T13:57:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bcad98700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4108610 0x7f6bc419c540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc007d70 tx=0x7f6bbc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.138+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc016020 con 0x7f6bc4108610 2024-04-25T13:57:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.139+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bbc00ad40 con 0x7f6bc4108610 2024-04-25T13:57:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.139+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc41a6ff0 con 0x7f6bc4108610 2024-04-25T13:57:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.139+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0115e0 con 0x7f6bc4108610 2024-04-25T13:57:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.139+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a7480 con 0x7f6bc4108610 2024-04-25T13:57:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.139+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bc4063da0 con 0x7f6bc4108610 2024-04-25T13:57:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.141+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6bbc019020 con 0x7f6bc4108610 2024-04-25T13:57:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.141+0000 7f6bbb7fe700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6bb00afc10 0x7f6bb00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.142+0000 7f6bca597700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6bb00afc10 0x7f6bb00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.142+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f6bbc0df0d0 con 0x7f6bc4108610 2024-04-25T13:57:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.142+0000 7f6bbb7fe700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] conn(0x7f6bb00b9440 0x7f6bb00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.142+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 --> [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bbc021e50 con 0x7f6bb00b9440 2024-04-25T13:57:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.142+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6bbc0df5b0 con 0x7f6bc4108610 2024-04-25T13:57:46.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.142+0000 7f6bca597700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6bb00afc10 0x7f6bb00b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009750 tx=0x7f6bc001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:46.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.142+0000 7f6bc9d96700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] conn(0x7f6bb00b9440 0x7f6bb00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:46.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.143+0000 7f6bc9d96700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] conn(0x7f6bb00b9440 0x7f6bb00bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:46.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.143+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== osd.4 v2:172.21.15.112:6808/3169717920 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bbc021e50 con 0x7f6bb00b9440 2024-04-25T13:57:46.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.160+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 --> [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bc4064220 con 0x7f6bb00b9440 2024-04-25T13:57:46.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.160+0000 7f6bbb7fe700 1 -- 172.21.15.78:0/945643117 <== osd.4 v2:172.21.15.112:6808/3169717920 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bc4064220 con 0x7f6bb00b9440 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] conn(0x7f6bb00b9440 msgr2=0x7f6bb00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] conn(0x7f6bb00b9440 0x7f6bb00bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6bb00afc10 msgr2=0x7f6bb00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6bb00afc10 0x7f6bb00b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009750 tx=0x7f6bc001a040 comp rx=0 tx=0).stop 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4108610 msgr2=0x7f6bc419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 --2- 172.21.15.78:0/945643117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4108610 0x7f6bc419c540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc007d70 tx=0x7f6bbc00a340 comp rx=0 tx=0).stop 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcad98700 1 -- 172.21.15.78:0/945643117 reap_dead start 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 shutdown_connections 2024-04-25T13:57:46.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 >> 172.21.15.78:0/945643117 conn(0x7f6bc40fec90 msgr2=0x7f6bc4101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:46.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 shutdown_connections 2024-04-25T13:57:46.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:46.161+0000 7f6bcc7fb700 1 -- 172.21.15.78:0/945643117 wait complete. 2024-04-25T13:57:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.947+0000 7f62b526b700 1 -- 172.21.15.78:0/2738049751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0107c90 msgr2=0x7f62b01080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.947+0000 7f62b526b700 1 --2- 172.21.15.78:0/2738049751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0107c90 0x7f62b01080a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f62980099c0 tx=0x7f6298009cd0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.947+0000 7f62b526b700 1 -- 172.21.15.78:0/2738049751 shutdown_connections 2024-04-25T13:57:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.947+0000 7f62b526b700 1 --2- 172.21.15.78:0/2738049751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b01085e0 0x7f62b010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.947+0000 7f62b526b700 1 --2- 172.21.15.78:0/2738049751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b0103360 0x7f62b0103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.947+0000 7f62b526b700 1 --2- 172.21.15.78:0/2738049751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0107c90 0x7f62b01080a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.947+0000 7f62b526b700 1 -- 172.21.15.78:0/2738049751 >> 172.21.15.78:0/2738049751 conn(0x7f62b00fec90 msgr2=0x7f62b01010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.948+0000 7f62b526b700 1 -- 172.21.15.78:0/2738049751 shutdown_connections 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.948+0000 7f62b526b700 1 -- 172.21.15.78:0/2738049751 wait complete. 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.948+0000 7f62b526b700 1 Processor -- start 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.948+0000 7f62b526b700 1 -- start start 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62b526b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0103360 0x7f62b01a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62b526b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b0107c90 0x7f62b01a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62b526b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b01085e0 0x7f62b019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62b526b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01131f0 con 0x7f62b0103360 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62b526b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0113070 con 0x7f62b01085e0 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62b526b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0113370 con 0x7f62b0107c90 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0103360 0x7f62b01a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0103360 0x7f62b01a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42000/0 (socket says 172.21.15.78:42000) 2024-04-25T13:57:47.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 -- 172.21.15.78:0/1673265264 learned_addr learned my addr 172.21.15.78:0/1673265264 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62af7fe700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b01085e0 0x7f62b019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62a7fff700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b0107c90 0x7f62b01a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 -- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b0107c90 msgr2=0x7f62b01a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62b0107c90 0x7f62b01a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 -- 172.21.15.78:0/1673265264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b01085e0 msgr2=0x7f62b019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62b01085e0 0x7f62b019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.949+0000 7f62aeffd700 1 -- 172.21.15.78:0/1673265264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b019b150 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.950+0000 7f62aeffd700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0103360 0x7f62b01a08a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c920 tx=0x7f62a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.950+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0004a30 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.950+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b019b440 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.950+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01a7750 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.950+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a00054d0 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.950+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0007bd0 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.951+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6290001060 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.953+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f62a0004bd0 con 0x7f62b0103360 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.953+0000 7f62acff9700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f629c0afc90 0x7f629c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.953+0000 7f62a7fff700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f629c0afc90 0x7f629c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.954+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f62a00dea20 con 0x7f62b0103360 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.954+0000 7f62acff9700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] conn(0x7f629c0b94c0 0x7f629c0bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.954+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 --> [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f629c0bbfe0 con 0x7f629c0b94c0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.954+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f62a00def00 con 0x7f62b0103360 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.954+0000 7f62a7fff700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f629c0afc90 0x7f629c0b21a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6298006010 tx=0x7f6298000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.954+0000 7f62af7fe700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] conn(0x7f629c0b94c0 0x7f629c0bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.955+0000 7f62af7fe700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] conn(0x7f629c0b94c0 0x7f629c0bb910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.956+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== osd.11 v2:172.21.15.129:6824/2167746323 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f629c0bbfe0 con 0x7f629c0b94c0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.972+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 --> [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6290002e90 con 0x7f629c0b94c0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.972+0000 7f62acff9700 1 -- 172.21.15.78:0/1673265264 <== osd.11 v2:172.21.15.129:6824/2167746323 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6290002e90 con 0x7f629c0b94c0 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] conn(0x7f629c0b94c0 msgr2=0x7f629c0bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] conn(0x7f629c0b94c0 0x7f629c0bb910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f629c0afc90 msgr2=0x7f629c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f629c0afc90 0x7f629c0b21a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6298006010 tx=0x7f6298000bc0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0103360 msgr2=0x7f62b01a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 --2- 172.21.15.78:0/1673265264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62b0103360 0x7f62b01a08a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c920 tx=0x7f62a000cc30 comp rx=0 tx=0).stop 2024-04-25T13:57:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62af7fe700 1 -- 172.21.15.78:0/1673265264 reap_dead start 2024-04-25T13:57:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 shutdown_connections 2024-04-25T13:57:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 >> 172.21.15.78:0/1673265264 conn(0x7f62b00fec90 msgr2=0x7f62b006b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 shutdown_connections 2024-04-25T13:57:47.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:43.973+0000 7f62b526b700 1 -- 172.21.15.78:0/1673265264 wait complete. 2024-04-25T13:57:47.318 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:47 smithi078 ceph-mon[29163]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:57:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:47 smithi112 ceph-mon[32617]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:57:47.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:47 smithi129 ceph-mon[32569]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T13:57:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 -- 172.21.15.78:0/850489391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 msgr2=0x7f8844109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 --2- 172.21.15.78:0/850489391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 0x7f8844109240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8834009990 tx=0x7f8834009ca0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 -- 172.21.15.78:0/850489391 shutdown_connections 2024-04-25T13:57:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 --2- 172.21.15.78:0/850489391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8844109780 0x7f88441110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 --2- 172.21.15.78:0/850489391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8844102a10 0x7f8844102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 --2- 172.21.15.78:0/850489391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 0x7f8844109240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 -- 172.21.15.78:0/850489391 >> 172.21.15.78:0/850489391 conn(0x7f88440fe3b0 msgr2=0x7f8844100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 -- 172.21.15.78:0/850489391 shutdown_connections 2024-04-25T13:57:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 -- 172.21.15.78:0/850489391 wait complete. 2024-04-25T13:57:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 Processor -- start 2024-04-25T13:57:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.760+0000 7f884c36a700 1 -- start start 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884c36a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8844102a10 0x7f8844103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884c36a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 0x7f8844103ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884c36a700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8844109780 0x7f88441043e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884c36a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8844114340 con 0x7f8844108e30 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884c36a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88441141c0 con 0x7f8844102a10 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884c36a700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88441144c0 con 0x7f8844109780 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 0x7f8844103ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 0x7f8844103ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54466/0 (socket says 172.21.15.78:54466) 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 -- 172.21.15.78:0/1190580561 learned_addr learned my addr 172.21.15.78:0/1190580561 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884a907700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8844109780 0x7f88441043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884a106700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8844102a10 0x7f8844103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 -- 172.21.15.78:0/1190580561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8844109780 msgr2=0x7f88441043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8844109780 0x7f88441043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 -- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8844102a10 msgr2=0x7f8844103960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8844102a10 0x7f8844103960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 -- 172.21.15.78:0/1190580561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88441a8f60 con 0x7f8844108e30 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f884a907700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8844109780 0x7f88441043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.761+0000 7f8849905700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 0x7f8844103ea0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f88340059d0 tx=0x7f8834003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.762+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834027070 con 0x7f8844108e30 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.762+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88441a91f0 con 0x7f8844108e30 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.762+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88340040a0 con 0x7f8844108e30 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.762+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834022e40 con 0x7f8844108e30 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.762+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88441a9650 con 0x7f8844108e30 2024-04-25T13:57:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.762+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8844063e20 con 0x7f8844108e30 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.764+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f883402a070 con 0x7f8844108e30 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.764+0000 7f883b7fe700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f88300afc10 0x7f88300b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.765+0000 7f884a106700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f88300afc10 0x7f88300b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.765+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f88340e0420 con 0x7f8844108e30 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.765+0000 7f883b7fe700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] conn(0x7f88300b9440 0x7f88300bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.765+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 --> [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8834023e50 con 0x7f88300b9440 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.765+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f88340e0900 con 0x7f8844108e30 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.765+0000 7f884a106700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f88300afc10 0x7f88300b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8840005ea0 tx=0x7f8840005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.765+0000 7f884a907700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] conn(0x7f88300b9440 0x7f88300bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:47.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.766+0000 7f884a907700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] conn(0x7f88300b9440 0x7f88300bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:47.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.767+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== osd.10 v2:172.21.15.112:6824/1454658808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8834023e50 con 0x7f88300b9440 2024-04-25T13:57:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.783+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 --> [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88440642a0 con 0x7f88300b9440 2024-04-25T13:57:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.783+0000 7f883b7fe700 1 -- 172.21.15.78:0/1190580561 <== osd.10 v2:172.21.15.112:6824/1454658808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f88440642a0 con 0x7f88300b9440 2024-04-25T13:57:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] conn(0x7f88300b9440 msgr2=0x7f88300bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] conn(0x7f88300b9440 0x7f88300bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f88300afc10 msgr2=0x7f88300b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f88300afc10 0x7f88300b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8840005ea0 tx=0x7f8840005e30 comp rx=0 tx=0).stop 2024-04-25T13:57:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 msgr2=0x7f8844103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 --2- 172.21.15.78:0/1190580561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8844108e30 0x7f8844103ea0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f88340059d0 tx=0x7f8834003680 comp rx=0 tx=0).stop 2024-04-25T13:57:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884a907700 1 -- 172.21.15.78:0/1190580561 reap_dead start 2024-04-25T13:57:47.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 shutdown_connections 2024-04-25T13:57:47.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 >> 172.21.15.78:0/1190580561 conn(0x7f88440fe3b0 msgr2=0x7f884410a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:47.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 shutdown_connections 2024-04-25T13:57:47.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:47.784+0000 7f884c36a700 1 -- 172.21.15.78:0/1190580561 wait complete. 2024-04-25T13:57:47.945 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 -- 172.21.15.78:0/1775024259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c108650 msgr2=0x7ff98c10fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 --2- 172.21.15.78:0/1775024259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c108650 0x7ff98c10fff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0099c0 tx=0x7ff97c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 -- 172.21.15.78:0/1775024259 shutdown_connections 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 --2- 172.21.15.78:0/1775024259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c108650 0x7ff98c10fff0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 --2- 172.21.15.78:0/1775024259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff98c1039e0 0x7ff98c103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 --2- 172.21.15.78:0/1775024259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff98c107d00 0x7ff98c108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 -- 172.21.15.78:0/1775024259 >> 172.21.15.78:0/1775024259 conn(0x7ff98c0fe370 msgr2=0x7ff98c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 -- 172.21.15.78:0/1775024259 shutdown_connections 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.797+0000 7ff991df8700 1 -- 172.21.15.78:0/1775024259 wait complete. 2024-04-25T13:57:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 Processor -- start 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 -- start start 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c1039e0 0x7ff98c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff98c107d00 0x7ff98c19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff98c108650 0x7ff98c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c113310 con 0x7ff98c1039e0 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c113190 con 0x7ff98c107d00 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff991df8700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c113490 con 0x7ff98c108650 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff98b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c1039e0 0x7ff98c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff98b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c1039e0 0x7ff98c19bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54494/0 (socket says 172.21.15.78:54494) 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.798+0000 7ff98b7fe700 1 -- 172.21.15.78:0/650313723 learned_addr learned my addr 172.21.15.78:0/650313723 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98affd700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff98c107d00 0x7ff98c19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98bfff700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff98c108650 0x7ff98c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98b7fe700 1 -- 172.21.15.78:0/650313723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff98c108650 msgr2=0x7ff98c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98b7fe700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff98c108650 0x7ff98c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98b7fe700 1 -- 172.21.15.78:0/650313723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff98c107d00 msgr2=0x7ff98c19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98b7fe700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff98c107d00 0x7ff98c19c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98b7fe700 1 -- 172.21.15.78:0/650313723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c1999c0 con 0x7ff98c1039e0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98bfff700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff98c108650 0x7ff98c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff98b7fe700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c1039e0 0x7ff98c19bbd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff9840060b0 tx=0x7ff98400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98401a070 con 0x7ff98c1039e0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98400ddd0 con 0x7ff98c1039e0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98c199cb0 con 0x7ff98c1039e0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984014720 con 0x7ff98c1039e0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.799+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98c19a1f0 con 0x7ff98c1039e0 2024-04-25T13:57:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.800+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff98c063da0 con 0x7ff98c1039e0 2024-04-25T13:57:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.802+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ff98401d070 con 0x7ff98c1039e0 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.802+0000 7ff988ff9700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9780afc10 0x7ff9780b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.802+0000 7ff98affd700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9780afc10 0x7ff9780b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.802+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff9840df000 con 0x7ff98c1039e0 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.803+0000 7ff988ff9700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] conn(0x7ff9780b9440 0x7ff9780bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.803+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 --> [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff98400ae50 con 0x7ff9780b9440 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.803+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff9840df4e0 con 0x7ff98c1039e0 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.803+0000 7ff98bfff700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] conn(0x7ff9780b9440 0x7ff9780bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.803+0000 7ff98affd700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9780afc10 0x7ff9780b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff98c1008b0 tx=0x7ff97400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.803+0000 7ff98bfff700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] conn(0x7ff9780b9440 0x7ff9780bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.804+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== osd.9 v2:172.21.15.78:6826/144296298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff98400ae50 con 0x7ff9780b9440 2024-04-25T13:57:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.820+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 --> [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff98c0641a0 con 0x7ff9780b9440 2024-04-25T13:57:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.820+0000 7ff988ff9700 1 -- 172.21.15.78:0/650313723 <== osd.9 v2:172.21.15.78:6826/144296298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff98c0641a0 con 0x7ff9780b9440 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.820+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] conn(0x7ff9780b9440 msgr2=0x7ff9780bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.820+0000 7ff991df8700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] conn(0x7ff9780b9440 0x7ff9780bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9780afc10 msgr2=0x7ff9780b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9780afc10 0x7ff9780b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff98c1008b0 tx=0x7ff97400a300 comp rx=0 tx=0).stop 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c1039e0 msgr2=0x7ff98c19bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 --2- 172.21.15.78:0/650313723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff98c1039e0 0x7ff98c19bbd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff9840060b0 tx=0x7ff98400d3b0 comp rx=0 tx=0).stop 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff98bfff700 1 -- 172.21.15.78:0/650313723 reap_dead start 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 shutdown_connections 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 >> 172.21.15.78:0/650313723 conn(0x7ff98c0fe370 msgr2=0x7ff98c0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 shutdown_connections 2024-04-25T13:57:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:48.821+0000 7ff991df8700 1 -- 172.21.15.78:0/650313723 wait complete. 2024-04-25T13:57:49.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.006+0000 7f19a755d700 1 -- 172.21.15.78:0/1259799616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 msgr2=0x7f19a0103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.006+0000 7f19a755d700 1 --2- 172.21.15.78:0/1259799616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 0x7f19a0103760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f19900099c0 tx=0x7f1990009cd0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.006+0000 7f19a755d700 1 -- 172.21.15.78:0/1259799616 shutdown_connections 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 --2- 172.21.15.78:0/1259799616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19a01085d0 0x7f19a010ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 --2- 172.21.15.78:0/1259799616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 0x7f19a0103760 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 --2- 172.21.15.78:0/1259799616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19a0107c80 0x7f19a0108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 -- 172.21.15.78:0/1259799616 >> 172.21.15.78:0/1259799616 conn(0x7f19a00fec30 msgr2=0x7f19a0101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 -- 172.21.15.78:0/1259799616 shutdown_connections 2024-04-25T13:57:49.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 -- 172.21.15.78:0/1259799616 wait complete. 2024-04-25T13:57:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 Processor -- start 2024-04-25T13:57:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.007+0000 7f19a755d700 1 -- start start 2024-04-25T13:57:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a755d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 0x7f19a019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a755d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19a0107c80 0x7f19a019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a755d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19a01085d0 0x7f19a019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a755d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0113280 con 0x7f19a0103350 2024-04-25T13:57:49.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a755d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0113100 con 0x7f19a01085d0 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a755d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0113400 con 0x7f19a0107c80 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 0x7f19a019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 0x7f19a019bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54514/0 (socket says 172.21.15.78:54514) 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 -- 172.21.15.78:0/2513381855 learned_addr learned my addr 172.21.15.78:0/2513381855 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a5afa700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19a01085d0 0x7f19a019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a4af8700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19a0107c80 0x7f19a019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 -- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19a0107c80 msgr2=0x7f19a019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19a0107c80 0x7f19a019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 -- 172.21.15.78:0/2513381855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19a01085d0 msgr2=0x7f19a019c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19a01085d0 0x7f19a019c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 -- 172.21.15.78:0/2513381855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a01a6e90 con 0x7f19a0103350 2024-04-25T13:57:49.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a5afa700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19a01085d0 0x7f19a019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:49.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a52f9700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 0x7f19a019bb40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f199000b580 tx=0x7f19900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:49.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990027070 con 0x7f19a0103350 2024-04-25T13:57:49.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1990004020 con 0x7f19a0103350 2024-04-25T13:57:49.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990022e30 con 0x7f19a0103350 2024-04-25T13:57:49.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.008+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a01a7120 con 0x7f19a0103350 2024-04-25T13:57:49.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.009+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a01a7520 con 0x7f19a0103350 2024-04-25T13:57:49.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.009+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19a0063da0 con 0x7f19a0103350 2024-04-25T13:57:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.011+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1990022590 con 0x7f19a0103350 2024-04-25T13:57:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.011+0000 7f19967fc700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f198c0afc90 0x7f198c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.012+0000 7f19a4af8700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f198c0afc90 0x7f198c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.012+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f19900e0060 con 0x7f19a0103350 2024-04-25T13:57:49.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.012+0000 7f19967fc700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] conn(0x7f198c0b9390 0x7f198c0bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.012+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 --> [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1990007e80 con 0x7f198c0b9390 2024-04-25T13:57:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.012+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f19900e0450 con 0x7f19a0103350 2024-04-25T13:57:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.012+0000 7f19a4af8700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f198c0afc90 0x7f198c0b21a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f199c005950 tx=0x7f199c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.012+0000 7f19a5afa700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] conn(0x7f198c0b9390 0x7f198c0bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.013+0000 7f19a5afa700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] conn(0x7f198c0b9390 0x7f198c0bb7e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:49.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.013+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== osd.5 v2:172.21.15.129:6808/1391792585 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1990007e80 con 0x7f198c0b9390 2024-04-25T13:57:49.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.030+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 --> [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f19a0064220 con 0x7f198c0b9390 2024-04-25T13:57:49.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.030+0000 7f19967fc700 1 -- 172.21.15.78:0/2513381855 <== osd.5 v2:172.21.15.129:6808/1391792585 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f19a0064220 con 0x7f198c0b9390 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.030+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] conn(0x7f198c0b9390 msgr2=0x7f198c0bb7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.030+0000 7f19a755d700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] conn(0x7f198c0b9390 0x7f198c0bb7e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f198c0afc90 msgr2=0x7f198c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f198c0afc90 0x7f198c0b21a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f199c005950 tx=0x7f199c0058e0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 msgr2=0x7f19a019bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 --2- 172.21.15.78:0/2513381855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a0103350 0x7f19a019bb40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f199000b580 tx=0x7f19900047b0 comp rx=0 tx=0).stop 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a5afa700 1 -- 172.21.15.78:0/2513381855 reap_dead start 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 shutdown_connections 2024-04-25T13:57:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 >> 172.21.15.78:0/2513381855 conn(0x7f19a00fec30 msgr2=0x7f19a0109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:49.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 shutdown_connections 2024-04-25T13:57:49.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:49.031+0000 7f19a755d700 1 -- 172.21.15.78:0/2513381855 wait complete. 2024-04-25T13:57:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:49 smithi078 ceph-mon[29163]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:57:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:49 smithi112 ceph-mon[32617]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:57:49.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:49 smithi129 ceph-mon[32569]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:57:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:51 smithi078 ceph-mon[29163]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:57:51.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:51 smithi112 ceph-mon[32617]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:57:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:51 smithi129 ceph-mon[32569]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:57:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1825922807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6181085d0 msgr2=0x7fa61810ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 --2- 172.21.15.78:0/1825922807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6181085d0 0x7fa61810ff30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa61000b0d0 tx=0x7fa61000b3e0 comp rx=0 tx=0).stop 2024-04-25T13:57:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1825922807 shutdown_connections 2024-04-25T13:57:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 --2- 172.21.15.78:0/1825922807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6181085d0 0x7fa61810ff30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 --2- 172.21.15.78:0/1825922807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa618103350 0x7fa618103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 --2- 172.21.15.78:0/1825922807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa618107c80 0x7fa618108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1825922807 >> 172.21.15.78:0/1825922807 conn(0x7fa6180fec50 msgr2=0x7fa618101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1825922807 shutdown_connections 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.602+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1825922807 wait complete. 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 Processor -- start 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 -- start start 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa618103350 0x7fa61819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa618107c80 0x7fa61819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6181085d0 0x7fa61819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa618113220 con 0x7fa618107c80 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6181130a0 con 0x7fa6181085d0 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61cb4d700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6181133a0 con 0x7fa618103350 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa618107c80 0x7fa61819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa618107c80 0x7fa61819c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54536/0 (socket says 172.21.15.78:54536) 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 -- 172.21.15.78:0/1330749470 learned_addr learned my addr 172.21.15.78:0/1330749470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa61659c700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa618103350 0x7fa61819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa616d9d700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6181085d0 0x7fa61819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 -- 172.21.15.78:0/1330749470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa618103350 msgr2=0x7fa61819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa618103350 0x7fa61819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 -- 172.21.15.78:0/1330749470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6181085d0 msgr2=0x7fa61819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6181085d0 0x7fa61819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa60ffff700 1 -- 172.21.15.78:0/1330749470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6181a6e30 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.603+0000 7fa616d9d700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6181085d0 0x7fa61819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.604+0000 7fa60ffff700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa618107c80 0x7fa61819c020 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d950 tx=0x7fa60000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.604+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600017070 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.604+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6181a7120 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.604+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa600005020 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.604+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600009ab0 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.604+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6181a7660 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.604+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa618063da0 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.606+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa600009c50 con 0x7fa618107c80 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa60f7fe700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6040afc10 0x7fa6040b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa61659c700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6040afc10 0x7fa6040b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fa6000de9d0 con 0x7fa618107c80 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa60f7fe700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] conn(0x7fa6040b9440 0x7fa6040bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 --> [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6040bbf60 con 0x7fa6040b9440 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fa6000deeb0 con 0x7fa618107c80 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa616d9d700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] conn(0x7fa6040b9440 0x7fa6040bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa61659c700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6040afc10 0x7fa6040b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa608005950 tx=0x7fa60800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.607+0000 7fa616d9d700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] conn(0x7fa6040b9440 0x7fa6040bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.608+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== osd.3 v2:172.21.15.78:6810/2856659666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6040bbf60 con 0x7fa6040b9440 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.624+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 --> [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa6180641a0 con 0x7fa6040b9440 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.624+0000 7fa60f7fe700 1 -- 172.21.15.78:0/1330749470 <== osd.3 v2:172.21.15.78:6810/2856659666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa6180641a0 con 0x7fa6040b9440 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] conn(0x7fa6040b9440 msgr2=0x7fa6040bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] conn(0x7fa6040b9440 0x7fa6040bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6040afc10 msgr2=0x7fa6040b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6040afc10 0x7fa6040b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa608005950 tx=0x7fa60800b410 comp rx=0 tx=0).stop 2024-04-25T13:57:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa618107c80 msgr2=0x7fa61819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 --2- 172.21.15.78:0/1330749470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa618107c80 0x7fa61819c020 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d950 tx=0x7fa60000dc60 comp rx=0 tx=0).stop 2024-04-25T13:57:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa616d9d700 1 -- 172.21.15.78:0/1330749470 reap_dead start 2024-04-25T13:57:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 shutdown_connections 2024-04-25T13:57:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 >> 172.21.15.78:0/1330749470 conn(0x7fa6180fec50 msgr2=0x7fa6181073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 shutdown_connections 2024-04-25T13:57:52.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:52.625+0000 7fa61cb4d700 1 -- 172.21.15.78:0/1330749470 wait complete. 2024-04-25T13:57:53.077 INFO:teuthology.orchestra.run.smithi078.stdout:274877906970 2024-04-25T13:57:53.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T13:57:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:53 smithi078 ceph-mon[29163]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:57:53.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:53 smithi112 ceph-mon[32617]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:57:53.592 INFO:teuthology.orchestra.run.smithi078.stdout:300647710740 2024-04-25T13:57:53.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T13:57:53.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:53 smithi129 ceph-mon[32569]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:57:54.020 INFO:teuthology.orchestra.run.smithi078.stdout:55834574919 2024-04-25T13:57:54.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T13:57:54.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:54 smithi078 ceph-mon[29163]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:57:54.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:54 smithi112 ceph-mon[32617]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:57:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:54 smithi129 ceph-mon[32569]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:57:54.837 INFO:teuthology.orchestra.run.smithi078.stdout:253403070495 2024-04-25T13:57:54.838 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T13:57:55.646 INFO:teuthology.orchestra.run.smithi078.stdout:107374182462 2024-04-25T13:57:55.647 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T13:57:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4058456169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f8102630 msgr2=0x7fe4f8102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 --2- 172.21.15.78:0/4058456169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f8102630 0x7fe4f8102a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009990 tx=0x7fe4e8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4058456169 shutdown_connections 2024-04-25T13:57:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 --2- 172.21.15.78:0/4058456169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4f8102f80 0x7fe4f8111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 --2- 172.21.15.78:0/4058456169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f81091a0 0x7fe4f81095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 --2- 172.21.15.78:0/4058456169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f8102630 0x7fe4f8102a40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4058456169 >> 172.21.15.78:0/4058456169 conn(0x7fe4f80fe3f0 msgr2=0x7fe4f8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4058456169 shutdown_connections 2024-04-25T13:57:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.849+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4058456169 wait complete. 2024-04-25T13:57:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 Processor -- start 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 -- start start 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4f8102630 0x7fe4f819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f8102f80 0x7fe4f819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f81091a0 0x7fe4f819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8114440 con 0x7fe4f81091a0 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81142c0 con 0x7fe4f8102630 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fffa9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81145c0 con 0x7fe4f8102f80 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fe546700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f81091a0 0x7fe4f819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fe546700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f81091a0 0x7fe4f819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54558/0 (socket says 172.21.15.78:54558) 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fe546700 1 -- 172.21.15.78:0/4037283605 learned_addr learned my addr 172.21.15.78:0/4037283605 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fdd45700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4f8102630 0x7fe4f819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.850+0000 7fe4fd544700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f8102f80 0x7fe4f819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fe546700 1 -- 172.21.15.78:0/4037283605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f8102f80 msgr2=0x7fe4f819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fe546700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f8102f80 0x7fe4f819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fe546700 1 -- 172.21.15.78:0/4037283605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4f8102630 msgr2=0x7fe4f819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fe546700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4f8102630 0x7fe4f819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fe546700 1 -- 172.21.15.78:0/4037283605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f819fca0 con 0x7fe4f81091a0 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fe546700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f81091a0 0x7fe4f819f600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0007ee0 tx=0x7fe4f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0004bb0 con 0x7fe4f81091a0 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f819ff90 con 0x7fe4f81091a0 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f0004d50 con 0x7fe4f81091a0 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0005920 con 0x7fe4f81091a0 2024-04-25T13:57:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.851+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f81a78b0 con 0x7fe4f81091a0 2024-04-25T13:57:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.852+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4f8063da0 con 0x7fe4f81091a0 2024-04-25T13:57:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.854+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe4f001a020 con 0x7fe4f81091a0 2024-04-25T13:57:55.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.855+0000 7fe4eeffd700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4e40afc10 0x7fe4e40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:55.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.855+0000 7fe4fdd45700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4e40afc10 0x7fe4e40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:55.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.855+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fe4f00de970 con 0x7fe4f81091a0 2024-04-25T13:57:55.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.856+0000 7fe4eeffd700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] conn(0x7fe4e40b9440 0x7fe4e40bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.856+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 --> [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4f001ae50 con 0x7fe4e40b9440 2024-04-25T13:57:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.856+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fe4f00dee50 con 0x7fe4f81091a0 2024-04-25T13:57:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.856+0000 7fe4fd544700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] conn(0x7fe4e40b9440 0x7fe4e40bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.856+0000 7fe4fdd45700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4e40afc10 0x7fe4e40b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8076150 tx=0x7fe4f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.856+0000 7fe4fd544700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] conn(0x7fe4e40b9440 0x7fe4e40bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.857+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== osd.0 v2:172.21.15.78:6802/2736058336 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f001ae50 con 0x7fe4e40b9440 2024-04-25T13:57:55.873 INFO:teuthology.orchestra.run.smithi078.stdout:154618822708 2024-04-25T13:57:55.874 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T13:57:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.874+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 --> [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4f80641a0 con 0x7fe4e40b9440 2024-04-25T13:57:55.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4eeffd700 1 -- 172.21.15.78:0/4037283605 <== osd.0 v2:172.21.15.78:6802/2736058336 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe4f80641a0 con 0x7fe4e40b9440 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] conn(0x7fe4e40b9440 msgr2=0x7fe4e40bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] conn(0x7fe4e40b9440 0x7fe4e40bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4e40afc10 msgr2=0x7fe4e40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4e40afc10 0x7fe4e40b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8076150 tx=0x7fe4f400b410 comp rx=0 tx=0).stop 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f81091a0 msgr2=0x7fe4f819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 --2- 172.21.15.78:0/4037283605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f81091a0 0x7fe4f819f600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0007ee0 tx=0x7fe4f000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fe546700 1 -- 172.21.15.78:0/4037283605 reap_dead start 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 shutdown_connections 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 >> 172.21.15.78:0/4037283605 conn(0x7fe4f80fe3f0 msgr2=0x7fe4f810a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 shutdown_connections 2024-04-25T13:57:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:55.875+0000 7fe4fffa9700 1 -- 172.21.15.78:0/4037283605 wait complete. 2024-04-25T13:57:56.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 -- 172.21.15.78:0/3323893223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 msgr2=0x7f05c41095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:56.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 --2- 172.21.15.78:0/3323893223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 0x7f05c41095a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009990 tx=0x7f05b4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:57:56.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 -- 172.21.15.78:0/3323893223 shutdown_connections 2024-04-25T13:57:56.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 --2- 172.21.15.78:0/3323893223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05c4103000 0x7f05c4107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:56.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 --2- 172.21.15.78:0/3323893223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 0x7f05c41095a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:56.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 --2- 172.21.15.78:0/3323893223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05c41026b0 0x7f05c4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:56.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 -- 172.21.15.78:0/3323893223 >> 172.21.15.78:0/3323893223 conn(0x7f05c40fe5c0 msgr2=0x7f05c41009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:56.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 -- 172.21.15.78:0/3323893223 shutdown_connections 2024-04-25T13:57:56.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.540+0000 7f05c8acd700 1 -- 172.21.15.78:0/3323893223 wait complete. 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 Processor -- start 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 -- start start 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05c41026b0 0x7f05c41a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05c4103000 0x7f05c41a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 0x7f05c41a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05c411a9f0 con 0x7f05c4109190 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05c411a870 con 0x7f05c41026b0 2024-04-25T13:57:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c8acd700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05c411ab70 con 0x7f05c4103000 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.541+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 0x7f05c41a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 0x7f05c41a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38876/0 (socket says 172.21.15.78:38876) 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 -- 172.21.15.78:0/275582117 learned_addr learned my addr 172.21.15.78:0/275582117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c259c700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05c41026b0 0x7f05c41a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c1d9b700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05c4103000 0x7f05c41a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 -- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05c4103000 msgr2=0x7f05c41a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05c4103000 0x7f05c41a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 -- 172.21.15.78:0/275582117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05c41026b0 msgr2=0x7f05c41a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05c41026b0 0x7f05c41a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 -- 172.21.15.78:0/275582117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c41a6240 con 0x7f05c4109190 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c259c700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05c41026b0 0x7f05c41a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c2d9d700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 0x7f05c41a5ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05b800c970 tx=0x7f05b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8004230 con 0x7f05c4109190 2024-04-25T13:57:56.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b80043d0 con 0x7f05c4109190 2024-04-25T13:57:56.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c41a6530 con 0x7f05c4109190 2024-04-25T13:57:56.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b800eb60 con 0x7f05c4109190 2024-04-25T13:57:56.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.542+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c41ade50 con 0x7f05c4109190 2024-04-25T13:57:56.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.543+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05c4107810 con 0x7f05c4109190 2024-04-25T13:57:56.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.545+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f05b80036a0 con 0x7f05c4109190 2024-04-25T13:57:56.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.545+0000 7f05b37fe700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05ac0afc10 0x7f05ac0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:56.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.545+0000 7f05c259c700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05ac0afc10 0x7f05ac0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:56.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.546+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f05b80dec90 con 0x7f05c4109190 2024-04-25T13:57:56.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.546+0000 7f05b37fe700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] conn(0x7f05ac0b9440 0x7f05ac0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:56.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.546+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 --> [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05b8016ec0 con 0x7f05ac0b9440 2024-04-25T13:57:56.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.546+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f05b80df170 con 0x7f05c4109190 2024-04-25T13:57:56.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.546+0000 7f05c259c700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05ac0afc10 0x7f05ac0b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009750 tx=0x7f05b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:56.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.546+0000 7f05c1d9b700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] conn(0x7f05ac0b9440 0x7f05ac0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:56.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.547+0000 7f05c1d9b700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] conn(0x7f05ac0b9440 0x7f05ac0bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:56.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.547+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== osd.8 v2:172.21.15.129:6816/4162935801 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05b8016ec0 con 0x7f05ac0b9440 2024-04-25T13:57:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.563+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 --> [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f05c41038d0 con 0x7f05ac0b9440 2024-04-25T13:57:56.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05b37fe700 1 -- 172.21.15.78:0/275582117 <== osd.8 v2:172.21.15.129:6816/4162935801 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f05c41038d0 con 0x7f05ac0b9440 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] conn(0x7f05ac0b9440 msgr2=0x7f05ac0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05c8acd700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] conn(0x7f05ac0b9440 0x7f05ac0bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05ac0afc10 msgr2=0x7f05ac0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05c8acd700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05ac0afc10 0x7f05ac0b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009750 tx=0x7f05b401a040 comp rx=0 tx=0).stop 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 msgr2=0x7f05c41a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05c8acd700 1 --2- 172.21.15.78:0/275582117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c4109190 0x7f05c41a5ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05b800c970 tx=0x7f05b800cc80 comp rx=0 tx=0).stop 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.564+0000 7f05c2d9d700 1 -- 172.21.15.78:0/275582117 reap_dead start 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.565+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 shutdown_connections 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.565+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 >> 172.21.15.78:0/275582117 conn(0x7f05c40fe5c0 msgr2=0x7f05c4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.565+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 shutdown_connections 2024-04-25T13:57:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:56.565+0000 7f05c8acd700 1 -- 172.21.15.78:0/275582117 wait complete. 2024-04-25T13:57:57.120 INFO:teuthology.orchestra.run.smithi078.stdout:34359738446 2024-04-25T13:57:57.120 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T13:57:57.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:57 smithi078 ceph-mon[29163]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:57:57.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:57 smithi112 ceph-mon[32617]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:57:57.541 INFO:teuthology.orchestra.run.smithi078.stdout:180388626478 2024-04-25T13:57:57.541 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T13:57:57.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:57 smithi129 ceph-mon[32569]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T13:57:58.478 INFO:teuthology.orchestra.run.smithi078.stdout:227633266726 2024-04-25T13:57:58.478 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T13:57:59.217 INFO:teuthology.orchestra.run.smithi078.stdout:133143986232 2024-04-25T13:57:59.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T13:57:59.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:57:59 smithi078 ceph-mon[29163]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:57:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:57:59 smithi112 ceph-mon[32617]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:57:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:57:59 smithi129 ceph-mon[32569]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T13:57:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.898+0000 7fb42d0e1700 1 -- 172.21.15.78:0/1419081972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 msgr2=0x7fb42810e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.898+0000 7fb42d0e1700 1 --2- 172.21.15.78:0/1419081972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 0x7fb42810e250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009990 tx=0x7fb41c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.898+0000 7fb42d0e1700 1 -- 172.21.15.78:0/1419081972 shutdown_connections 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.898+0000 7fb42d0e1700 1 --2- 172.21.15.78:0/1419081972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42810e7e0 0x7fb428110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.898+0000 7fb42d0e1700 1 --2- 172.21.15.78:0/1419081972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4281053b0 0x7fb4281057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.898+0000 7fb42d0e1700 1 --2- 172.21.15.78:0/1419081972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 0x7fb42810e250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.899+0000 7fb42d0e1700 1 -- 172.21.15.78:0/1419081972 >> 172.21.15.78:0/1419081972 conn(0x7fb4280fee00 msgr2=0x7fb428101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.899+0000 7fb42d0e1700 1 -- 172.21.15.78:0/1419081972 shutdown_connections 2024-04-25T13:57:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.899+0000 7fb42d0e1700 1 -- 172.21.15.78:0/1419081972 wait complete. 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.899+0000 7fb42d0e1700 1 Processor -- start 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42d0e1700 1 -- start start 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42d0e1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4281053b0 0x7fb42819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42d0e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 0x7fb42819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42d0e1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42810e7e0 0x7fb42819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42d0e1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281143c0 con 0x7fb428105d00 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42d0e1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb428114240 con 0x7fb42810e7e0 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42d0e1700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb428114540 con 0x7fb4281053b0 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 0x7fb42819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 0x7fb42819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38898/0 (socket says 172.21.15.78:38898) 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 -- 172.21.15.78:0/3022383859 learned_addr learned my addr 172.21.15.78:0/3022383859 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb426d9d700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4281053b0 0x7fb42819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42759e700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42810e7e0 0x7fb42819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 -- 172.21.15.78:0/3022383859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4281053b0 msgr2=0x7fb42819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4281053b0 0x7fb42819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 -- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42810e7e0 msgr2=0x7fb42819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb42810e7e0 0x7fb42819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.900+0000 7fb42659c700 1 -- 172.21.15.78:0/3022383859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42819fc00 con 0x7fb428105d00 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.901+0000 7fb42659c700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 0x7fb42819b2d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.901+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c027070 con 0x7fb428105d00 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.901+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41c003fe0 con 0x7fb428105d00 2024-04-25T13:57:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.901+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42819fe90 con 0x7fb428105d00 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.901+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c022e30 con 0x7fb428105d00 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.901+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4281a0370 con 0x7fb428105d00 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.902+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb428063e20 con 0x7fb428105d00 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.903+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb41c02a070 con 0x7fb428105d00 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.904+0000 7fb40ffff700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4100afc10 0x7fb4100b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.904+0000 7fb426d9d700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4100afc10 0x7fb4100b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.904+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fb41c0dfef0 con 0x7fb428105d00 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.904+0000 7fb40ffff700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] conn(0x7fb4100b9440 0x7fb4100bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.904+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 --> [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4100bbf60 con 0x7fb4100b9440 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.904+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fb41c0e03d0 con 0x7fb428105d00 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.905+0000 7fb42759e700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] conn(0x7fb4100b9440 0x7fb4100bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:57:59.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.905+0000 7fb426d9d700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4100afc10 0x7fb4100b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb418005950 tx=0x7fb41800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.905+0000 7fb42759e700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] conn(0x7fb4100b9440 0x7fb4100bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:57:59.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.906+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== osd.7 v2:172.21.15.112:6816/1198451048 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4100bbf60 con 0x7fb4100b9440 2024-04-25T13:57:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.922+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 --> [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4280642a0 con 0x7fb4100b9440 2024-04-25T13:57:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb40ffff700 1 -- 172.21.15.78:0/3022383859 <== osd.7 v2:172.21.15.112:6816/1198451048 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4280642a0 con 0x7fb4100b9440 2024-04-25T13:57:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] conn(0x7fb4100b9440 msgr2=0x7fb4100bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb42d0e1700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] conn(0x7fb4100b9440 0x7fb4100bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4100afc10 msgr2=0x7fb4100b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb42d0e1700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4100afc10 0x7fb4100b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb418005950 tx=0x7fb41800a300 comp rx=0 tx=0).stop 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 msgr2=0x7fb42819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb42d0e1700 1 --2- 172.21.15.78:0/3022383859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb428105d00 0x7fb42819b2d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c0047b0 comp rx=0 tx=0).stop 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.923+0000 7fb42759e700 1 -- 172.21.15.78:0/3022383859 reap_dead start 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.924+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 shutdown_connections 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.924+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 >> 172.21.15.78:0/3022383859 conn(0x7fb4280fee00 msgr2=0x7fb4280ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.924+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 shutdown_connections 2024-04-25T13:57:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:57:59.924+0000 7fb42d0e1700 1 -- 172.21.15.78:0/3022383859 wait complete. 2024-04-25T13:58:00.113 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:00.833 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:00.833 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:00.834 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:01.374 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:01 smithi078 ceph-mon[29163]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:58:01.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:01 smithi112 ceph-mon[32617]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:58:01.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:01 smithi129 ceph-mon[32569]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T13:58:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:03 smithi078 ceph-mon[29163]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T13:58:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:03 smithi112 ceph-mon[32617]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T13:58:03.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:03 smithi129 ceph-mon[32569]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T13:58:04.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:04 smithi112 ceph-mon[32617]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:58:04.662 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:04 smithi078 ceph-mon[29163]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:58:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:04 smithi129 ceph-mon[32569]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:58:05.708 INFO:teuthology.orchestra.run.smithi078.stdout:201863462956 2024-04-25T13:58:05.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T13:58:07.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:07 smithi078 ceph-mon[29163]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:07.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:07 smithi112 ceph-mon[32617]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:07.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:07 smithi129 ceph-mon[32569]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:08.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:58:08.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-25T13:58:08.282 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-25T13:58:08.282 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T13:58:08.282 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:58:08.282 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T13:58:08.282 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:58:08.282 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:58:08.283 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:08 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 -- 172.21.15.78:0/3323959124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 msgr2=0x7fa710103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 --2- 172.21.15.78:0/3323959124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa710103c20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa700009990 tx=0x7fa700009ca0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 -- 172.21.15.78:0/3323959124 shutdown_connections 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 --2- 172.21.15.78:0/3323959124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa710104160 0x7fa71010ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 --2- 172.21.15.78:0/3323959124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7101080b0 0x7fa7101084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 --2- 172.21.15.78:0/3323959124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa710103c20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 -- 172.21.15.78:0/3323959124 >> 172.21.15.78:0/3323959124 conn(0x7fa7100fe370 msgr2=0x7fa7101007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 -- 172.21.15.78:0/3323959124 shutdown_connections 2024-04-25T13:58:08.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.299+0000 7fa716822700 1 -- 172.21.15.78:0/3323959124 wait complete. 2024-04-25T13:58:08.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 Processor -- start 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 -- start start 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa71019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa710104160 0x7fa71019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7101080b0 0x7fa71019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa710113230 con 0x7fa710103810 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101130b0 con 0x7fa710104160 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa716822700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7101133b0 con 0x7fa7101080b0 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa70ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa71019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa70ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa71019ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45944/0 (socket says 172.21.15.78:45944) 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa70ffff700 1 -- 172.21.15.78:0/1114674092 learned_addr learned my addr 172.21.15.78:0/1114674092 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:08.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa714dbf700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7101080b0 0x7fa71019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.300+0000 7fa70f7fe700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa710104160 0x7fa71019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70ffff700 1 -- 172.21.15.78:0/1114674092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7101080b0 msgr2=0x7fa71019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70ffff700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7101080b0 0x7fa71019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70ffff700 1 -- 172.21.15.78:0/1114674092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa710104160 msgr2=0x7fa71019bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70ffff700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa710104160 0x7fa71019bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70ffff700 1 -- 172.21.15.78:0/1114674092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7101a6dc0 con 0x7fa710103810 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa714dbf700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7101080b0 0x7fa71019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70ffff700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa71019ba50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa70400d950 tx=0x7fa70400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70d7fa700 1 -- 172.21.15.78:0/1114674092 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704017070 con 0x7fa710103810 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7101a70b0 con 0x7fa710103810 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70d7fa700 1 -- 172.21.15.78:0/1114674092 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70400de30 con 0x7fa710103810 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa70d7fa700 1 -- 172.21.15.78:0/1114674092 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704003b10 con 0x7fa710103810 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.301+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7101a75f0 con 0x7fa710103810 2024-04-25T13:58:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.302+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa710051190 con 0x7fa710103810 2024-04-25T13:58:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.304+0000 7fa70d7fa700 1 -- 172.21.15.78:0/1114674092 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa704003cb0 con 0x7fa710103810 2024-04-25T13:58:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.304+0000 7fa70d7fa700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6f80afc10 0x7fa6f80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.305+0000 7fa70f7fe700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6f80afc10 0x7fa6f80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.305+0000 7fa70d7fa700 1 -- 172.21.15.78:0/1114674092 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8248+0+0 (secure 0 0 0) 0x7fa7040de950 con 0x7fa710103810 2024-04-25T13:58:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.305+0000 7fa70f7fe700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6f80afc10 0x7fa6f80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa700003800 tx=0x7fa70001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.309+0000 7fa70d7fa700 1 -- 172.21.15.78:0/1114674092 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa704020030 con 0x7fa710103810 2024-04-25T13:58:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.514+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa71002e6d0 con 0x7fa710103810 2024-04-25T13:58:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.514+0000 7fa70d7fa700 1 -- 172.21.15.78:0/1114674092 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa7040a73c0 con 0x7fa710103810 2024-04-25T13:58:08.514 INFO:teuthology.orchestra.run.smithi078.stdout:55834574924 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:58:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:58:08.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:08 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-25T13:58:08.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.516+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6f80afc10 msgr2=0x7fa6f80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.516+0000 7fa716822700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6f80afc10 0x7fa6f80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa700003800 tx=0x7fa70001a040 comp rx=0 tx=0).stop 2024-04-25T13:58:08.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.516+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 msgr2=0x7fa71019ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.516+0000 7fa716822700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa71019ba50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa70400d950 tx=0x7fa70400dd10 comp rx=0 tx=0).stop 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 shutdown_connections 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa6f80afc10 0x7fa6f80b2120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7101080b0 0x7fa71019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710103810 0x7fa71019ba50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 --2- 172.21.15.78:0/1114674092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa710104160 0x7fa71019bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 >> 172.21.15.78:0/1114674092 conn(0x7fa7100fe370 msgr2=0x7fa7100fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 shutdown_connections 2024-04-25T13:58:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.517+0000 7fa716822700 1 -- 172.21.15.78:0/1114674092 wait complete. 2024-04-25T13:58:08.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 -- 172.21.15.78:0/2463376442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 msgr2=0x7f0de810e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/2463376442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de810e270 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009990 tx=0x7f0ddc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 -- 172.21.15.78:0/2463376442 shutdown_connections 2024-04-25T13:58:08.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/2463376442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0de810e7b0 0x7f0de8110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/2463376442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0de81053d0 0x7f0de81057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/2463376442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de810e270 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 -- 172.21.15.78:0/2463376442 >> 172.21.15.78:0/2463376442 conn(0x7f0de80fec90 msgr2=0x7f0de81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 -- 172.21.15.78:0/2463376442 shutdown_connections 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.538+0000 7f0ded6f0700 1 -- 172.21.15.78:0/2463376442 wait complete. 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 Processor -- start 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 -- start start 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0de81053d0 0x7f0de81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de8103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0de810e7b0 0x7f0de8103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8114310 con 0x7f0de8105d20 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8114190 con 0x7f0de810e7b0 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.539+0000 7f0ded6f0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0de8114490 con 0x7f0de81053d0 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de8103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de8103600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45972/0 (socket says 172.21.15.78:45972) 2024-04-25T13:58:08.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 -- 172.21.15.78:0/1361190966 learned_addr learned my addr 172.21.15.78:0/1361190966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de77fe700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0de810e7b0 0x7f0de8103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de6ffd700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0de81053d0 0x7f0de81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 -- 172.21.15.78:0/1361190966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0de81053d0 msgr2=0x7f0de81030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0de81053d0 0x7f0de81030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 -- 172.21.15.78:0/1361190966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0de810e7b0 msgr2=0x7f0de8103b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0de810e7b0 0x7f0de8103b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 -- 172.21.15.78:0/1361190966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de81a8f90 con 0x7f0de8105d20 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0de67fc700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de8103600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0057c0 tx=0x7f0ddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0dcffff700 1 -- 172.21.15.78:0/1361190966 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc027070 con 0x7f0de8105d20 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0dcffff700 1 -- 172.21.15.78:0/1361190966 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ddc003f20 con 0x7f0de8105d20 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de81a9220 con 0x7f0de8105d20 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0dcffff700 1 -- 172.21.15.78:0/1361190966 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc022c30 con 0x7f0de8105d20 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.540+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de81a9700 con 0x7f0de8105d20 2024-04-25T13:58:08.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.542+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de8051190 con 0x7f0de8105d20 2024-04-25T13:58:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.543+0000 7f0dcffff700 1 -- 172.21.15.78:0/1361190966 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0ddc02a070 con 0x7f0de8105d20 2024-04-25T13:58:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.544+0000 7f0dcffff700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0dd00afc10 0x7f0dd00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.544+0000 7f0de6ffd700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0dd00afc10 0x7f0dd00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.544+0000 7f0dcffff700 1 -- 172.21.15.78:0/1361190966 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8248+0+0 (secure 0 0 0) 0x7f0ddc0e0590 con 0x7f0de8105d20 2024-04-25T13:58:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.544+0000 7f0de6ffd700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0dd00afc10 0x7f0dd00b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005ea0 tx=0x7f0dd8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.548+0000 7f0dcffff700 1 -- 172.21.15.78:0/1361190966 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ddc031080 con 0x7f0de8105d20 2024-04-25T13:58:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T13:58:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-25T13:58:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-25T13:58:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-25T13:58:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:58:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T13:58:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:58:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:58:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:08 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-25T13:58:08.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.751+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0de802e510 con 0x7f0de8105d20 2024-04-25T13:58:08.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.751+0000 7f0dcffff700 1 -- 172.21.15.78:0/1361190966 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ddc013070 con 0x7f0de8105d20 2024-04-25T13:58:08.751 INFO:teuthology.orchestra.run.smithi078.stdout:77309411399 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0dd00afc10 msgr2=0x7f0dd00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0dd00afc10 0x7f0dd00b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005ea0 tx=0x7f0dd8005e30 comp rx=0 tx=0).stop 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 msgr2=0x7f0de8103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de8103600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0057c0 tx=0x7f0ddc0057f0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 shutdown_connections 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0dd00afc10 0x7f0dd00b2120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0de810e7b0 0x7f0de8103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0de81053d0 0x7f0de81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 --2- 172.21.15.78:0/1361190966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0de8105d20 0x7f0de8103600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 >> 172.21.15.78:0/1361190966 conn(0x7f0de80fec90 msgr2=0x7f0de80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 shutdown_connections 2024-04-25T13:58:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:08.754+0000 7f0ded6f0700 1 -- 172.21.15.78:0/1361190966 wait complete. 2024-04-25T13:58:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:58:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-25T13:58:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-25T13:58:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T13:58:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:58:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-04-25T13:58:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:58:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:58:09.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-25T13:58:09.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T13:58:09.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1114674092' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:58:09.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1361190966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:58:09.476 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574924 for osd.1 2024-04-25T13:58:09.476 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:58:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-25T13:58:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-25T13:58:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T13:58:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:58:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-04-25T13:58:09.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:58:09.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:58:09.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-25T13:58:09.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T13:58:09.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1114674092' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:58:09.517 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1361190966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:58:09.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1114674092' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:58:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1361190966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:58:09.908 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:09.909 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:09.909 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:09.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.934+0000 7f1d45ffb700 1 -- 172.21.15.78:0/555651786 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38003c60 con 0x7f1d48108640 2024-04-25T13:58:09.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 -- 172.21.15.78:0/555651786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48108640 msgr2=0x7f1d4810ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:09.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 --2- 172.21.15.78:0/555651786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48108640 0x7f1d4810ffa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1d380099c0 tx=0x7f1d38009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:09.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 -- 172.21.15.78:0/555651786 shutdown_connections 2024-04-25T13:58:09.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 --2- 172.21.15.78:0/555651786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48108640 0x7f1d4810ffa0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:09.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 --2- 172.21.15.78:0/555651786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d48103af0 0x7f1d48103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:09.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 --2- 172.21.15.78:0/555651786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48107cf0 0x7f1d48108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:09.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 -- 172.21.15.78:0/555651786 >> 172.21.15.78:0/555651786 conn(0x7f1d480fe5c0 msgr2=0x7f1d481009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:09.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 -- 172.21.15.78:0/555651786 shutdown_connections 2024-04-25T13:58:09.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.935+0000 7f1d4d997700 1 -- 172.21.15.78:0/555651786 wait complete. 2024-04-25T13:58:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 Processor -- start 2024-04-25T13:58:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 -- start start 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48103af0 0x7f1d4819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48107cf0 0x7f1d4819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d48108640 0x7f1d48199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113330 con 0x7f1d48103af0 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481131b0 con 0x7f1d48107cf0 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d4d997700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1d481134b0 con 0x7f1d48108640 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48103af0 0x7f1d4819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d46ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48103af0 0x7f1d4819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45998/0 (socket says 172.21.15.78:45998) 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d46ffd700 1 -- 172.21.15.78:0/596066319 learned_addr learned my addr 172.21.15.78:0/596066319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.936+0000 7f1d467fc700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48107cf0 0x7f1d4819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d477fe700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d48108640 0x7f1d48199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d46ffd700 1 -- 172.21.15.78:0/596066319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d48108640 msgr2=0x7f1d48199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d46ffd700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d48108640 0x7f1d48199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d46ffd700 1 -- 172.21.15.78:0/596066319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48107cf0 msgr2=0x7f1d4819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d46ffd700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48107cf0 0x7f1d4819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d46ffd700 1 -- 172.21.15.78:0/596066319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4819a0b0 con 0x7f1d48103af0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d46ffd700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48103af0 0x7f1d4819f850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1d40007f10 tx=0x7f1d4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d4c995700 1 -- 172.21.15.78:0/596066319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000dd20 con 0x7f1d48103af0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d4c995700 1 -- 172.21.15.78:0/596066319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d40018070 con 0x7f1d48103af0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d4c995700 1 -- 172.21.15.78:0/596066319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400125b0 con 0x7f1d48103af0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4819a3a0 con 0x7f1d48103af0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.937+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d48102750 con 0x7f1d48103af0 2024-04-25T13:58:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.938+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d48104870 con 0x7f1d48103af0 2024-04-25T13:58:09.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.940+0000 7f1d4c995700 1 -- 172.21.15.78:0/596066319 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1d40004ad0 con 0x7f1d48103af0 2024-04-25T13:58:09.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.940+0000 7f1d4c995700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d300afc90 0x7f1d300b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:09.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.941+0000 7f1d467fc700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d300afc90 0x7f1d300b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:09.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.941+0000 7f1d4c995700 1 -- 172.21.15.78:0/596066319 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1d400deda0 con 0x7f1d48103af0 2024-04-25T13:58:09.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.941+0000 7f1d467fc700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d300afc90 0x7f1d300b21a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1d481065c0 tx=0x7f1d3c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:09.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:09.944+0000 7f1d4c995700 1 -- 172.21.15.78:0/596066319 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d40010070 con 0x7f1d48103af0 2024-04-25T13:58:10.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.151+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f1d4819afa0 con 0x7f1d48103af0 2024-04-25T13:58:10.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.151+0000 7f1d4c995700 1 -- 172.21.15.78:0/596066319 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1d400a7810 con 0x7f1d48103af0 2024-04-25T13:58:10.151 INFO:teuthology.orchestra.run.smithi078.stdout:300647710745 2024-04-25T13:58:10.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.153+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d300afc90 msgr2=0x7f1d300b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.153+0000 7f1d4d997700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d300afc90 0x7f1d300b21a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1d481065c0 tx=0x7f1d3c00a380 comp rx=0 tx=0).stop 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.153+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48103af0 msgr2=0x7f1d4819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.153+0000 7f1d4d997700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48103af0 0x7f1d4819f850 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1d40007f10 tx=0x7f1d4000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 shutdown_connections 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d300afc90 0x7f1d300b21a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d48108640 0x7f1d48199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d48103af0 0x7f1d4819f850 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 --2- 172.21.15.78:0/596066319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d48107cf0 0x7f1d4819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 >> 172.21.15.78:0/596066319 conn(0x7f1d480fe5c0 msgr2=0x7f1d48109550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 shutdown_connections 2024-04-25T13:58:10.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:10.154+0000 7f1d4d997700 1 -- 172.21.15.78:0/596066319 wait complete. 2024-04-25T13:58:10.463 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:10 smithi078 ceph-mon[29163]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T13:58:10.463 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:10 smithi078 ceph-mon[29163]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:58:10.463 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:10 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/596066319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:58:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:10 smithi112 ceph-mon[32617]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T13:58:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:10 smithi112 ceph-mon[32617]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:58:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:10 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/596066319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:58:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:10 smithi129 ceph-mon[32569]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T13:58:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:10 smithi129 ceph-mon[32569]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:58:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:10 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/596066319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:58:11.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:11 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:58:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:11 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:58:11.562 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:11 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:58:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 -- 172.21.15.78:0/3072581494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8108600 msgr2=0x7efcb810ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 --2- 172.21.15.78:0/3072581494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8108600 0x7efcb810ff60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efcb000b0d0 tx=0x7efcb000b3e0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 -- 172.21.15.78:0/3072581494 shutdown_connections 2024-04-25T13:58:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 --2- 172.21.15.78:0/3072581494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8108600 0x7efcb810ff60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 --2- 172.21.15.78:0/3072581494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb8103380 0x7efcb8103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 --2- 172.21.15.78:0/3072581494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8107cb0 0x7efcb81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 -- 172.21.15.78:0/3072581494 >> 172.21.15.78:0/3072581494 conn(0x7efcb80fecb0 msgr2=0x7efcb81010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 -- 172.21.15.78:0/3072581494 shutdown_connections 2024-04-25T13:58:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.635+0000 7efcbf180700 1 -- 172.21.15.78:0/3072581494 wait complete. 2024-04-25T13:58:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 Processor -- start 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 -- start start 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8103380 0x7efcb819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8107cb0 0x7efcb819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb8108600 0x7efcb81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81132a0 con 0x7efcb8103380 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8113120 con 0x7efcb8107cb0 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.636+0000 7efcbf180700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8113420 con 0x7efcb8108600 2024-04-25T13:58:11.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8103380 0x7efcb819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8103380 0x7efcb819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46018/0 (socket says 172.21.15.78:46018) 2024-04-25T13:58:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 -- 172.21.15.78:0/1454066952 learned_addr learned my addr 172.21.15.78:0/1454066952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcb7fff700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8107cb0 0x7efcb819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbd71d700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb8108600 0x7efcb81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 -- 172.21.15.78:0/1454066952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb8108600 msgr2=0x7efcb81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb8108600 0x7efcb81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 -- 172.21.15.78:0/1454066952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8107cb0 msgr2=0x7efcb819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8107cb0 0x7efcb819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 -- 172.21.15.78:0/1454066952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb819a070 con 0x7efcb8103380 2024-04-25T13:58:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbd71d700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb8108600 0x7efcb81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbcf1c700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8103380 0x7efcb819f7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efcac00d950 tx=0x7efcac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcb5ffb700 1 -- 172.21.15.78:0/1454066952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac016070 con 0x7efcb8103380 2024-04-25T13:58:11.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb819a360 con 0x7efcb8103380 2024-04-25T13:58:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcb5ffb700 1 -- 172.21.15.78:0/1454066952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac009df0 con 0x7efcb8103380 2024-04-25T13:58:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.637+0000 7efcb5ffb700 1 -- 172.21.15.78:0/1454066952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac00f3c0 con 0x7efcb8103380 2024-04-25T13:58:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.638+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81a6630 con 0x7efcb8103380 2024-04-25T13:58:11.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.638+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb8051190 con 0x7efcb8103380 2024-04-25T13:58:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.640+0000 7efcb5ffb700 1 -- 172.21.15.78:0/1454066952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7efcac019070 con 0x7efcb8103380 2024-04-25T13:58:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.640+0000 7efcb5ffb700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efca80afc10 0x7efca80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:11.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.640+0000 7efcb7fff700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efca80afc10 0x7efca80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.640+0000 7efcb5ffb700 1 -- 172.21.15.78:0/1454066952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efcac0de9e0 con 0x7efcb8103380 2024-04-25T13:58:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.641+0000 7efcb7fff700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efca80afc10 0x7efca80b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efcb8106ab0 tx=0x7efca4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:11.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.645+0000 7efcb5ffb700 1 -- 172.21.15.78:0/1454066952 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efcac0a7450 con 0x7efcb8103380 2024-04-25T13:58:11.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.852+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efcb802e510 con 0x7efcb8103380 2024-04-25T13:58:11.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.853+0000 7efcb5ffb700 1 -- 172.21.15.78:0/1454066952 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efcac0a6d50 con 0x7efcb8103380 2024-04-25T13:58:11.853 INFO:teuthology.orchestra.run.smithi078.stdout:34359738449 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efca80afc10 msgr2=0x7efca80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efca80afc10 0x7efca80b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efcb8106ab0 tx=0x7efca4009190 comp rx=0 tx=0).stop 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8103380 msgr2=0x7efcb819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8103380 0x7efcb819f7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efcac00d950 tx=0x7efcac00dc60 comp rx=0 tx=0).stop 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 shutdown_connections 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efca80afc10 0x7efca80b2120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb8108600 0x7efcb81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb8103380 0x7efcb819f7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 --2- 172.21.15.78:0/1454066952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb8107cb0 0x7efcb819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:11.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 >> 172.21.15.78:0/1454066952 conn(0x7efcb80fecb0 msgr2=0x7efcb8107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:11.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 shutdown_connections 2024-04-25T13:58:11.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:11.855+0000 7efcbf180700 1 -- 172.21.15.78:0/1454066952 wait complete. 2024-04-25T13:58:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1454066952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:58:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:12 smithi112 ceph-mon[32617]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:58:12.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1454066952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:58:12.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:12 smithi078 ceph-mon[29163]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:58:12.551 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1454066952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:58:12.552 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:12 smithi129 ceph-mon[32569]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T13:58:13.354 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:13 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.355 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:13 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.355 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:13 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.355 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:13 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.381 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411399 for osd.2 2024-04-25T13:58:13.381 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:13.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:13 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:13 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:13 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:13 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:13 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:13 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:13 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:13 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:14.395 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:14 smithi078 ceph-mon[29163]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 202 B/s, 0 objects/s recovering 2024-04-25T13:58:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:14 smithi129 ceph-mon[32569]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 202 B/s, 0 objects/s recovering 2024-04-25T13:58:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:14 smithi112 ceph-mon[32617]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 202 B/s, 0 objects/s recovering 2024-04-25T13:58:16.815 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738449 for osd.0 2024-04-25T13:58:16.816 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:17.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:17 smithi078 ceph-mon[29163]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 202 B/s, 0 objects/s recovering 2024-04-25T13:58:17.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:17 smithi112 ceph-mon[32617]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 202 B/s, 0 objects/s recovering 2024-04-25T13:58:17.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:17 smithi129 ceph-mon[32569]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s; 202 B/s, 0 objects/s recovering 2024-04-25T13:58:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.683+0000 7f82a2cde700 1 -- 172.21.15.78:0/1304501775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 msgr2=0x7f829c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.683+0000 7f82a2cde700 1 --2- 172.21.15.78:0/1304501775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c10ff40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f829400b0d0 tx=0x7f829400b3e0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.683+0000 7f82a2cde700 1 -- 172.21.15.78:0/1304501775 shutdown_connections 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.683+0000 7f82a2cde700 1 --2- 172.21.15.78:0/1304501775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c10ff40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.683+0000 7f82a2cde700 1 --2- 172.21.15.78:0/1304501775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829c103360 0x7f829c103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 --2- 172.21.15.78:0/1304501775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f829c107c90 0x7f829c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 -- 172.21.15.78:0/1304501775 >> 172.21.15.78:0/1304501775 conn(0x7f829c0fec90 msgr2=0x7f829c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 -- 172.21.15.78:0/1304501775 shutdown_connections 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 -- 172.21.15.78:0/1304501775 wait complete. 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 Processor -- start 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 -- start start 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829c103360 0x7f829c1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.684+0000 7f82a2cde700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f829c107c90 0x7f829c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a2cde700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a2cde700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f829c113210 con 0x7f829c1085e0 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a2cde700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f829c113090 con 0x7f829c103360 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a2cde700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f829c113390 con 0x7f829c107c90 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41340/0 (socket says 172.21.15.78:41340) 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 -- 172.21.15.78:0/434693300 learned_addr learned my addr 172.21.15.78:0/434693300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a0a7a700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829c103360 0x7f829c1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f829bfff700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f829c107c90 0x7f829c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 -- 172.21.15.78:0/434693300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f829c107c90 msgr2=0x7f829c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f829c107c90 0x7f829c1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 -- 172.21.15.78:0/434693300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829c103360 msgr2=0x7f829c1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829c103360 0x7f829c1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 -- 172.21.15.78:0/434693300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c19b080 con 0x7f829c1085e0 2024-04-25T13:58:17.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a127b700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c19aa40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f82940095c0 tx=0x7f8294007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f8299ffb700 1 -- 172.21.15.78:0/434693300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829401f070 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f8299ffb700 1 -- 172.21.15.78:0/434693300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8294003c00 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f8299ffb700 1 -- 172.21.15.78:0/434693300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829401b6a0 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c19b310 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.685+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c19b7f0 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.687+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c051190 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.688+0000 7f8299ffb700 1 -- 172.21.15.78:0/434693300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8294022070 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.689+0000 7f8299ffb700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f828c0afc10 0x7f828c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.689+0000 7f82a0a7a700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f828c0afc10 0x7f828c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.689+0000 7f8299ffb700 1 -- 172.21.15.78:0/434693300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f829400f020 con 0x7f829c1085e0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.689+0000 7f82a0a7a700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f828c0afc10 0x7f828c0b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f829c19bb80 tx=0x7f829000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.692+0000 7f8299ffb700 1 -- 172.21.15.78:0/434693300 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8294029020 con 0x7f829c1085e0 2024-04-25T13:58:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.900+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f829c02e510 con 0x7f829c1085e0 2024-04-25T13:58:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.900+0000 7f8299ffb700 1 -- 172.21.15.78:0/434693300 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f82940a8f30 con 0x7f829c1085e0 2024-04-25T13:58:17.900 INFO:teuthology.orchestra.run.smithi078.stdout:154618822713 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f828c0afc10 msgr2=0x7f828c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f828c0afc10 0x7f828c0b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f829c19bb80 tx=0x7f829000b410 comp rx=0 tx=0).stop 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 msgr2=0x7f829c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c19aa40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f82940095c0 tx=0x7f8294007a20 comp rx=0 tx=0).stop 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 shutdown_connections 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f828c0afc10 0x7f828c0b2120 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f829c1085e0 0x7f829c19aa40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f829c103360 0x7f829c1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 --2- 172.21.15.78:0/434693300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f829c107c90 0x7f829c1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 >> 172.21.15.78:0/434693300 conn(0x7f829c0fec90 msgr2=0x7f829c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 shutdown_connections 2024-04-25T13:58:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.903+0000 7f82a2cde700 1 -- 172.21.15.78:0/434693300 wait complete. 2024-04-25T13:58:17.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.940+0000 7faefe0c4700 1 -- 172.21.15.78:0/2696149804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef81037e0 msgr2=0x7faef8103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.940+0000 7faefe0c4700 1 --2- 172.21.15.78:0/2696149804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef81037e0 0x7faef8103bf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faeec009990 tx=0x7faeec009ca0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.940+0000 7faefe0c4700 1 -- 172.21.15.78:0/2696149804 shutdown_connections 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.940+0000 7faefe0c4700 1 --2- 172.21.15.78:0/2696149804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faef8104130 0x7faef810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.940+0000 7faefe0c4700 1 --2- 172.21.15.78:0/2696149804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faef8109200 0x7faef8109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.940+0000 7faefe0c4700 1 --2- 172.21.15.78:0/2696149804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef81037e0 0x7faef8103bf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.941+0000 7faefe0c4700 1 -- 172.21.15.78:0/2696149804 >> 172.21.15.78:0/2696149804 conn(0x7faef80fe3f0 msgr2=0x7faef8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.941+0000 7faefe0c4700 1 -- 172.21.15.78:0/2696149804 shutdown_connections 2024-04-25T13:58:17.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.941+0000 7faefe0c4700 1 -- 172.21.15.78:0/2696149804 wait complete. 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.941+0000 7faefe0c4700 1 Processor -- start 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.941+0000 7faefe0c4700 1 -- start start 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faefe0c4700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faef81037e0 0x7faef819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faefe0c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faef8104130 0x7faef819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faefe0c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8109200 0x7faef819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faefe0c4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faef8113330 con 0x7faef8109200 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faefe0c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faef81131b0 con 0x7faef8104130 2024-04-25T13:58:17.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faefe0c4700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faef81134b0 con 0x7faef81037e0 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8109200 0x7faef819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8109200 0x7faef819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41362/0 (socket says 172.21.15.78:41362) 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 -- 172.21.15.78:0/3285302278 learned_addr learned my addr 172.21.15.78:0/3285302278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef77fe700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faef81037e0 0x7faef819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef6ffd700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faef8104130 0x7faef819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 -- 172.21.15.78:0/3285302278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faef81037e0 msgr2=0x7faef819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faef81037e0 0x7faef819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 -- 172.21.15.78:0/3285302278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faef8104130 msgr2=0x7faef819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faef8104130 0x7faef819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 -- 172.21.15.78:0/3285302278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef81a6fc0 con 0x7faef8109200 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.942+0000 7faef7fff700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8109200 0x7faef819c710 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faef0007ee0 tx=0x7faef000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.943+0000 7faef4ff9700 1 -- 172.21.15.78:0/3285302278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef000dd20 con 0x7faef8109200 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.943+0000 7faef4ff9700 1 -- 172.21.15.78:0/3285302278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef0004020 con 0x7faef8109200 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.943+0000 7faef4ff9700 1 -- 172.21.15.78:0/3285302278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0015660 con 0x7faef8109200 2024-04-25T13:58:17.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.943+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef81a72b0 con 0x7faef8109200 2024-04-25T13:58:17.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.943+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81a77a0 con 0x7faef8109200 2024-04-25T13:58:17.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.944+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef810b640 con 0x7faef8109200 2024-04-25T13:58:17.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.946+0000 7faef4ff9700 1 -- 172.21.15.78:0/3285302278 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7faef00041d0 con 0x7faef8109200 2024-04-25T13:58:17.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.946+0000 7faef4ff9700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7faee00afc90 0x7faee00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:17.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.947+0000 7faef77fe700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7faee00afc90 0x7faee00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:17.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.947+0000 7faef4ff9700 1 -- 172.21.15.78:0/3285302278 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7faef00def70 con 0x7faef8109200 2024-04-25T13:58:17.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.947+0000 7faef77fe700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7faee00afc90 0x7faee00b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faee8005950 tx=0x7faee800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:17.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.950+0000 7faef4ff9700 1 -- 172.21.15.78:0/3285302278 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faef0010070 con 0x7faef8109200 2024-04-25T13:58:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.153+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7faef8102ce0 con 0x7faef8109200 2024-04-25T13:58:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.153+0000 7faef4ff9700 1 -- 172.21.15.78:0/3285302278 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faef00a79e0 con 0x7faef8109200 2024-04-25T13:58:18.153 INFO:teuthology.orchestra.run.smithi078.stdout:253403070501 2024-04-25T13:58:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7faee00afc90 msgr2=0x7faee00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:18.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7faee00afc90 0x7faee00b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faee8005950 tx=0x7faee800a300 comp rx=0 tx=0).stop 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8109200 msgr2=0x7faef819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8109200 0x7faef819c710 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faef0007ee0 tx=0x7faef000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 shutdown_connections 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7faee00afc90 0x7faee00b21a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faef8109200 0x7faef819c710 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faef81037e0 0x7faef819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 --2- 172.21.15.78:0/3285302278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faef8104130 0x7faef819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 >> 172.21.15.78:0/3285302278 conn(0x7faef80fe3f0 msgr2=0x7faef8108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 shutdown_connections 2024-04-25T13:58:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:18.155+0000 7faefe0c4700 1 -- 172.21.15.78:0/3285302278 wait complete. 2024-04-25T13:58:18.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:18 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4036481821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:58:18.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:18 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/434693300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:58:18.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:18 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3285302278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:58:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:18 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4036481821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:58:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:18 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/434693300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:58:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:18 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3285302278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:58:18.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822708 got 154618822713 for osd.5 2024-04-25T13:58:18.582 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:18 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4036481821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:58:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:18 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/434693300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:58:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:18 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3285302278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:58:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 -- 172.21.15.78:0/3469304521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 msgr2=0x7fe8981080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 --2- 172.21.15.78:0/3469304521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe8981080a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe8800099c0 tx=0x7fe880009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 -- 172.21.15.78:0/3469304521 shutdown_connections 2024-04-25T13:58:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 --2- 172.21.15.78:0/3469304521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8981085e0 0x7fe89810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 --2- 172.21.15.78:0/3469304521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe898103360 0x7fe898103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 --2- 172.21.15.78:0/3469304521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe8981080a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 -- 172.21.15.78:0/3469304521 >> 172.21.15.78:0/3469304521 conn(0x7fe8980fec90 msgr2=0x7fe8981010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 -- 172.21.15.78:0/3469304521 shutdown_connections 2024-04-25T13:58:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.187+0000 7fe89d283700 1 -- 172.21.15.78:0/3469304521 wait complete. 2024-04-25T13:58:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 Processor -- start 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 -- start start 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe898103360 0x7fe89819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe89819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8981085e0 0x7fe89819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981132e0 con 0x7fe898107c90 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe898113160 con 0x7fe8981085e0 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe89d283700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe898113460 con 0x7fe898103360 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe88e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe89819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe88e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe89819c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41382/0 (socket says 172.21.15.78:41382) 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe88e7fc700 1 -- 172.21.15.78:0/725952212 learned_addr learned my addr 172.21.15.78:0/725952212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.188+0000 7fe8977fe700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8981085e0 0x7fe89819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe896ffd700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe898103360 0x7fe89819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe88e7fc700 1 -- 172.21.15.78:0/725952212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe898103360 msgr2=0x7fe89819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe88e7fc700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe898103360 0x7fe89819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe88e7fc700 1 -- 172.21.15.78:0/725952212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8981085e0 msgr2=0x7fe89819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe88e7fc700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8981085e0 0x7fe89819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe88e7fc700 1 -- 172.21.15.78:0/725952212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8981a6f50 con 0x7fe898107c90 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe88e7fc700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe89819c120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe880009f50 tx=0x7fe8800037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe894ff9700 1 -- 172.21.15.78:0/725952212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880027070 con 0x7fe898107c90 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8981a71e0 con 0x7fe898107c90 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe894ff9700 1 -- 172.21.15.78:0/725952212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe880003fa0 con 0x7fe898107c90 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe894ff9700 1 -- 172.21.15.78:0/725952212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880022d20 con 0x7fe898107c90 2024-04-25T13:58:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.189+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981a7690 con 0x7fe898107c90 2024-04-25T13:58:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.190+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe898051190 con 0x7fe898107c90 2024-04-25T13:58:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.191+0000 7fe894ff9700 1 -- 172.21.15.78:0/725952212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe88002a070 con 0x7fe898107c90 2024-04-25T13:58:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.192+0000 7fe894ff9700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe8840afc10 0x7fe8840b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.192+0000 7fe896ffd700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe8840afc10 0x7fe8840b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.192+0000 7fe894ff9700 1 -- 172.21.15.78:0/725952212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe8800dfcb0 con 0x7fe898107c90 2024-04-25T13:58:19.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.193+0000 7fe896ffd700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe8840afc10 0x7fe8840b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe888009a10 tx=0x7fe888009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.197+0000 7fe894ff9700 1 -- 172.21.15.78:0/725952212 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe880013070 con 0x7fe898107c90 2024-04-25T13:58:19.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:19 smithi078 ceph-mon[29163]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 162 B/s, 0 objects/s recovering 2024-04-25T13:58:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.403+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe89802e510 con 0x7fe898107c90 2024-04-25T13:58:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.404+0000 7fe894ff9700 1 -- 172.21.15.78:0/725952212 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe8800a87b0 con 0x7fe898107c90 2024-04-25T13:58:19.404 INFO:teuthology.orchestra.run.smithi078.stdout:180388626484 2024-04-25T13:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe8840afc10 msgr2=0x7fe8840b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe8840afc10 0x7fe8840b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe888009a10 tx=0x7fe888009190 comp rx=0 tx=0).stop 2024-04-25T13:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 msgr2=0x7fe89819c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe89819c120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe880009f50 tx=0x7fe8800037c0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 shutdown_connections 2024-04-25T13:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe8840afc10 0x7fe8840b2120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8981085e0 0x7fe89819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe898103360 0x7fe89819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 --2- 172.21.15.78:0/725952212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe898107c90 0x7fe89819c120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 >> 172.21.15.78:0/725952212 conn(0x7fe8980fec90 msgr2=0x7fe8981010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:19.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 shutdown_connections 2024-04-25T13:58:19.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:19.406+0000 7fe89d283700 1 -- 172.21.15.78:0/725952212 wait complete. 2024-04-25T13:58:19.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:19 smithi112 ceph-mon[32617]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 162 B/s, 0 objects/s recovering 2024-04-25T13:58:19.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:19 smithi129 ceph-mon[32569]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 162 B/s, 0 objects/s recovering 2024-04-25T13:58:19.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 -- 172.21.15.78:0/1062874707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940107c90 msgr2=0x7f09401080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 --2- 172.21.15.78:0/1062874707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940107c90 0x7f09401080a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f09300099c0 tx=0x7f0930009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 -- 172.21.15.78:0/1062874707 shutdown_connections 2024-04-25T13:58:19.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 --2- 172.21.15.78:0/1062874707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09401085e0 0x7f094010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 --2- 172.21.15.78:0/1062874707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0940103360 0x7f0940103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 --2- 172.21.15.78:0/1062874707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940107c90 0x7f09401080a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 -- 172.21.15.78:0/1062874707 >> 172.21.15.78:0/1062874707 conn(0x7f09400fec90 msgr2=0x7f09401010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 -- 172.21.15.78:0/1062874707 shutdown_connections 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.021+0000 7f09482ae700 1 -- 172.21.15.78:0/1062874707 wait complete. 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 Processor -- start 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 -- start start 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940103360 0x7f094019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0940107c90 0x7f094019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09401085e0 0x7f0940199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09401131f0 con 0x7f0940103360 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0940113070 con 0x7f0940107c90 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f09482ae700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0940113370 con 0x7f09401085e0 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f094604a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940103360 0x7f094019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f094604a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940103360 0x7f094019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41320/0 (socket says 172.21.15.78:41320) 2024-04-25T13:58:19.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.022+0000 7f094604a700 1 -- 172.21.15.78:0/4036481821 learned_addr learned my addr 172.21.15.78:0/4036481821 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:19.737 INFO:teuthology.orchestra.run.smithi078.stdout:107374182466 2024-04-25T13:58:19.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f0945849700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0940107c90 0x7f094019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f094684b700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09401085e0 0x7f0940199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f094604a700 1 -- 172.21.15.78:0/4036481821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09401085e0 msgr2=0x7f0940199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f094604a700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09401085e0 0x7f0940199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f094604a700 1 -- 172.21.15.78:0/4036481821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0940107c90 msgr2=0x7f094019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f094604a700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0940107c90 0x7f094019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f094604a700 1 -- 172.21.15.78:0/4036481821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094019a040 con 0x7f0940103360 2024-04-25T13:58:19.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f094604a700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940103360 0x7f094019f770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f093c00d950 tx=0x7f093c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:19.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f09377fe700 1 -- 172.21.15.78:0/4036481821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c016070 con 0x7f0940103360 2024-04-25T13:58:19.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094019a330 con 0x7f0940103360 2024-04-25T13:58:19.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f09377fe700 1 -- 172.21.15.78:0/4036481821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f093c00fd40 con 0x7f0940103360 2024-04-25T13:58:19.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f09377fe700 1 -- 172.21.15.78:0/4036481821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c00e3b0 con 0x7f0940103360 2024-04-25T13:58:19.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.023+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094006b4e0 con 0x7f0940103360 2024-04-25T13:58:19.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.025+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0940051190 con 0x7f0940103360 2024-04-25T13:58:19.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.027+0000 7f09377fe700 1 -- 172.21.15.78:0/4036481821 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f093c019070 con 0x7f0940103360 2024-04-25T13:58:19.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.028+0000 7f09377fe700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f092c0afc10 0x7f092c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:19.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.028+0000 7f0945849700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f092c0afc10 0x7f092c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:19.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.028+0000 7f09377fe700 1 -- 172.21.15.78:0/4036481821 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f093c0de820 con 0x7f0940103360 2024-04-25T13:58:19.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.028+0000 7f0945849700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f092c0afc10 0x7f092c0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0930006010 tx=0x7f0930000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:19.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.032+0000 7f09377fe700 1 -- 172.21.15.78:0/4036481821 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f093c0a7290 con 0x7f0940103360 2024-04-25T13:58:19.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.273+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f094006bfa0 con 0x7f0940103360 2024-04-25T13:58:19.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.273+0000 7f09377fe700 1 -- 172.21.15.78:0/4036481821 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f093c0a6b90 con 0x7f0940103360 2024-04-25T13:58:19.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f092c0afc10 msgr2=0x7f092c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f092c0afc10 0x7f092c0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0930006010 tx=0x7f0930000bc0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940103360 msgr2=0x7f094019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:19.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940103360 0x7f094019f770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f093c00d950 tx=0x7f093c00dc60 comp rx=0 tx=0).stop 2024-04-25T13:58:19.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 shutdown_connections 2024-04-25T13:58:19.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f092c0afc10 0x7f092c0b2120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09401085e0 0x7f0940199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0940103360 0x7f094019f770 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 --2- 172.21.15.78:0/4036481821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0940107c90 0x7f094019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:19.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.275+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 >> 172.21.15.78:0/4036481821 conn(0x7f09400fec90 msgr2=0x7f0940109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:19.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.276+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 shutdown_connections 2024-04-25T13:58:19.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:17.276+0000 7f09482ae700 1 -- 172.21.15.78:0/4036481821 wait complete. 2024-04-25T13:58:20.311 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:20 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/725952212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:58:20.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182466 for osd.3 2024-04-25T13:58:20.342 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:20.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:20 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/725952212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:58:20.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:20 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/725952212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:58:21.103 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626478 got 180388626484 for osd.6 2024-04-25T13:58:21.104 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:21.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.309+0000 7f55aeee9700 1 -- 172.21.15.78:0/2522253672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 msgr2=0x7f55a8108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:21.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.309+0000 7f55aeee9700 1 --2- 172.21.15.78:0/2522253672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a8108cd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f559c009990 tx=0x7f559c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 -- 172.21.15.78:0/2522253672 shutdown_connections 2024-04-25T13:58:21.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 --2- 172.21.15.78:0/2522253672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a81031e0 0x7f55a810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 --2- 172.21.15.78:0/2522253672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a8108cd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 --2- 172.21.15.78:0/2522253672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55a8102890 0x7f55a8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 -- 172.21.15.78:0/2522253672 >> 172.21.15.78:0/2522253672 conn(0x7f55a80fe3f0 msgr2=0x7f55a8100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 -- 172.21.15.78:0/2522253672 shutdown_connections 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 -- 172.21.15.78:0/2522253672 wait complete. 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 Processor -- start 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.310+0000 7f55aeee9700 1 -- start start 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55aeee9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55a8102890 0x7f55a819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55aeee9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a81031e0 0x7f55a819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55aeee9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55aeee9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81121c0 con 0x7f55a81088c0 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55aeee9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8112040 con 0x7f55a81031e0 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55aeee9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8112340 con 0x7f55a8102890 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:21.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41410/0 (socket says 172.21.15.78:41410) 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 -- 172.21.15.78:0/1955694355 learned_addr learned my addr 172.21.15.78:0/1955694355 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55a7fff700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a81031e0 0x7f55a819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 -- 172.21.15.78:0/1955694355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55a8102890 msgr2=0x7f55a819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55acc85700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55a8102890 0x7f55a819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55a8102890 0x7f55a819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 -- 172.21.15.78:0/1955694355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a81031e0 msgr2=0x7f55a819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a81031e0 0x7f55a819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 -- 172.21.15.78:0/1955694355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a819fc70 con 0x7f55a81088c0 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55ad486700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a819f5d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009fa0 tx=0x7f55a00076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55a5ffb700 1 -- 172.21.15.78:0/1955694355 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0019070 con 0x7f55a81088c0 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55a5ffb700 1 -- 172.21.15.78:0/1955694355 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a0003c20 con 0x7f55a81088c0 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a819ff60 con 0x7f55a81088c0 2024-04-25T13:58:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.311+0000 7f55a5ffb700 1 -- 172.21.15.78:0/1955694355 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a00146e0 con 0x7f55a81088c0 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.312+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a81a7880 con 0x7f55a81088c0 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.312+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a8051190 con 0x7f55a81088c0 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.314+0000 7f55a5ffb700 1 -- 172.21.15.78:0/1955694355 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f55a001c070 con 0x7f55a81088c0 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.315+0000 7f55a5ffb700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f55980afc10 0x7f55980b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.315+0000 7f55acc85700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f55980afc10 0x7f55980b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.315+0000 7f55a5ffb700 1 -- 172.21.15.78:0/1955694355 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f55a00de9b0 con 0x7f55a81088c0 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.315+0000 7f55acc85700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f55980afc10 0x7f55980b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b2e0 tx=0x7f559c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:21.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.318+0000 7f55a5ffb700 1 -- 172.21.15.78:0/1955694355 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55a0023080 con 0x7f55a81088c0 2024-04-25T13:58:21.354 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:21 smithi078 ceph-mon[29163]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 147 B/s, 0 objects/s recovering 2024-04-25T13:58:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:21 smithi112 ceph-mon[32617]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 147 B/s, 0 objects/s recovering 2024-04-25T13:58:21.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.524+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f55a802e510 con 0x7f55a81088c0 2024-04-25T13:58:21.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.524+0000 7f55a5ffb700 1 -- 172.21.15.78:0/1955694355 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55a0011070 con 0x7f55a81088c0 2024-04-25T13:58:21.524 INFO:teuthology.orchestra.run.smithi078.stdout:133143986239 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f55980afc10 msgr2=0x7f55980b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f55980afc10 0x7f55980b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b2e0 tx=0x7f559c005fb0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 msgr2=0x7f55a819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a819f5d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009fa0 tx=0x7f55a00076c0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 shutdown_connections 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f55980afc10 0x7f55980b2120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a81088c0 0x7f55a819f5d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55a8102890 0x7f55a819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 --2- 172.21.15.78:0/1955694355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a81031e0 0x7f55a819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 >> 172.21.15.78:0/1955694355 conn(0x7f55a80fe3f0 msgr2=0x7f55a8100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 shutdown_connections 2024-04-25T13:58:21.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:21.527+0000 7f55aeee9700 1 -- 172.21.15.78:0/1955694355 wait complete. 2024-04-25T13:58:21.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:21 smithi129 ceph-mon[32569]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 147 B/s, 0 objects/s recovering 2024-04-25T13:58:22.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:22 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1955694355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:58:22.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:22 smithi078 ceph-mon[29163]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T13:58:22.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070495 got 253403070501 for osd.9 2024-04-25T13:58:22.420 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:22.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:22 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1955694355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:58:22.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:22 smithi112 ceph-mon[32617]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T13:58:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:22 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1955694355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:58:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:22 smithi129 ceph-mon[32569]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T13:58:22.797 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710740 got 300647710745 for osd.11 2024-04-25T13:58:22.797 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:22.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.984+0000 7fdca64ab700 1 -- 172.21.15.78:0/1620488441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 msgr2=0x7fdca01091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.984+0000 7fdca64ab700 1 --2- 172.21.15.78:0/1620488441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca01091f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc940099c0 tx=0x7fdc94009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 -- 172.21.15.78:0/1620488441 shutdown_connections 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 --2- 172.21.15.78:0/1620488441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdca0109730 0x7fdca0111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 --2- 172.21.15.78:0/1620488441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0102960 0x7fdca0102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 --2- 172.21.15.78:0/1620488441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca01091f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 -- 172.21.15.78:0/1620488441 >> 172.21.15.78:0/1620488441 conn(0x7fdca00fe3f0 msgr2=0x7fdca0100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 -- 172.21.15.78:0/1620488441 shutdown_connections 2024-04-25T13:58:22.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 -- 172.21.15.78:0/1620488441 wait complete. 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 Processor -- start 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.985+0000 7fdca64ab700 1 -- start start 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca64ab700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdca0102960 0x7fdca019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca64ab700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca64ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109730 0x7fdca019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca64ab700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0114450 con 0x7fdca0108de0 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca64ab700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01142d0 con 0x7fdca0109730 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca64ab700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01145d0 con 0x7fdca0102960 2024-04-25T13:58:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41428/0 (socket says 172.21.15.78:41428) 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 -- 172.21.15.78:0/40971563 learned_addr learned my addr 172.21.15.78:0/40971563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9ffff700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdca0102960 0x7fdca019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca4a48700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109730 0x7fdca019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 -- 172.21.15.78:0/40971563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdca0102960 msgr2=0x7fdca019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdca0102960 0x7fdca019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 -- 172.21.15.78:0/40971563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109730 msgr2=0x7fdca019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109730 0x7fdca019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdc9f7fe700 1 -- 172.21.15.78:0/40971563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca019fcd0 con 0x7fdca0108de0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.986+0000 7fdca4a48700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109730 0x7fdca019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.987+0000 7fdc9f7fe700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca019b3a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009f50 tx=0x7fdc940037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.987+0000 7fdc9d7fa700 1 -- 172.21.15.78:0/40971563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94027070 con 0x7fdca0108de0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.987+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca019ff60 con 0x7fdca0108de0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.987+0000 7fdc9d7fa700 1 -- 172.21.15.78:0/40971563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc94004090 con 0x7fdca0108de0 2024-04-25T13:58:22.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.987+0000 7fdc9d7fa700 1 -- 172.21.15.78:0/40971563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94022e60 con 0x7fdca0108de0 2024-04-25T13:58:22.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.987+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a78e0 con 0x7fdca0108de0 2024-04-25T13:58:22.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.988+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca0051190 con 0x7fdca0108de0 2024-04-25T13:58:22.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.989+0000 7fdc9d7fa700 1 -- 172.21.15.78:0/40971563 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fdc9402a070 con 0x7fdca0108de0 2024-04-25T13:58:22.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.990+0000 7fdc9d7fa700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc880afc10 0x7fdc880b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:22.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.990+0000 7fdc9d7fa700 1 -- 172.21.15.78:0/40971563 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdc940dfdb0 con 0x7fdca0108de0 2024-04-25T13:58:22.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.990+0000 7fdc9ffff700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc880afc10 0x7fdc880b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:22.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.991+0000 7fdc9ffff700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc880afc10 0x7fdc880b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdca006ae50 tx=0x7fdc90009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:22.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:22.994+0000 7fdc9d7fa700 1 -- 172.21.15.78:0/40971563 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc94013070 con 0x7fdca0108de0 2024-04-25T13:58:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.201+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fdca002e510 con 0x7fdca0108de0 2024-04-25T13:58:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.201+0000 7fdc9d7fa700 1 -- 172.21.15.78:0/40971563 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdc940a88b0 con 0x7fdca0108de0 2024-04-25T13:58:23.201 INFO:teuthology.orchestra.run.smithi078.stdout:227633266731 2024-04-25T13:58:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc880afc10 msgr2=0x7fdc880b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc880afc10 0x7fdc880b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdca006ae50 tx=0x7fdc90009190 comp rx=0 tx=0).stop 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 msgr2=0x7fdca019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca019b3a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009f50 tx=0x7fdc940037c0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 shutdown_connections 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc880afc10 0x7fdc880b2120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdca0109730 0x7fdca019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdca0102960 0x7fdca019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 --2- 172.21.15.78:0/40971563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0108de0 0x7fdca019b3a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 >> 172.21.15.78:0/40971563 conn(0x7fdca00fe3f0 msgr2=0x7fdca010a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 shutdown_connections 2024-04-25T13:58:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.204+0000 7fdca64ab700 1 -- 172.21.15.78:0/40971563 wait complete. 2024-04-25T13:58:23.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:23 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/40971563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:58:23.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986232 got 133143986239 for osd.4 2024-04-25T13:58:23.393 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:23.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:23 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/40971563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:58:23.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:23 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/40971563' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:58:23.601 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.606+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3324420343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105390 msgr2=0x7fb2881057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.606+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3324420343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105390 0x7fb2881057a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009990 tx=0x7fb27c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.606+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3324420343 shutdown_connections 2024-04-25T13:58:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.606+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3324420343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb28810e7c0 0x7fb288110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.606+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3324420343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105390 0x7fb2881057a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.606+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3324420343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb288105ce0 0x7fb28810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.606+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3324420343 >> 172.21.15.78:0/3324420343 conn(0x7fb2880fec90 msgr2=0x7fb2881010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.607+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3324420343 shutdown_connections 2024-04-25T13:58:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.607+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3324420343 wait complete. 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.607+0000 7fb28ddf6700 1 Processor -- start 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.607+0000 7fb28ddf6700 1 -- start start 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb28ddf6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288105390 0x7fb28819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb28ddf6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105ce0 0x7fb28819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb28ddf6700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28810e7c0 0x7fb28819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb28ddf6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2881143e0 con 0x7fb288105ce0 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb28ddf6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb288114260 con 0x7fb288105390 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb28ddf6700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb288114560 con 0x7fb28810e7c0 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105ce0 0x7fb28819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105ce0 0x7fb28819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41444/0 (socket says 172.21.15.78:41444) 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 -- 172.21.15.78:0/3022702147 learned_addr learned my addr 172.21.15.78:0/3022702147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb287fff700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28810e7c0 0x7fb28819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb2877fe700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288105390 0x7fb28819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 -- 172.21.15.78:0/3022702147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28810e7c0 msgr2=0x7fb28819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28810e7c0 0x7fb28819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 -- 172.21.15.78:0/3022702147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288105390 msgr2=0x7fb28819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288105390 0x7fb28819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.608+0000 7fb286ffd700 1 -- 172.21.15.78:0/3022702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28819fb90 con 0x7fb288105ce0 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.609+0000 7fb286ffd700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105ce0 0x7fb28819b310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb27800c9b0 tx=0x7fb27800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.609+0000 7fb284ff9700 1 -- 172.21.15.78:0/3022702147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278007b00 con 0x7fb288105ce0 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.609+0000 7fb284ff9700 1 -- 172.21.15.78:0/3022702147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27800ce50 con 0x7fb288105ce0 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.609+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28819fe80 con 0x7fb288105ce0 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.609+0000 7fb284ff9700 1 -- 172.21.15.78:0/3022702147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278018660 con 0x7fb288105ce0 2024-04-25T13:58:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.609+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881a03c0 con 0x7fb288105ce0 2024-04-25T13:58:23.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.610+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb288051190 con 0x7fb288105ce0 2024-04-25T13:58:23.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.611+0000 7fb284ff9700 1 -- 172.21.15.78:0/3022702147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb27800f450 con 0x7fb288105ce0 2024-04-25T13:58:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.612+0000 7fb284ff9700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2700afc10 0x7fb2700b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.612+0000 7fb2877fe700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2700afc10 0x7fb2700b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.613+0000 7fb284ff9700 1 -- 172.21.15.78:0/3022702147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb2780defd0 con 0x7fb288105ce0 2024-04-25T13:58:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.613+0000 7fb2877fe700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2700afc10 0x7fb2700b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009750 tx=0x7fb27c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.616+0000 7fb284ff9700 1 -- 172.21.15.78:0/3022702147 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb278014030 con 0x7fb288105ce0 2024-04-25T13:58:23.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.822+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb28802e510 con 0x7fb288105ce0 2024-04-25T13:58:23.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.822+0000 7fb284ff9700 1 -- 172.21.15.78:0/3022702147 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb2780a7a70 con 0x7fb288105ce0 2024-04-25T13:58:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:274877906977 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2700afc10 msgr2=0x7fb2700b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2700afc10 0x7fb2700b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009750 tx=0x7fb27c005fd0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105ce0 msgr2=0x7fb28819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105ce0 0x7fb28819b310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb27800c9b0 tx=0x7fb27800ccc0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 shutdown_connections 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb2700afc10 0x7fb2700b2120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb28810e7c0 0x7fb28819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb288105390 0x7fb28819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 --2- 172.21.15.78:0/3022702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb288105ce0 0x7fb28819b310 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 >> 172.21.15.78:0/3022702147 conn(0x7fb2880fec90 msgr2=0x7fb288100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 shutdown_connections 2024-04-25T13:58:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:23.824+0000 7fb28ddf6700 1 -- 172.21.15.78:0/3022702147 wait complete. 2024-04-25T13:58:24.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:24 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3022702147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:58:24.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:24 smithi078 ceph-mon[29163]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T13:58:24.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:24 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3022702147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:58:24.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:24 smithi112 ceph-mon[32617]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T13:58:24.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906977 for osd.10 2024-04-25T13:58:24.612 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:24 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3022702147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:58:24.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:24 smithi129 ceph-mon[32569]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-25T13:58:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 -- 172.21.15.78:0/1058036418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 msgr2=0x7f39981080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 --2- 172.21.15.78:0/1058036418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f39981080a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f39880099c0 tx=0x7f3988009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 -- 172.21.15.78:0/1058036418 shutdown_connections 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 --2- 172.21.15.78:0/1058036418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39981085e0 0x7f399810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 --2- 172.21.15.78:0/1058036418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3998103360 0x7f3998103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 --2- 172.21.15.78:0/1058036418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f39981080a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 -- 172.21.15.78:0/1058036418 >> 172.21.15.78:0/1058036418 conn(0x7f39980fecb0 msgr2=0x7f39981010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 -- 172.21.15.78:0/1058036418 shutdown_connections 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.359+0000 7f399e3a7700 1 -- 172.21.15.78:0/1058036418 wait complete. 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 Processor -- start 2024-04-25T13:58:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 -- start start 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3998103360 0x7f3998110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f3998110c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39981085e0 0x7f39981111a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3998114360 con 0x7f3998107c90 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39981141e0 con 0x7f39981085e0 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f399e3a7700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39981144e0 con 0x7f3998103360 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f39977fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f3998110c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f39977fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f3998110c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41460/0 (socket says 172.21.15.78:41460) 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.360+0000 7f39977fe700 1 -- 172.21.15.78:0/2588384190 learned_addr learned my addr 172.21.15.78:0/2588384190 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f399c944700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39981085e0 0x7f39981111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f3997fff700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3998103360 0x7f3998110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39977fe700 1 -- 172.21.15.78:0/2588384190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3998103360 msgr2=0x7f3998110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39977fe700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3998103360 0x7f3998110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39977fe700 1 -- 172.21.15.78:0/2588384190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39981085e0 msgr2=0x7f39981111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39977fe700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39981085e0 0x7f39981111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39977fe700 1 -- 172.21.15.78:0/2588384190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39981a8fe0 con 0x7f3998107c90 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39977fe700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f3998110c60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3988009f50 tx=0x7f39880037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39957fa700 1 -- 172.21.15.78:0/2588384190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988027070 con 0x7f3998107c90 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39981a9270 con 0x7f3998107c90 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39957fa700 1 -- 172.21.15.78:0/2588384190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3988003fa0 con 0x7f3998107c90 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f39957fa700 1 -- 172.21.15.78:0/2588384190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988022d20 con 0x7f3998107c90 2024-04-25T13:58:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.361+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39981a9720 con 0x7f3998107c90 2024-04-25T13:58:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.362+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3998051190 con 0x7f3998107c90 2024-04-25T13:58:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.364+0000 7f39957fa700 1 -- 172.21.15.78:0/2588384190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f398802a070 con 0x7f3998107c90 2024-04-25T13:58:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.364+0000 7f39957fa700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39800afc10 0x7f39800b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.364+0000 7f3997fff700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39800afc10 0x7f39800b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.364+0000 7f39957fa700 1 -- 172.21.15.78:0/2588384190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f39880dfe00 con 0x7f3998107c90 2024-04-25T13:58:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.365+0000 7f3997fff700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39800afc10 0x7f39800b2120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f398c00ac50 tx=0x7f398c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.369+0000 7f39957fa700 1 -- 172.21.15.78:0/2588384190 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3988013070 con 0x7f3998107c90 2024-04-25T13:58:26.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.571+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f399802e510 con 0x7f3998107c90 2024-04-25T13:58:26.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.572+0000 7f39957fa700 1 -- 172.21.15.78:0/2588384190 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f39880a8900 con 0x7f3998107c90 2024-04-25T13:58:26.571 INFO:teuthology.orchestra.run.smithi078.stdout:201863462961 2024-04-25T13:58:26.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39800afc10 msgr2=0x7f39800b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:26.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39800afc10 0x7f39800b2120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f398c00ac50 tx=0x7f398c00a380 comp rx=0 tx=0).stop 2024-04-25T13:58:26.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 msgr2=0x7f3998110c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:26.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f3998110c60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3988009f50 tx=0x7f39880037c0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 shutdown_connections 2024-04-25T13:58:26.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39800afc10 0x7f39800b2120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39981085e0 0x7f39981111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3998103360 0x7f3998110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 --2- 172.21.15.78:0/2588384190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3998107c90 0x7f3998110c60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:26.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 >> 172.21.15.78:0/2588384190 conn(0x7f39980fecb0 msgr2=0x7f3998109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:26.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 shutdown_connections 2024-04-25T13:58:26.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:26.574+0000 7f399e3a7700 1 -- 172.21.15.78:0/2588384190 wait complete. 2024-04-25T13:58:26.826 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266726 got 227633266731 for osd.8 2024-04-25T13:58:26.826 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:27.446 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:27 smithi078 ceph-mon[29163]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T13:58:27.447 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:27 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2588384190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:58:27.474 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462956 got 201863462961 for osd.7 2024-04-25T13:58:27.475 DEBUG:teuthology.parallel:result is None 2024-04-25T13:58:27.475 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T13:58:27.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:58:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:27 smithi112 ceph-mon[32617]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T13:58:27.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:27 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2588384190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:58:27.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:27 smithi129 ceph-mon[32569]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T13:58:27.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:27 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2588384190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:58:28.288 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:29.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:29 smithi078 ceph-mon[29163]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-25T13:58:29.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:29 smithi112 ceph-mon[32617]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-25T13:58:29.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:29 smithi129 ceph-mon[32569]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 -- 172.21.15.78:0/1103442979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80108db0 msgr2=0x7fcd801091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 --2- 172.21.15.78:0/1103442979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80108db0 0x7fcd801091c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcd740099c0 tx=0x7fcd74009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 -- 172.21.15.78:0/1103442979 shutdown_connections 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 --2- 172.21.15.78:0/1103442979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd80109700 0x7fcd80111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 --2- 172.21.15.78:0/1103442979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80104480 0x7fcd80104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 --2- 172.21.15.78:0/1103442979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80108db0 0x7fcd801091c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 -- 172.21.15.78:0/1103442979 >> 172.21.15.78:0/1103442979 conn(0x7fcd800ffd70 msgr2=0x7fcd801021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 -- 172.21.15.78:0/1103442979 shutdown_connections 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.007+0000 7fcd86a58700 1 -- 172.21.15.78:0/1103442979 wait complete. 2024-04-25T13:58:30.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.008+0000 7fcd86a58700 1 Processor -- start 2024-04-25T13:58:30.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.008+0000 7fcd86a58700 1 -- start start 2024-04-25T13:58:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.008+0000 7fcd86a58700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd80104480 0x7fcd80076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd86a58700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80108db0 0x7fcd80076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd86a58700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80109700 0x7fcd80077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd86a58700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80114310 con 0x7fcd80109700 2024-04-25T13:58:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd86a58700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80114190 con 0x7fcd80108db0 2024-04-25T13:58:30.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd86a58700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80114490 con 0x7fcd80104480 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd84ff5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80109700 0x7fcd80077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80108db0 0x7fcd80076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7ffff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd80104480 0x7fcd80076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd84ff5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80109700 0x7fcd80077520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49042/0 (socket says 172.21.15.78:49042) 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd84ff5700 1 -- 172.21.15.78:0/247151882 learned_addr learned my addr 172.21.15.78:0/247151882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7f7fe700 1 -- 172.21.15.78:0/247151882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd80104480 msgr2=0x7fcd80076aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7f7fe700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd80104480 0x7fcd80076aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7f7fe700 1 -- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80109700 msgr2=0x7fcd80077520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7f7fe700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80109700 0x7fcd80077520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7f7fe700 1 -- 172.21.15.78:0/247151882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd801a8fd0 con 0x7fcd80108db0 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.009+0000 7fcd7f7fe700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80108db0 0x7fcd80076fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400b580 tx=0x7fcd740047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.010+0000 7fcd7d7fa700 1 -- 172.21.15.78:0/247151882 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74027070 con 0x7fcd80108db0 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.010+0000 7fcd7d7fa700 1 -- 172.21.15.78:0/247151882 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd74003f20 con 0x7fcd80108db0 2024-04-25T13:58:30.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.010+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd801a9260 con 0x7fcd80108db0 2024-04-25T13:58:30.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.010+0000 7fcd7d7fa700 1 -- 172.21.15.78:0/247151882 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74022bc0 con 0x7fcd80108db0 2024-04-25T13:58:30.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.010+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd801a96b0 con 0x7fcd80108db0 2024-04-25T13:58:30.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.010+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd80051190 con 0x7fcd80108db0 2024-04-25T13:58:30.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.012+0000 7fcd7d7fa700 1 -- 172.21.15.78:0/247151882 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcd7402a070 con 0x7fcd80108db0 2024-04-25T13:58:30.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.013+0000 7fcd7d7fa700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcd680afc10 0x7fcd680b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:30.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.013+0000 7fcd7ffff700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcd680afc10 0x7fcd680b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:30.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.013+0000 7fcd7d7fa700 1 -- 172.21.15.78:0/247151882 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fcd740dfff0 con 0x7fcd80108db0 2024-04-25T13:58:30.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.013+0000 7fcd7ffff700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcd680afc10 0x7fcd680b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005950 tx=0x7fcd7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:30.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.017+0000 7fcd7d7fa700 1 -- 172.21.15.78:0/247151882 <== mon.2 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd74031080 con 0x7fcd80108db0 2024-04-25T13:58:30.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.230+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd80077fd0 con 0x7fcd680afc10 2024-04-25T13:58:30.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.238+0000 7fcd7d7fa700 1 -- 172.21.15.78:0/247151882 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258979 (secure 0 0 0) 0x7fcd80077fd0 con 0x7fcd680afc10 2024-04-25T13:58:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:30.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcd680afc10 msgr2=0x7fcd680b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcd680afc10 0x7fcd680b2120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005950 tx=0x7fcd7000b410 comp rx=0 tx=0).stop 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80108db0 msgr2=0x7fcd80076fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80108db0 0x7fcd80076fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400b580 tx=0x7fcd740047b0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 shutdown_connections 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcd680afc10 0x7fcd680b2120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd80109700 0x7fcd80077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd80104480 0x7fcd80076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.242+0000 7fcd86a58700 1 --2- 172.21.15.78:0/247151882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd80108db0 0x7fcd80076fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.243+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 >> 172.21.15.78:0/247151882 conn(0x7fcd800ffd70 msgr2=0x7fcd80108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.243+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 shutdown_connections 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:30.243+0000 7fcd86a58700 1 -- 172.21.15.78:0/247151882 wait complete. 2024-04-25T13:58:30.245 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T13:58:30.936 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-25T13:58:30.147280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":530,"num_read_kb":360,"num_write":282,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953457,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371912,"kb_used_data":10212,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569912,"statfs":{"total":1151940427776,"available":1151559589888,"internally_reserved":0,"allocated":10457088,"data_stored":6779724,"data_compressed":182799,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":34,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003122"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210093+0000","last_change":"2024-04-25T13:57:12.147384+0000","last_active":"2024-04-25T13:58:09.210093+0000","last_peered":"2024-04-25T13:58:09.210093+0000","last_clean":"2024-04-25T13:58:09.210093+0000","last_became_active":"2024-04-25T13:57:12.146349+0000","last_became_peered":"2024-04-25T13:57:12.146349+0000","last_unstale":"2024-04-25T13:58:09.210093+0000","last_undegraded":"2024-04-25T13:58:09.210093+0000","last_fullsized":"2024-04-25T13:58:09.210093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210422+0000","last_change":"2024-04-25T13:57:12.156698+0000","last_active":"2024-04-25T13:58:09.210422+0000","last_peered":"2024-04-25T13:58:09.210422+0000","last_clean":"2024-04-25T13:58:09.210422+0000","last_became_active":"2024-04-25T13:57:12.155790+0000","last_became_peered":"2024-04-25T13:57:12.155790+0000","last_unstale":"2024-04-25T13:58:09.210422+0000","last_undegraded":"2024-04-25T13:58:09.210422+0000","last_fullsized":"2024-04-25T13:58:09.210422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:25.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209785+0000","last_change":"2024-04-25T13:57:12.149458+0000","last_active":"2024-04-25T13:58:09.209785+0000","last_peered":"2024-04-25T13:58:09.209785+0000","last_clean":"2024-04-25T13:58:09.209785+0000","last_became_active":"2024-04-25T13:57:12.148179+0000","last_became_peered":"2024-04-25T13:57:12.148179+0000","last_unstale":"2024-04-25T13:58:09.209785+0000","last_undegraded":"2024-04-25T13:58:09.209785+0000","last_fullsized":"2024-04-25T13:58:09.209785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:00.502798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208220+0000","last_change":"2024-04-25T13:57:12.145776+0000","last_active":"2024-04-25T13:58:09.208220+0000","last_peered":"2024-04-25T13:58:09.208220+0000","last_clean":"2024-04-25T13:58:09.208220+0000","last_became_active":"2024-04-25T13:57:12.145038+0000","last_became_peered":"2024-04-25T13:57:12.145038+0000","last_unstale":"2024-04-25T13:58:09.208220+0000","last_undegraded":"2024-04-25T13:58:09.208220+0000","last_fullsized":"2024-04-25T13:58:09.208220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:52.742746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208411+0000","last_change":"2024-04-25T13:57:12.160273+0000","last_active":"2024-04-25T13:58:09.208411+0000","last_peered":"2024-04-25T13:58:09.208411+0000","last_clean":"2024-04-25T13:58:09.208411+0000","last_became_active":"2024-04-25T13:57:12.159805+0000","last_became_peered":"2024-04-25T13:57:12.159805+0000","last_unstale":"2024-04-25T13:58:09.208411+0000","last_undegraded":"2024-04-25T13:58:09.208411+0000","last_fullsized":"2024-04-25T13:58:09.208411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:15.044610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324782+0000","last_change":"2024-04-25T13:57:12.151597+0000","last_active":"2024-04-25T13:58:09.324782+0000","last_peered":"2024-04-25T13:58:09.324782+0000","last_clean":"2024-04-25T13:58:09.324782+0000","last_became_active":"2024-04-25T13:57:12.151203+0000","last_became_peered":"2024-04-25T13:57:12.151203+0000","last_unstale":"2024-04-25T13:58:09.324782+0000","last_undegraded":"2024-04-25T13:58:09.324782+0000","last_fullsized":"2024-04-25T13:58:09.324782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.645264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208087+0000","last_change":"2024-04-25T13:57:12.144959+0000","last_active":"2024-04-25T13:58:09.208087+0000","last_peered":"2024-04-25T13:58:09.208087+0000","last_clean":"2024-04-25T13:58:09.208087+0000","last_became_active":"2024-04-25T13:57:12.144587+0000","last_became_peered":"2024-04-25T13:57:12.144587+0000","last_unstale":"2024-04-25T13:58:09.208087+0000","last_undegraded":"2024-04-25T13:58:09.208087+0000","last_fullsized":"2024-04-25T13:58:09.208087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:45.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212860+0000","last_change":"2024-04-25T13:57:12.157016+0000","last_active":"2024-04-25T13:58:09.212860+0000","last_peered":"2024-04-25T13:58:09.212860+0000","last_clean":"2024-04-25T13:58:09.212860+0000","last_became_active":"2024-04-25T13:57:12.156209+0000","last_became_peered":"2024-04-25T13:57:12.156209+0000","last_unstale":"2024-04-25T13:58:09.212860+0000","last_undegraded":"2024-04-25T13:58:09.212860+0000","last_fullsized":"2024-04-25T13:58:09.212860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325937+0000","last_change":"2024-04-25T13:57:12.146696+0000","last_active":"2024-04-25T13:58:09.325937+0000","last_peered":"2024-04-25T13:58:09.325937+0000","last_clean":"2024-04-25T13:58:09.325937+0000","last_became_active":"2024-04-25T13:57:12.146317+0000","last_became_peered":"2024-04-25T13:57:12.146317+0000","last_unstale":"2024-04-25T13:58:09.325937+0000","last_undegraded":"2024-04-25T13:58:09.325937+0000","last_fullsized":"2024-04-25T13:58:09.325937+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.078941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325705+0000","last_change":"2024-04-25T13:57:12.155100+0000","last_active":"2024-04-25T13:58:09.325705+0000","last_peered":"2024-04-25T13:58:09.325705+0000","last_clean":"2024-04-25T13:58:09.325705+0000","last_became_active":"2024-04-25T13:57:12.154602+0000","last_became_peered":"2024-04-25T13:57:12.154602+0000","last_unstale":"2024-04-25T13:58:09.325705+0000","last_undegraded":"2024-04-25T13:58:09.325705+0000","last_fullsized":"2024-04-25T13:58:09.325705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.837117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208824+0000","last_change":"2024-04-25T13:57:12.149386+0000","last_active":"2024-04-25T13:58:09.208824+0000","last_peered":"2024-04-25T13:58:09.208824+0000","last_clean":"2024-04-25T13:58:09.208824+0000","last_became_active":"2024-04-25T13:57:12.148710+0000","last_became_peered":"2024-04-25T13:57:12.148710+0000","last_unstale":"2024-04-25T13:58:09.208824+0000","last_undegraded":"2024-04-25T13:58:09.208824+0000","last_fullsized":"2024-04-25T13:58:09.208824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:17.655260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208203+0000","last_change":"2024-04-25T13:57:12.156103+0000","last_active":"2024-04-25T13:58:09.208203+0000","last_peered":"2024-04-25T13:58:09.208203+0000","last_clean":"2024-04-25T13:58:09.208203+0000","last_became_active":"2024-04-25T13:57:12.155738+0000","last_became_peered":"2024-04-25T13:57:12.155738+0000","last_unstale":"2024-04-25T13:58:09.208203+0000","last_undegraded":"2024-04-25T13:58:09.208203+0000","last_fullsized":"2024-04-25T13:58:09.208203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:38.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210430+0000","last_change":"2024-04-25T13:57:12.154873+0000","last_active":"2024-04-25T13:58:09.210430+0000","last_peered":"2024-04-25T13:58:09.210430+0000","last_clean":"2024-04-25T13:58:09.210430+0000","last_became_active":"2024-04-25T13:57:12.154523+0000","last_became_peered":"2024-04-25T13:57:12.154523+0000","last_unstale":"2024-04-25T13:58:09.210430+0000","last_undegraded":"2024-04-25T13:58:09.210430+0000","last_fullsized":"2024-04-25T13:58:09.210430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:28.376005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210007+0000","last_change":"2024-04-25T13:57:12.156536+0000","last_active":"2024-04-25T13:58:09.210007+0000","last_peered":"2024-04-25T13:58:09.210007+0000","last_clean":"2024-04-25T13:58:09.210007+0000","last_became_active":"2024-04-25T13:57:12.156143+0000","last_became_peered":"2024-04-25T13:57:12.156143+0000","last_unstale":"2024-04-25T13:58:09.210007+0000","last_undegraded":"2024-04-25T13:58:09.210007+0000","last_fullsized":"2024-04-25T13:58:09.210007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:35.210059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209472+0000","last_change":"2024-04-25T13:57:12.144477+0000","last_active":"2024-04-25T13:58:09.209472+0000","last_peered":"2024-04-25T13:58:09.209472+0000","last_clean":"2024-04-25T13:58:09.209472+0000","last_became_active":"2024-04-25T13:57:12.144115+0000","last_became_peered":"2024-04-25T13:57:12.144115+0000","last_unstale":"2024-04-25T13:58:09.209472+0000","last_undegraded":"2024-04-25T13:58:09.209472+0000","last_fullsized":"2024-04-25T13:58:09.209472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:30.733713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324890+0000","last_change":"2024-04-25T13:57:12.148837+0000","last_active":"2024-04-25T13:58:09.324890+0000","last_peered":"2024-04-25T13:58:09.324890+0000","last_clean":"2024-04-25T13:58:09.324890+0000","last_became_active":"2024-04-25T13:57:12.148494+0000","last_became_peered":"2024-04-25T13:57:12.148494+0000","last_unstale":"2024-04-25T13:58:09.324890+0000","last_undegraded":"2024-04-25T13:58:09.324890+0000","last_fullsized":"2024-04-25T13:58:09.324890+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:50.414344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209622+0000","last_change":"2024-04-25T13:57:12.148142+0000","last_active":"2024-04-25T13:58:09.209622+0000","last_peered":"2024-04-25T13:58:09.209622+0000","last_clean":"2024-04-25T13:58:09.209622+0000","last_became_active":"2024-04-25T13:57:12.147738+0000","last_became_peered":"2024-04-25T13:57:12.147738+0000","last_unstale":"2024-04-25T13:58:09.209622+0000","last_undegraded":"2024-04-25T13:58:09.209622+0000","last_fullsized":"2024-04-25T13:58:09.209622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:42.014605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.206009+0000","last_change":"2024-04-25T13:57:12.160059+0000","last_active":"2024-04-25T13:58:09.206009+0000","last_peered":"2024-04-25T13:58:09.206009+0000","last_clean":"2024-04-25T13:58:09.206009+0000","last_became_active":"2024-04-25T13:57:12.159685+0000","last_became_peered":"2024-04-25T13:57:12.159685+0000","last_unstale":"2024-04-25T13:58:09.206009+0000","last_undegraded":"2024-04-25T13:58:09.206009+0000","last_fullsized":"2024-04-25T13:58:09.206009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:48.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216794+0000","last_change":"2024-04-25T13:57:12.157950+0000","last_active":"2024-04-25T13:58:09.216794+0000","last_peered":"2024-04-25T13:58:09.216794+0000","last_clean":"2024-04-25T13:58:09.216794+0000","last_became_active":"2024-04-25T13:57:12.157612+0000","last_became_peered":"2024-04-25T13:57:12.157612+0000","last_unstale":"2024-04-25T13:58:09.216794+0000","last_undegraded":"2024-04-25T13:58:09.216794+0000","last_fullsized":"2024-04-25T13:58:09.216794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:55.521381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.222728+0000","last_change":"2024-04-25T13:58:09.222728+0000","last_active":"2024-04-25T13:58:09.222728+0000","last_peered":"2024-04-25T13:58:09.222728+0000","last_clean":"2024-04-25T13:58:09.222728+0000","last_became_active":"2024-04-25T13:58:09.210819+0000","last_became_peered":"2024-04-25T13:58:09.210819+0000","last_unstale":"2024-04-25T13:58:09.222728+0000","last_undegraded":"2024-04-25T13:58:09.222728+0000","last_fullsized":"2024-04-25T13:58:09.222728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:00.606838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212961+0000","last_change":"2024-04-25T13:57:12.151301+0000","last_active":"2024-04-25T13:58:09.212961+0000","last_peered":"2024-04-25T13:58:09.212961+0000","last_clean":"2024-04-25T13:58:09.212961+0000","last_became_active":"2024-04-25T13:57:12.150941+0000","last_became_peered":"2024-04-25T13:57:12.150941+0000","last_unstale":"2024-04-25T13:58:09.212961+0000","last_undegraded":"2024-04-25T13:58:09.212961+0000","last_fullsized":"2024-04-25T13:58:09.212961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:22.259683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213261+0000","last_change":"2024-04-25T13:57:11.147314+0000","last_active":"2024-04-25T13:58:09.213261+0000","last_peered":"2024-04-25T13:58:09.213261+0000","last_clean":"2024-04-25T13:58:09.213261+0000","last_became_active":"2024-04-25T13:57:11.146894+0000","last_became_peered":"2024-04-25T13:57:11.146894+0000","last_unstale":"2024-04-25T13:58:09.213261+0000","last_undegraded":"2024-04-25T13:58:09.213261+0000","last_fullsized":"2024-04-25T13:58:09.213261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:55.927007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210152+0000","last_change":"2024-04-25T13:57:12.149186+0000","last_active":"2024-04-25T13:58:09.210152+0000","last_peered":"2024-04-25T13:58:09.210152+0000","last_clean":"2024-04-25T13:58:09.210152+0000","last_became_active":"2024-04-25T13:57:12.148155+0000","last_became_peered":"2024-04-25T13:57:12.148155+0000","last_unstale":"2024-04-25T13:58:09.210152+0000","last_undegraded":"2024-04-25T13:58:09.210152+0000","last_fullsized":"2024-04-25T13:58:09.210152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209681+0000","last_change":"2024-04-25T13:57:11.140086+0000","last_active":"2024-04-25T13:58:09.209681+0000","last_peered":"2024-04-25T13:58:09.209681+0000","last_clean":"2024-04-25T13:58:09.209681+0000","last_became_active":"2024-04-25T13:57:11.139751+0000","last_became_peered":"2024-04-25T13:57:11.139751+0000","last_unstale":"2024-04-25T13:58:09.209681+0000","last_undegraded":"2024-04-25T13:58:09.209681+0000","last_fullsized":"2024-04-25T13:58:09.209681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.552503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325309+0000","last_change":"2024-04-25T13:57:12.157029+0000","last_active":"2024-04-25T13:58:09.325309+0000","last_peered":"2024-04-25T13:58:09.325309+0000","last_clean":"2024-04-25T13:58:09.325309+0000","last_became_active":"2024-04-25T13:57:12.156668+0000","last_became_peered":"2024-04-25T13:57:12.156668+0000","last_unstale":"2024-04-25T13:58:09.325309+0000","last_undegraded":"2024-04-25T13:58:09.325309+0000","last_fullsized":"2024-04-25T13:58:09.325309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208828+0000","last_change":"2024-04-25T13:57:11.140592+0000","last_active":"2024-04-25T13:58:09.208828+0000","last_peered":"2024-04-25T13:58:09.208828+0000","last_clean":"2024-04-25T13:58:09.208828+0000","last_became_active":"2024-04-25T13:57:11.140244+0000","last_became_peered":"2024-04-25T13:57:11.140244+0000","last_unstale":"2024-04-25T13:58:09.208828+0000","last_undegraded":"2024-04-25T13:58:09.208828+0000","last_fullsized":"2024-04-25T13:58:09.208828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:53.071092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.326770+0000","last_change":"2024-04-25T13:58:09.326284+0000","last_active":"2024-04-25T13:58:09.326770+0000","last_peered":"2024-04-25T13:58:09.326770+0000","last_clean":"2024-04-25T13:58:09.326770+0000","last_became_active":"2024-04-25T13:58:09.325888+0000","last_became_peered":"2024-04-25T13:58:09.325888+0000","last_unstale":"2024-04-25T13:58:09.326770+0000","last_undegraded":"2024-04-25T13:58:09.326770+0000","last_fullsized":"2024-04-25T13:58:09.326770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:39.151793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":113,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:26.245669+0000","last_change":"2024-04-25T13:58:09.221709+0000","last_active":"2024-04-25T13:58:26.245669+0000","last_peered":"2024-04-25T13:58:26.245669+0000","last_clean":"2024-04-25T13:58:26.245669+0000","last_became_active":"2024-04-25T13:58:09.215262+0000","last_became_peered":"2024-04-25T13:58:09.215262+0000","last_unstale":"2024-04-25T13:58:26.245669+0000","last_undegraded":"2024-04-25T13:58:26.245669+0000","last_fullsized":"2024-04-25T13:58:26.245669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:05.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208574+0000","last_change":"2024-04-25T13:57:12.148787+0000","last_active":"2024-04-25T13:58:09.208574+0000","last_peered":"2024-04-25T13:58:09.208574+0000","last_clean":"2024-04-25T13:58:09.208574+0000","last_became_active":"2024-04-25T13:57:12.148424+0000","last_became_peered":"2024-04-25T13:57:12.148424+0000","last_unstale":"2024-04-25T13:58:09.208574+0000","last_undegraded":"2024-04-25T13:58:09.208574+0000","last_fullsized":"2024-04-25T13:58:09.208574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:11.921873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324717+0000","last_change":"2024-04-25T13:57:11.142424+0000","last_active":"2024-04-25T13:58:09.324717+0000","last_peered":"2024-04-25T13:58:09.324717+0000","last_clean":"2024-04-25T13:58:09.324717+0000","last_became_active":"2024-04-25T13:57:11.142047+0000","last_became_peered":"2024-04-25T13:57:11.142047+0000","last_unstale":"2024-04-25T13:58:09.324717+0000","last_undegraded":"2024-04-25T13:58:09.324717+0000","last_fullsized":"2024-04-25T13:58:09.324717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:13.455149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214205+0000","last_change":"2024-04-25T13:57:12.155940+0000","last_active":"2024-04-25T13:58:09.214205+0000","last_peered":"2024-04-25T13:58:09.214205+0000","last_clean":"2024-04-25T13:58:09.214205+0000","last_became_active":"2024-04-25T13:57:12.155503+0000","last_became_peered":"2024-04-25T13:57:12.155503+0000","last_unstale":"2024-04-25T13:58:09.214205+0000","last_undegraded":"2024-04-25T13:58:09.214205+0000","last_fullsized":"2024-04-25T13:58:09.214205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:48.345437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209853+0000","last_change":"2024-04-25T13:57:11.131160+0000","last_active":"2024-04-25T13:58:09.209853+0000","last_peered":"2024-04-25T13:58:09.209853+0000","last_clean":"2024-04-25T13:58:09.209853+0000","last_became_active":"2024-04-25T13:57:11.130753+0000","last_became_peered":"2024-04-25T13:57:11.130753+0000","last_unstale":"2024-04-25T13:58:09.209853+0000","last_undegraded":"2024-04-25T13:58:09.209853+0000","last_fullsized":"2024-04-25T13:58:09.209853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:48.297546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212990+0000","last_change":"2024-04-25T13:57:12.155779+0000","last_active":"2024-04-25T13:58:09.212990+0000","last_peered":"2024-04-25T13:58:09.212990+0000","last_clean":"2024-04-25T13:58:09.212990+0000","last_became_active":"2024-04-25T13:57:12.154682+0000","last_became_peered":"2024-04-25T13:57:12.154682+0000","last_unstale":"2024-04-25T13:58:09.212990+0000","last_undegraded":"2024-04-25T13:58:09.212990+0000","last_fullsized":"2024-04-25T13:58:09.212990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:20.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211839+0000","last_change":"2024-04-25T13:57:11.140211+0000","last_active":"2024-04-25T13:58:09.211839+0000","last_peered":"2024-04-25T13:58:09.211839+0000","last_clean":"2024-04-25T13:58:09.211839+0000","last_became_active":"2024-04-25T13:57:11.139891+0000","last_became_peered":"2024-04-25T13:57:11.139891+0000","last_unstale":"2024-04-25T13:58:09.211839+0000","last_undegraded":"2024-04-25T13:58:09.211839+0000","last_fullsized":"2024-04-25T13:58:09.211839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:21.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.205883+0000","last_change":"2024-04-25T13:57:12.153886+0000","last_active":"2024-04-25T13:58:09.205883+0000","last_peered":"2024-04-25T13:58:09.205883+0000","last_clean":"2024-04-25T13:58:09.205883+0000","last_became_active":"2024-04-25T13:57:12.153500+0000","last_became_peered":"2024-04-25T13:57:12.153500+0000","last_unstale":"2024-04-25T13:58:09.205883+0000","last_undegraded":"2024-04-25T13:58:09.205883+0000","last_fullsized":"2024-04-25T13:58:09.205883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:53.549745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'3","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:24.241631+0000","last_change":"2024-04-25T13:57:11.138829+0000","last_active":"2024-04-25T13:58:24.241631+0000","last_peered":"2024-04-25T13:58:24.241631+0000","last_clean":"2024-04-25T13:58:24.241631+0000","last_became_active":"2024-04-25T13:57:11.138461+0000","last_became_peered":"2024-04-25T13:57:11.138461+0000","last_unstale":"2024-04-25T13:58:24.241631+0000","last_undegraded":"2024-04-25T13:58:24.241631+0000","last_fullsized":"2024-04-25T13:58:24.241631+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.134061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325531+0000","last_change":"2024-04-25T13:57:12.157394+0000","last_active":"2024-04-25T13:58:09.325531+0000","last_peered":"2024-04-25T13:58:09.325531+0000","last_clean":"2024-04-25T13:58:09.325531+0000","last_became_active":"2024-04-25T13:57:12.156408+0000","last_became_peered":"2024-04-25T13:57:12.156408+0000","last_unstale":"2024-04-25T13:58:09.325531+0000","last_undegraded":"2024-04-25T13:58:09.325531+0000","last_fullsized":"2024-04-25T13:58:09.325531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:36.741367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208928+0000","last_change":"2024-04-25T13:57:11.142846+0000","last_active":"2024-04-25T13:58:09.208928+0000","last_peered":"2024-04-25T13:58:09.208928+0000","last_clean":"2024-04-25T13:58:09.208928+0000","last_became_active":"2024-04-25T13:57:11.142387+0000","last_became_peered":"2024-04-25T13:57:11.142387+0000","last_unstale":"2024-04-25T13:58:09.208928+0000","last_undegraded":"2024-04-25T13:58:09.208928+0000","last_fullsized":"2024-04-25T13:58:09.208928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.164994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210009+0000","last_change":"2024-04-25T13:57:12.145617+0000","last_active":"2024-04-25T13:58:09.210009+0000","last_peered":"2024-04-25T13:58:09.210009+0000","last_clean":"2024-04-25T13:58:09.210009+0000","last_became_active":"2024-04-25T13:57:12.145263+0000","last_became_peered":"2024-04-25T13:57:12.145263+0000","last_unstale":"2024-04-25T13:58:09.210009+0000","last_undegraded":"2024-04-25T13:58:09.210009+0000","last_fullsized":"2024-04-25T13:58:09.210009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.886538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212101+0000","last_change":"2024-04-25T13:57:11.136972+0000","last_active":"2024-04-25T13:58:09.212101+0000","last_peered":"2024-04-25T13:58:09.212101+0000","last_clean":"2024-04-25T13:58:09.212101+0000","last_became_active":"2024-04-25T13:57:11.136392+0000","last_became_peered":"2024-04-25T13:57:11.136392+0000","last_unstale":"2024-04-25T13:58:09.212101+0000","last_undegraded":"2024-04-25T13:58:09.212101+0000","last_fullsized":"2024-04-25T13:58:09.212101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:45.841670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325608+0000","last_change":"2024-04-25T13:57:12.144905+0000","last_active":"2024-04-25T13:58:09.325608+0000","last_peered":"2024-04-25T13:58:09.325608+0000","last_clean":"2024-04-25T13:58:09.325608+0000","last_became_active":"2024-04-25T13:57:12.144404+0000","last_became_peered":"2024-04-25T13:57:12.144404+0000","last_unstale":"2024-04-25T13:58:09.325608+0000","last_undegraded":"2024-04-25T13:58:09.325608+0000","last_fullsized":"2024-04-25T13:58:09.325608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.186492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'28","reported_seq":197,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:25.238728+0000","last_change":"2024-04-25T13:57:11.139041+0000","last_active":"2024-04-25T13:58:25.238728+0000","last_peered":"2024-04-25T13:58:25.238728+0000","last_clean":"2024-04-25T13:58:25.238728+0000","last_became_active":"2024-04-25T13:57:11.138629+0000","last_became_peered":"2024-04-25T13:57:11.138629+0000","last_unstale":"2024-04-25T13:58:25.238728+0000","last_undegraded":"2024-04-25T13:58:25.238728+0000","last_fullsized":"2024-04-25T13:58:25.238728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:04.863015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":150,"num_read_kb":134,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209260+0000","last_change":"2024-04-25T13:57:12.160897+0000","last_active":"2024-04-25T13:58:09.209260+0000","last_peered":"2024-04-25T13:58:09.209260+0000","last_clean":"2024-04-25T13:58:09.209260+0000","last_became_active":"2024-04-25T13:57:12.160322+0000","last_became_peered":"2024-04-25T13:57:12.160322+0000","last_unstale":"2024-04-25T13:58:09.209260+0000","last_undegraded":"2024-04-25T13:58:09.209260+0000","last_fullsized":"2024-04-25T13:58:09.209260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:26.180324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216914+0000","last_change":"2024-04-25T13:57:11.137104+0000","last_active":"2024-04-25T13:58:09.216914+0000","last_peered":"2024-04-25T13:58:09.216914+0000","last_clean":"2024-04-25T13:58:09.216914+0000","last_became_active":"2024-04-25T13:57:11.136720+0000","last_became_peered":"2024-04-25T13:57:11.136720+0000","last_unstale":"2024-04-25T13:58:09.216914+0000","last_undegraded":"2024-04-25T13:58:09.216914+0000","last_fullsized":"2024-04-25T13:58:09.216914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:41.262776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210048+0000","last_change":"2024-04-25T13:57:12.147831+0000","last_active":"2024-04-25T13:58:09.210048+0000","last_peered":"2024-04-25T13:58:09.210048+0000","last_clean":"2024-04-25T13:58:09.210048+0000","last_became_active":"2024-04-25T13:57:12.147270+0000","last_became_peered":"2024-04-25T13:57:12.147270+0000","last_unstale":"2024-04-25T13:58:09.210048+0000","last_undegraded":"2024-04-25T13:58:09.210048+0000","last_fullsized":"2024-04-25T13:58:09.210048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:31.574661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211663+0000","last_change":"2024-04-25T13:57:11.138254+0000","last_active":"2024-04-25T13:58:09.211663+0000","last_peered":"2024-04-25T13:58:09.211663+0000","last_clean":"2024-04-25T13:58:09.211663+0000","last_became_active":"2024-04-25T13:57:11.137924+0000","last_became_peered":"2024-04-25T13:57:11.137924+0000","last_unstale":"2024-04-25T13:58:09.211663+0000","last_undegraded":"2024-04-25T13:58:09.211663+0000","last_fullsized":"2024-04-25T13:58:09.211663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:27.746203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208758+0000","last_change":"2024-04-25T13:57:12.145743+0000","last_active":"2024-04-25T13:58:09.208758+0000","last_peered":"2024-04-25T13:58:09.208758+0000","last_clean":"2024-04-25T13:58:09.208758+0000","last_became_active":"2024-04-25T13:57:12.145337+0000","last_became_peered":"2024-04-25T13:57:12.145337+0000","last_unstale":"2024-04-25T13:58:09.208758+0000","last_undegraded":"2024-04-25T13:58:09.208758+0000","last_fullsized":"2024-04-25T13:58:09.208758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.692900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'4","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:25.198125+0000","last_change":"2024-04-25T13:57:11.138059+0000","last_active":"2024-04-25T13:58:25.198125+0000","last_peered":"2024-04-25T13:58:25.198125+0000","last_clean":"2024-04-25T13:58:25.198125+0000","last_became_active":"2024-04-25T13:57:11.137698+0000","last_became_peered":"2024-04-25T13:57:11.137698+0000","last_unstale":"2024-04-25T13:58:25.198125+0000","last_undegraded":"2024-04-25T13:58:25.198125+0000","last_fullsized":"2024-04-25T13:58:25.198125+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:03.595126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208160+0000","last_change":"2024-04-25T13:57:12.155507+0000","last_active":"2024-04-25T13:58:09.208160+0000","last_peered":"2024-04-25T13:58:09.208160+0000","last_clean":"2024-04-25T13:58:09.208160+0000","last_became_active":"2024-04-25T13:57:12.155175+0000","last_became_peered":"2024-04-25T13:57:12.155175+0000","last_unstale":"2024-04-25T13:58:09.208160+0000","last_undegraded":"2024-04-25T13:58:09.208160+0000","last_fullsized":"2024-04-25T13:58:09.208160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:21.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'15","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208247+0000","last_change":"2024-04-25T13:57:11.141672+0000","last_active":"2024-04-25T13:58:09.208247+0000","last_peered":"2024-04-25T13:58:09.208247+0000","last_clean":"2024-04-25T13:58:09.208247+0000","last_became_active":"2024-04-25T13:57:11.140957+0000","last_became_peered":"2024-04-25T13:57:11.140957+0000","last_unstale":"2024-04-25T13:58:09.208247+0000","last_undegraded":"2024-04-25T13:58:09.208247+0000","last_fullsized":"2024-04-25T13:58:09.208247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:47.362949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211503+0000","last_change":"2024-04-25T13:57:12.147881+0000","last_active":"2024-04-25T13:58:09.211503+0000","last_peered":"2024-04-25T13:58:09.211503+0000","last_clean":"2024-04-25T13:58:09.211503+0000","last_became_active":"2024-04-25T13:57:12.146200+0000","last_became_peered":"2024-04-25T13:57:12.146200+0000","last_unstale":"2024-04-25T13:58:09.211503+0000","last_undegraded":"2024-04-25T13:58:09.211503+0000","last_fullsized":"2024-04-25T13:58:09.211503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:51.711045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209516+0000","last_change":"2024-04-25T13:57:11.139574+0000","last_active":"2024-04-25T13:58:09.209516+0000","last_peered":"2024-04-25T13:58:09.209516+0000","last_clean":"2024-04-25T13:58:09.209516+0000","last_became_active":"2024-04-25T13:57:11.139232+0000","last_became_peered":"2024-04-25T13:57:11.139232+0000","last_unstale":"2024-04-25T13:58:09.209516+0000","last_undegraded":"2024-04-25T13:58:09.209516+0000","last_fullsized":"2024-04-25T13:58:09.209516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:08.590270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209697+0000","last_change":"2024-04-25T13:57:12.147122+0000","last_active":"2024-04-25T13:58:09.209697+0000","last_peered":"2024-04-25T13:58:09.209697+0000","last_clean":"2024-04-25T13:58:09.209697+0000","last_became_active":"2024-04-25T13:57:12.146708+0000","last_became_peered":"2024-04-25T13:57:12.146708+0000","last_unstale":"2024-04-25T13:58:09.209697+0000","last_undegraded":"2024-04-25T13:58:09.209697+0000","last_fullsized":"2024-04-25T13:58:09.209697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:47.858290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.205903+0000","last_change":"2024-04-25T13:57:11.146288+0000","last_active":"2024-04-25T13:58:09.205903+0000","last_peered":"2024-04-25T13:58:09.205903+0000","last_clean":"2024-04-25T13:58:09.205903+0000","last_became_active":"2024-04-25T13:57:11.145851+0000","last_became_peered":"2024-04-25T13:57:11.145851+0000","last_unstale":"2024-04-25T13:58:09.205903+0000","last_undegraded":"2024-04-25T13:58:09.205903+0000","last_fullsized":"2024-04-25T13:58:09.205903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325156+0000","last_change":"2024-04-25T13:57:12.154576+0000","last_active":"2024-04-25T13:58:09.325156+0000","last_peered":"2024-04-25T13:58:09.325156+0000","last_clean":"2024-04-25T13:58:09.325156+0000","last_became_active":"2024-04-25T13:57:12.154254+0000","last_became_peered":"2024-04-25T13:57:12.154254+0000","last_unstale":"2024-04-25T13:58:09.325156+0000","last_undegraded":"2024-04-25T13:58:09.325156+0000","last_fullsized":"2024-04-25T13:58:09.325156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:33.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208334+0000","last_change":"2024-04-25T13:57:11.139941+0000","last_active":"2024-04-25T13:58:09.208334+0000","last_peered":"2024-04-25T13:58:09.208334+0000","last_clean":"2024-04-25T13:58:09.208334+0000","last_became_active":"2024-04-25T13:57:11.139614+0000","last_became_peered":"2024-04-25T13:57:11.139614+0000","last_unstale":"2024-04-25T13:58:09.208334+0000","last_undegraded":"2024-04-25T13:58:09.208334+0000","last_fullsized":"2024-04-25T13:58:09.208334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:39.656085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325028+0000","last_change":"2024-04-25T13:57:12.155935+0000","last_active":"2024-04-25T13:58:09.325028+0000","last_peered":"2024-04-25T13:58:09.325028+0000","last_clean":"2024-04-25T13:58:09.325028+0000","last_became_active":"2024-04-25T13:57:12.155261+0000","last_became_peered":"2024-04-25T13:57:12.155261+0000","last_unstale":"2024-04-25T13:58:09.325028+0000","last_undegraded":"2024-04-25T13:58:09.325028+0000","last_fullsized":"2024-04-25T13:58:09.325028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:54.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217792+0000","last_change":"2024-04-25T13:58:09.217792+0000","last_active":"2024-04-25T13:58:09.217792+0000","last_peered":"2024-04-25T13:58:09.217792+0000","last_clean":"2024-04-25T13:58:09.217792+0000","last_became_active":"2024-04-25T13:58:09.217386+0000","last_became_peered":"2024-04-25T13:58:09.217386+0000","last_unstale":"2024-04-25T13:58:09.217792+0000","last_undegraded":"2024-04-25T13:58:09.217792+0000","last_fullsized":"2024-04-25T13:58:09.217792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:13.833587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'4","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:26.198771+0000","last_change":"2024-04-25T13:57:11.135226+0000","last_active":"2024-04-25T13:58:26.198771+0000","last_peered":"2024-04-25T13:58:26.198771+0000","last_clean":"2024-04-25T13:58:26.198771+0000","last_became_active":"2024-04-25T13:57:11.134886+0000","last_became_peered":"2024-04-25T13:57:11.134886+0000","last_unstale":"2024-04-25T13:58:26.198771+0000","last_undegraded":"2024-04-25T13:58:26.198771+0000","last_fullsized":"2024-04-25T13:58:26.198771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:00.153597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213327+0000","last_change":"2024-04-25T13:57:12.149758+0000","last_active":"2024-04-25T13:58:09.213327+0000","last_peered":"2024-04-25T13:58:09.213327+0000","last_clean":"2024-04-25T13:58:09.213327+0000","last_became_active":"2024-04-25T13:57:12.149428+0000","last_became_peered":"2024-04-25T13:57:12.149428+0000","last_unstale":"2024-04-25T13:58:09.213327+0000","last_undegraded":"2024-04-25T13:58:09.213327+0000","last_fullsized":"2024-04-25T13:58:09.213327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.739074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207722+0000","last_change":"2024-04-25T13:57:11.135968+0000","last_active":"2024-04-25T13:58:09.207722+0000","last_peered":"2024-04-25T13:58:09.207722+0000","last_clean":"2024-04-25T13:58:09.207722+0000","last_became_active":"2024-04-25T13:57:11.135577+0000","last_became_peered":"2024-04-25T13:57:11.135577+0000","last_unstale":"2024-04-25T13:58:09.207722+0000","last_undegraded":"2024-04-25T13:58:09.207722+0000","last_fullsized":"2024-04-25T13:58:09.207722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:00.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208081+0000","last_change":"2024-04-25T13:55:57.666399+0000","last_active":"2024-04-25T13:58:09.208081+0000","last_peered":"2024-04-25T13:58:09.208081+0000","last_clean":"2024-04-25T13:58:09.208081+0000","last_became_active":"2024-04-25T13:55:57.660470+0000","last_became_peered":"2024-04-25T13:55:57.660470+0000","last_unstale":"2024-04-25T13:58:09.208081+0000","last_undegraded":"2024-04-25T13:58:09.208081+0000","last_fullsized":"2024-04-25T13:58:09.208081+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_clean_scrub_stamp":"2024-04-25T13:52:33.081441+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:22.788632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209521+0000","last_change":"2024-04-25T13:57:12.141923+0000","last_active":"2024-04-25T13:58:09.209521+0000","last_peered":"2024-04-25T13:58:09.209521+0000","last_clean":"2024-04-25T13:58:09.209521+0000","last_became_active":"2024-04-25T13:57:12.140946+0000","last_became_peered":"2024-04-25T13:57:12.140946+0000","last_unstale":"2024-04-25T13:58:09.209521+0000","last_undegraded":"2024-04-25T13:58:09.209521+0000","last_fullsized":"2024-04-25T13:58:09.209521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:49.294598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208387+0000","last_change":"2024-04-25T13:57:11.139851+0000","last_active":"2024-04-25T13:58:09.208387+0000","last_peered":"2024-04-25T13:58:09.208387+0000","last_clean":"2024-04-25T13:58:09.208387+0000","last_became_active":"2024-04-25T13:57:11.139476+0000","last_became_peered":"2024-04-25T13:57:11.139476+0000","last_unstale":"2024-04-25T13:58:09.208387+0000","last_undegraded":"2024-04-25T13:58:09.208387+0000","last_fullsized":"2024-04-25T13:58:09.208387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:29.435917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.220107+0000","last_change":"2024-04-25T13:58:09.219619+0000","last_active":"2024-04-25T13:58:09.220107+0000","last_peered":"2024-04-25T13:58:09.220107+0000","last_clean":"2024-04-25T13:58:09.220107+0000","last_became_active":"2024-04-25T13:58:09.211808+0000","last_became_peered":"2024-04-25T13:58:09.211808+0000","last_unstale":"2024-04-25T13:58:09.220107+0000","last_undegraded":"2024-04-25T13:58:09.220107+0000","last_fullsized":"2024-04-25T13:58:09.220107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:20.208191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210754+0000","last_change":"2024-04-25T13:57:12.136204+0000","last_active":"2024-04-25T13:58:09.210754+0000","last_peered":"2024-04-25T13:58:09.210754+0000","last_clean":"2024-04-25T13:58:09.210754+0000","last_became_active":"2024-04-25T13:57:12.135798+0000","last_became_peered":"2024-04-25T13:57:12.135798+0000","last_unstale":"2024-04-25T13:58:09.210754+0000","last_undegraded":"2024-04-25T13:58:09.210754+0000","last_fullsized":"2024-04-25T13:58:09.210754+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:36.894774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.206095+0000","last_change":"2024-04-25T13:57:11.138699+0000","last_active":"2024-04-25T13:58:09.206095+0000","last_peered":"2024-04-25T13:58:09.206095+0000","last_clean":"2024-04-25T13:58:09.206095+0000","last_became_active":"2024-04-25T13:57:11.138335+0000","last_became_peered":"2024-04-25T13:57:11.138335+0000","last_unstale":"2024-04-25T13:58:09.206095+0000","last_undegraded":"2024-04-25T13:58:09.206095+0000","last_fullsized":"2024-04-25T13:58:09.206095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:44.413228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214134+0000","last_change":"2024-04-25T13:57:12.155851+0000","last_active":"2024-04-25T13:58:09.214134+0000","last_peered":"2024-04-25T13:58:09.214134+0000","last_clean":"2024-04-25T13:58:09.214134+0000","last_became_active":"2024-04-25T13:57:12.155416+0000","last_became_peered":"2024-04-25T13:57:12.155416+0000","last_unstale":"2024-04-25T13:58:09.214134+0000","last_undegraded":"2024-04-25T13:58:09.214134+0000","last_fullsized":"2024-04-25T13:58:09.214134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:22.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207744+0000","last_change":"2024-04-25T13:57:11.147465+0000","last_active":"2024-04-25T13:58:09.207744+0000","last_peered":"2024-04-25T13:58:09.207744+0000","last_clean":"2024-04-25T13:58:09.207744+0000","last_became_active":"2024-04-25T13:57:11.147096+0000","last_became_peered":"2024-04-25T13:57:11.147096+0000","last_unstale":"2024-04-25T13:58:09.207744+0000","last_undegraded":"2024-04-25T13:58:09.207744+0000","last_fullsized":"2024-04-25T13:58:09.207744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.229356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210528+0000","last_change":"2024-04-25T13:57:12.146446+0000","last_active":"2024-04-25T13:58:09.210528+0000","last_peered":"2024-04-25T13:58:09.210528+0000","last_clean":"2024-04-25T13:58:09.210528+0000","last_became_active":"2024-04-25T13:57:12.146129+0000","last_became_peered":"2024-04-25T13:57:12.146129+0000","last_unstale":"2024-04-25T13:58:09.210528+0000","last_undegraded":"2024-04-25T13:58:09.210528+0000","last_fullsized":"2024-04-25T13:58:09.210528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.977663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325014+0000","last_change":"2024-04-25T13:57:11.145127+0000","last_active":"2024-04-25T13:58:09.325014+0000","last_peered":"2024-04-25T13:58:09.325014+0000","last_clean":"2024-04-25T13:58:09.325014+0000","last_became_active":"2024-04-25T13:57:11.144822+0000","last_became_peered":"2024-04-25T13:57:11.144822+0000","last_unstale":"2024-04-25T13:58:09.325014+0000","last_undegraded":"2024-04-25T13:58:09.325014+0000","last_fullsized":"2024-04-25T13:58:09.325014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:00.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208402+0000","last_change":"2024-04-25T13:57:12.156184+0000","last_active":"2024-04-25T13:58:09.208402+0000","last_peered":"2024-04-25T13:58:09.208402+0000","last_clean":"2024-04-25T13:58:09.208402+0000","last_became_active":"2024-04-25T13:57:12.155855+0000","last_became_peered":"2024-04-25T13:57:12.155855+0000","last_unstale":"2024-04-25T13:58:09.208402+0000","last_undegraded":"2024-04-25T13:58:09.208402+0000","last_fullsized":"2024-04-25T13:58:09.208402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213952+0000","last_change":"2024-04-25T13:57:11.137616+0000","last_active":"2024-04-25T13:58:09.213952+0000","last_peered":"2024-04-25T13:58:09.213952+0000","last_clean":"2024-04-25T13:58:09.213952+0000","last_became_active":"2024-04-25T13:57:11.137041+0000","last_became_peered":"2024-04-25T13:57:11.137041+0000","last_unstale":"2024-04-25T13:58:09.213952+0000","last_undegraded":"2024-04-25T13:58:09.213952+0000","last_fullsized":"2024-04-25T13:58:09.213952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:54.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208531+0000","last_change":"2024-04-25T13:57:12.151223+0000","last_active":"2024-04-25T13:58:09.208531+0000","last_peered":"2024-04-25T13:58:09.208531+0000","last_clean":"2024-04-25T13:58:09.208531+0000","last_became_active":"2024-04-25T13:57:12.150871+0000","last_became_peered":"2024-04-25T13:57:12.150871+0000","last_unstale":"2024-04-25T13:58:09.208531+0000","last_undegraded":"2024-04-25T13:58:09.208531+0000","last_fullsized":"2024-04-25T13:58:09.208531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:17.957728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216414+0000","last_change":"2024-04-25T13:57:11.136524+0000","last_active":"2024-04-25T13:58:09.216414+0000","last_peered":"2024-04-25T13:58:09.216414+0000","last_clean":"2024-04-25T13:58:09.216414+0000","last_became_active":"2024-04-25T13:57:11.136152+0000","last_became_peered":"2024-04-25T13:57:11.136152+0000","last_unstale":"2024-04-25T13:58:09.216414+0000","last_undegraded":"2024-04-25T13:58:09.216414+0000","last_fullsized":"2024-04-25T13:58:09.216414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:23.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214204+0000","last_change":"2024-04-25T13:57:12.157641+0000","last_active":"2024-04-25T13:58:09.214204+0000","last_peered":"2024-04-25T13:58:09.214204+0000","last_clean":"2024-04-25T13:58:09.214204+0000","last_became_active":"2024-04-25T13:57:12.157281+0000","last_became_peered":"2024-04-25T13:57:12.157281+0000","last_unstale":"2024-04-25T13:58:09.214204+0000","last_undegraded":"2024-04-25T13:58:09.214204+0000","last_fullsized":"2024-04-25T13:58:09.214204+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:28.245071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208565+0000","last_change":"2024-04-25T13:57:11.139036+0000","last_active":"2024-04-25T13:58:09.208565+0000","last_peered":"2024-04-25T13:58:09.208565+0000","last_clean":"2024-04-25T13:58:09.208565+0000","last_became_active":"2024-04-25T13:57:11.138681+0000","last_became_peered":"2024-04-25T13:57:11.138681+0000","last_unstale":"2024-04-25T13:58:09.208565+0000","last_undegraded":"2024-04-25T13:58:09.208565+0000","last_fullsized":"2024-04-25T13:58:09.208565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:48.584313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208577+0000","last_change":"2024-04-25T13:57:12.153072+0000","last_active":"2024-04-25T13:58:09.208577+0000","last_peered":"2024-04-25T13:58:09.208577+0000","last_clean":"2024-04-25T13:58:09.208577+0000","last_became_active":"2024-04-25T13:57:12.151556+0000","last_became_peered":"2024-04-25T13:57:12.151556+0000","last_unstale":"2024-04-25T13:58:09.208577+0000","last_undegraded":"2024-04-25T13:58:09.208577+0000","last_fullsized":"2024-04-25T13:58:09.208577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:20.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217008+0000","last_change":"2024-04-25T13:57:11.148823+0000","last_active":"2024-04-25T13:58:09.217008+0000","last_peered":"2024-04-25T13:58:09.217008+0000","last_clean":"2024-04-25T13:58:09.217008+0000","last_became_active":"2024-04-25T13:57:11.148436+0000","last_became_peered":"2024-04-25T13:57:11.148436+0000","last_unstale":"2024-04-25T13:58:09.217008+0000","last_undegraded":"2024-04-25T13:58:09.217008+0000","last_fullsized":"2024-04-25T13:58:09.217008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:28.496184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208931+0000","last_change":"2024-04-25T13:57:12.149082+0000","last_active":"2024-04-25T13:58:09.208931+0000","last_peered":"2024-04-25T13:58:09.208931+0000","last_clean":"2024-04-25T13:58:09.208931+0000","last_became_active":"2024-04-25T13:57:12.148260+0000","last_became_peered":"2024-04-25T13:57:12.148260+0000","last_unstale":"2024-04-25T13:58:09.208931+0000","last_undegraded":"2024-04-25T13:58:09.208931+0000","last_fullsized":"2024-04-25T13:58:09.208931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:49.733114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211832+0000","last_change":"2024-04-25T13:57:11.136736+0000","last_active":"2024-04-25T13:58:09.211832+0000","last_peered":"2024-04-25T13:58:09.211832+0000","last_clean":"2024-04-25T13:58:09.211832+0000","last_became_active":"2024-04-25T13:57:11.135831+0000","last_became_peered":"2024-04-25T13:57:11.135831+0000","last_unstale":"2024-04-25T13:58:09.211832+0000","last_undegraded":"2024-04-25T13:58:09.211832+0000","last_fullsized":"2024-04-25T13:58:09.211832+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:49.654642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208827+0000","last_change":"2024-04-25T13:57:12.149797+0000","last_active":"2024-04-25T13:58:09.208827+0000","last_peered":"2024-04-25T13:58:09.208827+0000","last_clean":"2024-04-25T13:58:09.208827+0000","last_became_active":"2024-04-25T13:57:12.149413+0000","last_became_peered":"2024-04-25T13:57:12.149413+0000","last_unstale":"2024-04-25T13:58:09.208827+0000","last_undegraded":"2024-04-25T13:58:09.208827+0000","last_fullsized":"2024-04-25T13:58:09.208827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:58.517208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.328809+0000","last_change":"2024-04-25T13:58:09.328809+0000","last_active":"2024-04-25T13:58:09.328809+0000","last_peered":"2024-04-25T13:58:09.328809+0000","last_clean":"2024-04-25T13:58:09.328809+0000","last_became_active":"2024-04-25T13:58:09.326068+0000","last_became_peered":"2024-04-25T13:58:09.326068+0000","last_unstale":"2024-04-25T13:58:09.328809+0000","last_undegraded":"2024-04-25T13:58:09.328809+0000","last_fullsized":"2024-04-25T13:58:09.328809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:14.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209112+0000","last_change":"2024-04-25T13:57:12.150247+0000","last_active":"2024-04-25T13:58:09.209112+0000","last_peered":"2024-04-25T13:58:09.209112+0000","last_clean":"2024-04-25T13:58:09.209112+0000","last_became_active":"2024-04-25T13:57:12.149882+0000","last_became_peered":"2024-04-25T13:57:12.149882+0000","last_unstale":"2024-04-25T13:58:09.209112+0000","last_undegraded":"2024-04-25T13:58:09.209112+0000","last_fullsized":"2024-04-25T13:58:09.209112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.624928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208702+0000","last_change":"2024-04-25T13:57:12.154618+0000","last_active":"2024-04-25T13:58:09.208702+0000","last_peered":"2024-04-25T13:58:09.208702+0000","last_clean":"2024-04-25T13:58:09.208702+0000","last_became_active":"2024-04-25T13:57:12.153648+0000","last_became_peered":"2024-04-25T13:57:12.153648+0000","last_unstale":"2024-04-25T13:58:09.208702+0000","last_undegraded":"2024-04-25T13:58:09.208702+0000","last_fullsized":"2024-04-25T13:58:09.208702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:24.010980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216677+0000","last_change":"2024-04-25T13:57:12.143097+0000","last_active":"2024-04-25T13:58:09.216677+0000","last_peered":"2024-04-25T13:58:09.216677+0000","last_clean":"2024-04-25T13:58:09.216677+0000","last_became_active":"2024-04-25T13:57:12.142295+0000","last_became_peered":"2024-04-25T13:57:12.142295+0000","last_unstale":"2024-04-25T13:58:09.216677+0000","last_undegraded":"2024-04-25T13:58:09.216677+0000","last_fullsized":"2024-04-25T13:58:09.216677+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.907643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216299+0000","last_change":"2024-04-25T13:57:12.155659+0000","last_active":"2024-04-25T13:58:09.216299+0000","last_peered":"2024-04-25T13:58:09.216299+0000","last_clean":"2024-04-25T13:58:09.216299+0000","last_became_active":"2024-04-25T13:57:12.155308+0000","last_became_peered":"2024-04-25T13:57:12.155308+0000","last_unstale":"2024-04-25T13:58:09.216299+0000","last_undegraded":"2024-04-25T13:58:09.216299+0000","last_fullsized":"2024-04-25T13:58:09.216299+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:12.361660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212066+0000","last_change":"2024-04-25T13:58:09.212066+0000","last_active":"2024-04-25T13:58:09.212066+0000","last_peered":"2024-04-25T13:58:09.212066+0000","last_clean":"2024-04-25T13:58:09.212066+0000","last_became_active":"2024-04-25T13:58:09.211708+0000","last_became_peered":"2024-04-25T13:58:09.211708+0000","last_unstale":"2024-04-25T13:58:09.212066+0000","last_undegraded":"2024-04-25T13:58:09.212066+0000","last_fullsized":"2024-04-25T13:58:09.212066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:34.585999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210854+0000","last_change":"2024-04-25T13:57:12.142134+0000","last_active":"2024-04-25T13:58:09.210854+0000","last_peered":"2024-04-25T13:58:09.210854+0000","last_clean":"2024-04-25T13:58:09.210854+0000","last_became_active":"2024-04-25T13:57:12.141776+0000","last_became_peered":"2024-04-25T13:57:12.141776+0000","last_unstale":"2024-04-25T13:58:09.210854+0000","last_undegraded":"2024-04-25T13:58:09.210854+0000","last_fullsized":"2024-04-25T13:58:09.210854+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:07.621761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213725+0000","last_change":"2024-04-25T13:58:09.213725+0000","last_active":"2024-04-25T13:58:09.213725+0000","last_peered":"2024-04-25T13:58:09.213725+0000","last_clean":"2024-04-25T13:58:09.213725+0000","last_became_active":"2024-04-25T13:58:09.213229+0000","last_became_peered":"2024-04-25T13:58:09.213229+0000","last_unstale":"2024-04-25T13:58:09.213725+0000","last_undegraded":"2024-04-25T13:58:09.213725+0000","last_fullsized":"2024-04-25T13:58:09.213725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:14.728468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217140+0000","last_change":"2024-04-25T13:57:12.147818+0000","last_active":"2024-04-25T13:58:09.217140+0000","last_peered":"2024-04-25T13:58:09.217140+0000","last_clean":"2024-04-25T13:58:09.217140+0000","last_became_active":"2024-04-25T13:57:12.146764+0000","last_became_peered":"2024-04-25T13:57:12.146764+0000","last_unstale":"2024-04-25T13:58:09.217140+0000","last_undegraded":"2024-04-25T13:58:09.217140+0000","last_fullsized":"2024-04-25T13:58:09.217140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.750115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214031+0000","last_change":"2024-04-25T13:57:12.152591+0000","last_active":"2024-04-25T13:58:09.214031+0000","last_peered":"2024-04-25T13:58:09.214031+0000","last_clean":"2024-04-25T13:58:09.214031+0000","last_became_active":"2024-04-25T13:57:12.152198+0000","last_became_peered":"2024-04-25T13:57:12.152198+0000","last_unstale":"2024-04-25T13:58:09.214031+0000","last_undegraded":"2024-04-25T13:58:09.214031+0000","last_fullsized":"2024-04-25T13:58:09.214031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:05.868774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211990+0000","last_change":"2024-04-25T13:57:12.154924+0000","last_active":"2024-04-25T13:58:09.211990+0000","last_peered":"2024-04-25T13:58:09.211990+0000","last_clean":"2024-04-25T13:58:09.211990+0000","last_became_active":"2024-04-25T13:57:12.154549+0000","last_became_peered":"2024-04-25T13:57:12.154549+0000","last_unstale":"2024-04-25T13:58:09.211990+0000","last_undegraded":"2024-04-25T13:58:09.211990+0000","last_fullsized":"2024-04-25T13:58:09.211990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:49.418436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217391+0000","last_change":"2024-04-25T13:58:09.216933+0000","last_active":"2024-04-25T13:58:09.217391+0000","last_peered":"2024-04-25T13:58:09.217391+0000","last_clean":"2024-04-25T13:58:09.217391+0000","last_became_active":"2024-04-25T13:58:09.216615+0000","last_became_peered":"2024-04-25T13:58:09.216615+0000","last_unstale":"2024-04-25T13:58:09.217391+0000","last_undegraded":"2024-04-25T13:58:09.217391+0000","last_fullsized":"2024-04-25T13:58:09.217391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:25.597023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325474+0000","last_change":"2024-04-25T13:57:12.158672+0000","last_active":"2024-04-25T13:58:09.325474+0000","last_peered":"2024-04-25T13:58:09.325474+0000","last_clean":"2024-04-25T13:58:09.325474+0000","last_became_active":"2024-04-25T13:57:12.158391+0000","last_became_peered":"2024-04-25T13:57:12.158391+0000","last_unstale":"2024-04-25T13:58:09.325474+0000","last_undegraded":"2024-04-25T13:58:09.325474+0000","last_fullsized":"2024-04-25T13:58:09.325474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.369907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210535+0000","last_change":"2024-04-25T13:57:12.139310+0000","last_active":"2024-04-25T13:58:09.210535+0000","last_peered":"2024-04-25T13:58:09.210535+0000","last_clean":"2024-04-25T13:58:09.210535+0000","last_became_active":"2024-04-25T13:57:12.138942+0000","last_became_peered":"2024-04-25T13:57:12.138942+0000","last_unstale":"2024-04-25T13:58:09.210535+0000","last_undegraded":"2024-04-25T13:58:09.210535+0000","last_fullsized":"2024-04-25T13:58:09.210535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:01.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209279+0000","last_change":"2024-04-25T13:57:12.147451+0000","last_active":"2024-04-25T13:58:09.209279+0000","last_peered":"2024-04-25T13:58:09.209279+0000","last_clean":"2024-04-25T13:58:09.209279+0000","last_became_active":"2024-04-25T13:57:12.146746+0000","last_became_peered":"2024-04-25T13:57:12.146746+0000","last_unstale":"2024-04-25T13:58:09.209279+0000","last_undegraded":"2024-04-25T13:58:09.209279+0000","last_fullsized":"2024-04-25T13:58:09.209279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247568,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":454,"num_read_kb":296,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1617,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196314,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710749,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":402153,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906978,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1054467,"data_compressed":43506,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":401785,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266733,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":839680,"data_stored":401683,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462962,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":993444,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626487,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":831488,"data_stored":401174,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":462013,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":402445,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":905216,"data_stored":462741,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":401485,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986241,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":843776,"data_stored":403015,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822716,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":993319,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61774,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63015,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:58:30.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:58:31.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:31 smithi078 ceph-mon[29163]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T13:58:31.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:58:31.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:58:31.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.193 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:31 smithi112 ceph-mon[32617]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T13:58:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:58:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:58:31.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:31 smithi129 ceph-mon[32569]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T13:58:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:31.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:58:31.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:58:31.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:58:32.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:32 smithi078 ceph-mon[29163]: from='client.24746 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:58:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:32 smithi112 ceph-mon[32617]: from='client.24746 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:58:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:32 smithi129 ceph-mon[32569]: from='client.24746 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:58:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 -- 172.21.15.78:0/134519014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 msgr2=0x7f19cc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 --2- 172.21.15.78:0/134519014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc1091d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f19c00099c0 tx=0x7f19c0009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 -- 172.21.15.78:0/134519014 shutdown_connections 2024-04-25T13:58:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 --2- 172.21.15.78:0/134519014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19cc109710 0x7f19cc111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 --2- 172.21.15.78:0/134519014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19cc104490 0x7f19cc1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 --2- 172.21.15.78:0/134519014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc1091d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 -- 172.21.15.78:0/134519014 >> 172.21.15.78:0/134519014 conn(0x7f19cc0ffda0 msgr2=0x7f19cc1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 -- 172.21.15.78:0/134519014 shutdown_connections 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.931+0000 7f19d28aa700 1 -- 172.21.15.78:0/134519014 wait complete. 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 Processor -- start 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 -- start start 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19cc104490 0x7f19cc076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19cc109710 0x7f19cc077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc114410 con 0x7f19cc108dc0 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc114290 con 0x7f19cc109710 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.932+0000 7f19d28aa700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc114590 con 0x7f19cc104490 2024-04-25T13:58:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc076fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49046/0 (socket says 172.21.15.78:49046) 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 -- 172.21.15.78:0/293779157 learned_addr learned my addr 172.21.15.78:0/293779157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19d0e47700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19cc109710 0x7f19cc077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cbfff700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19cc104490 0x7f19cc076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 -- 172.21.15.78:0/293779157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19cc104490 msgr2=0x7f19cc076aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19cc104490 0x7f19cc076aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 -- 172.21.15.78:0/293779157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19cc109710 msgr2=0x7f19cc077520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19cc109710 0x7f19cc077520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 -- 172.21.15.78:0/293779157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc1a9140 con 0x7f19cc108dc0 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cbfff700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19cc104490 0x7f19cc076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19cb7fe700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc076fe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009f50 tx=0x7f19c00037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19c97fa700 1 -- 172.21.15.78:0/293779157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0027070 con 0x7f19cc108dc0 2024-04-25T13:58:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc1a93d0 con 0x7f19cc108dc0 2024-04-25T13:58:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19c97fa700 1 -- 172.21.15.78:0/293779157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19c0004090 con 0x7f19cc108dc0 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19c97fa700 1 -- 172.21.15.78:0/293779157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0022e60 con 0x7f19cc108dc0 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.933+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc1a9880 con 0x7f19cc108dc0 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.934+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19cc051190 con 0x7f19cc108dc0 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.936+0000 7f19c97fa700 1 -- 172.21.15.78:0/293779157 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f19c002a070 con 0x7f19cc108dc0 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.936+0000 7f19c97fa700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b40afc10 0x7f19b40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.936+0000 7f19cbfff700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b40afc10 0x7f19b40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.936+0000 7f19c97fa700 1 -- 172.21.15.78:0/293779157 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f19c00dff00 con 0x7f19cc108dc0 2024-04-25T13:58:32.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.937+0000 7f19cbfff700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b40afc10 0x7f19b40b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19bc007900 tx=0x7f19bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:32.941+0000 7f19c97fa700 1 -- 172.21.15.78:0/293779157 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19c0013070 con 0x7f19cc108dc0 2024-04-25T13:58:33.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.137+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19cc063ad0 con 0x7f19b40afc10 2024-04-25T13:58:33.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.147+0000 7f19c97fa700 1 -- 172.21.15.78:0/293779157 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258980 (secure 0 0 0) 0x7f19cc063ad0 con 0x7f19b40afc10 2024-04-25T13:58:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b40afc10 msgr2=0x7f19b40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b40afc10 0x7f19b40b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19bc007900 tx=0x7f19bc008040 comp rx=0 tx=0).stop 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 msgr2=0x7f19cc076fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc076fe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009f50 tx=0x7f19c00037c0 comp rx=0 tx=0).stop 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 shutdown_connections 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b40afc10 0x7f19b40b2120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19cc109710 0x7f19cc077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:33.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19cc104490 0x7f19cc076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:33.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 --2- 172.21.15.78:0/293779157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19cc108dc0 0x7f19cc076fe0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:33.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 >> 172.21.15.78:0/293779157 conn(0x7f19cc0ffda0 msgr2=0x7f19cc10a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:33.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 shutdown_connections 2024-04-25T13:58:33.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:33.150+0000 7f19d28aa700 1 -- 172.21.15.78:0/293779157 wait complete. 2024-04-25T13:58:33.153 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T13:58:33.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:33 smithi078 ceph-mon[29163]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:33 smithi112 ceph-mon[32617]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:33.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:33 smithi129 ceph-mon[32569]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:33.838 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-04-25T13:58:32.147750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":550,"num_read_kb":370,"num_write":282,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953457,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371912,"kb_used_data":10212,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569912,"statfs":{"total":1151940427776,"available":1151559589888,"internally_reserved":0,"allocated":10457088,"data_stored":6779724,"data_compressed":182799,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003030"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210093+0000","last_change":"2024-04-25T13:57:12.147384+0000","last_active":"2024-04-25T13:58:09.210093+0000","last_peered":"2024-04-25T13:58:09.210093+0000","last_clean":"2024-04-25T13:58:09.210093+0000","last_became_active":"2024-04-25T13:57:12.146349+0000","last_became_peered":"2024-04-25T13:57:12.146349+0000","last_unstale":"2024-04-25T13:58:09.210093+0000","last_undegraded":"2024-04-25T13:58:09.210093+0000","last_fullsized":"2024-04-25T13:58:09.210093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210422+0000","last_change":"2024-04-25T13:57:12.156698+0000","last_active":"2024-04-25T13:58:09.210422+0000","last_peered":"2024-04-25T13:58:09.210422+0000","last_clean":"2024-04-25T13:58:09.210422+0000","last_became_active":"2024-04-25T13:57:12.155790+0000","last_became_peered":"2024-04-25T13:57:12.155790+0000","last_unstale":"2024-04-25T13:58:09.210422+0000","last_undegraded":"2024-04-25T13:58:09.210422+0000","last_fullsized":"2024-04-25T13:58:09.210422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:25.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209785+0000","last_change":"2024-04-25T13:57:12.149458+0000","last_active":"2024-04-25T13:58:09.209785+0000","last_peered":"2024-04-25T13:58:09.209785+0000","last_clean":"2024-04-25T13:58:09.209785+0000","last_became_active":"2024-04-25T13:57:12.148179+0000","last_became_peered":"2024-04-25T13:57:12.148179+0000","last_unstale":"2024-04-25T13:58:09.209785+0000","last_undegraded":"2024-04-25T13:58:09.209785+0000","last_fullsized":"2024-04-25T13:58:09.209785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:00.502798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208220+0000","last_change":"2024-04-25T13:57:12.145776+0000","last_active":"2024-04-25T13:58:09.208220+0000","last_peered":"2024-04-25T13:58:09.208220+0000","last_clean":"2024-04-25T13:58:09.208220+0000","last_became_active":"2024-04-25T13:57:12.145038+0000","last_became_peered":"2024-04-25T13:57:12.145038+0000","last_unstale":"2024-04-25T13:58:09.208220+0000","last_undegraded":"2024-04-25T13:58:09.208220+0000","last_fullsized":"2024-04-25T13:58:09.208220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:52.742746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208411+0000","last_change":"2024-04-25T13:57:12.160273+0000","last_active":"2024-04-25T13:58:09.208411+0000","last_peered":"2024-04-25T13:58:09.208411+0000","last_clean":"2024-04-25T13:58:09.208411+0000","last_became_active":"2024-04-25T13:57:12.159805+0000","last_became_peered":"2024-04-25T13:57:12.159805+0000","last_unstale":"2024-04-25T13:58:09.208411+0000","last_undegraded":"2024-04-25T13:58:09.208411+0000","last_fullsized":"2024-04-25T13:58:09.208411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:15.044610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324782+0000","last_change":"2024-04-25T13:57:12.151597+0000","last_active":"2024-04-25T13:58:09.324782+0000","last_peered":"2024-04-25T13:58:09.324782+0000","last_clean":"2024-04-25T13:58:09.324782+0000","last_became_active":"2024-04-25T13:57:12.151203+0000","last_became_peered":"2024-04-25T13:57:12.151203+0000","last_unstale":"2024-04-25T13:58:09.324782+0000","last_undegraded":"2024-04-25T13:58:09.324782+0000","last_fullsized":"2024-04-25T13:58:09.324782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.645264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208087+0000","last_change":"2024-04-25T13:57:12.144959+0000","last_active":"2024-04-25T13:58:09.208087+0000","last_peered":"2024-04-25T13:58:09.208087+0000","last_clean":"2024-04-25T13:58:09.208087+0000","last_became_active":"2024-04-25T13:57:12.144587+0000","last_became_peered":"2024-04-25T13:57:12.144587+0000","last_unstale":"2024-04-25T13:58:09.208087+0000","last_undegraded":"2024-04-25T13:58:09.208087+0000","last_fullsized":"2024-04-25T13:58:09.208087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:45.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212860+0000","last_change":"2024-04-25T13:57:12.157016+0000","last_active":"2024-04-25T13:58:09.212860+0000","last_peered":"2024-04-25T13:58:09.212860+0000","last_clean":"2024-04-25T13:58:09.212860+0000","last_became_active":"2024-04-25T13:57:12.156209+0000","last_became_peered":"2024-04-25T13:57:12.156209+0000","last_unstale":"2024-04-25T13:58:09.212860+0000","last_undegraded":"2024-04-25T13:58:09.212860+0000","last_fullsized":"2024-04-25T13:58:09.212860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325937+0000","last_change":"2024-04-25T13:57:12.146696+0000","last_active":"2024-04-25T13:58:09.325937+0000","last_peered":"2024-04-25T13:58:09.325937+0000","last_clean":"2024-04-25T13:58:09.325937+0000","last_became_active":"2024-04-25T13:57:12.146317+0000","last_became_peered":"2024-04-25T13:57:12.146317+0000","last_unstale":"2024-04-25T13:58:09.325937+0000","last_undegraded":"2024-04-25T13:58:09.325937+0000","last_fullsized":"2024-04-25T13:58:09.325937+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.078941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325705+0000","last_change":"2024-04-25T13:57:12.155100+0000","last_active":"2024-04-25T13:58:09.325705+0000","last_peered":"2024-04-25T13:58:09.325705+0000","last_clean":"2024-04-25T13:58:09.325705+0000","last_became_active":"2024-04-25T13:57:12.154602+0000","last_became_peered":"2024-04-25T13:57:12.154602+0000","last_unstale":"2024-04-25T13:58:09.325705+0000","last_undegraded":"2024-04-25T13:58:09.325705+0000","last_fullsized":"2024-04-25T13:58:09.325705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.837117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208824+0000","last_change":"2024-04-25T13:57:12.149386+0000","last_active":"2024-04-25T13:58:09.208824+0000","last_peered":"2024-04-25T13:58:09.208824+0000","last_clean":"2024-04-25T13:58:09.208824+0000","last_became_active":"2024-04-25T13:57:12.148710+0000","last_became_peered":"2024-04-25T13:57:12.148710+0000","last_unstale":"2024-04-25T13:58:09.208824+0000","last_undegraded":"2024-04-25T13:58:09.208824+0000","last_fullsized":"2024-04-25T13:58:09.208824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:17.655260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208203+0000","last_change":"2024-04-25T13:57:12.156103+0000","last_active":"2024-04-25T13:58:09.208203+0000","last_peered":"2024-04-25T13:58:09.208203+0000","last_clean":"2024-04-25T13:58:09.208203+0000","last_became_active":"2024-04-25T13:57:12.155738+0000","last_became_peered":"2024-04-25T13:57:12.155738+0000","last_unstale":"2024-04-25T13:58:09.208203+0000","last_undegraded":"2024-04-25T13:58:09.208203+0000","last_fullsized":"2024-04-25T13:58:09.208203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:38.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210430+0000","last_change":"2024-04-25T13:57:12.154873+0000","last_active":"2024-04-25T13:58:09.210430+0000","last_peered":"2024-04-25T13:58:09.210430+0000","last_clean":"2024-04-25T13:58:09.210430+0000","last_became_active":"2024-04-25T13:57:12.154523+0000","last_became_peered":"2024-04-25T13:57:12.154523+0000","last_unstale":"2024-04-25T13:58:09.210430+0000","last_undegraded":"2024-04-25T13:58:09.210430+0000","last_fullsized":"2024-04-25T13:58:09.210430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:28.376005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210007+0000","last_change":"2024-04-25T13:57:12.156536+0000","last_active":"2024-04-25T13:58:09.210007+0000","last_peered":"2024-04-25T13:58:09.210007+0000","last_clean":"2024-04-25T13:58:09.210007+0000","last_became_active":"2024-04-25T13:57:12.156143+0000","last_became_peered":"2024-04-25T13:57:12.156143+0000","last_unstale":"2024-04-25T13:58:09.210007+0000","last_undegraded":"2024-04-25T13:58:09.210007+0000","last_fullsized":"2024-04-25T13:58:09.210007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:35.210059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209472+0000","last_change":"2024-04-25T13:57:12.144477+0000","last_active":"2024-04-25T13:58:09.209472+0000","last_peered":"2024-04-25T13:58:09.209472+0000","last_clean":"2024-04-25T13:58:09.209472+0000","last_became_active":"2024-04-25T13:57:12.144115+0000","last_became_peered":"2024-04-25T13:57:12.144115+0000","last_unstale":"2024-04-25T13:58:09.209472+0000","last_undegraded":"2024-04-25T13:58:09.209472+0000","last_fullsized":"2024-04-25T13:58:09.209472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:30.733713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324890+0000","last_change":"2024-04-25T13:57:12.148837+0000","last_active":"2024-04-25T13:58:09.324890+0000","last_peered":"2024-04-25T13:58:09.324890+0000","last_clean":"2024-04-25T13:58:09.324890+0000","last_became_active":"2024-04-25T13:57:12.148494+0000","last_became_peered":"2024-04-25T13:57:12.148494+0000","last_unstale":"2024-04-25T13:58:09.324890+0000","last_undegraded":"2024-04-25T13:58:09.324890+0000","last_fullsized":"2024-04-25T13:58:09.324890+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:50.414344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209622+0000","last_change":"2024-04-25T13:57:12.148142+0000","last_active":"2024-04-25T13:58:09.209622+0000","last_peered":"2024-04-25T13:58:09.209622+0000","last_clean":"2024-04-25T13:58:09.209622+0000","last_became_active":"2024-04-25T13:57:12.147738+0000","last_became_peered":"2024-04-25T13:57:12.147738+0000","last_unstale":"2024-04-25T13:58:09.209622+0000","last_undegraded":"2024-04-25T13:58:09.209622+0000","last_fullsized":"2024-04-25T13:58:09.209622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:42.014605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.206009+0000","last_change":"2024-04-25T13:57:12.160059+0000","last_active":"2024-04-25T13:58:09.206009+0000","last_peered":"2024-04-25T13:58:09.206009+0000","last_clean":"2024-04-25T13:58:09.206009+0000","last_became_active":"2024-04-25T13:57:12.159685+0000","last_became_peered":"2024-04-25T13:57:12.159685+0000","last_unstale":"2024-04-25T13:58:09.206009+0000","last_undegraded":"2024-04-25T13:58:09.206009+0000","last_fullsized":"2024-04-25T13:58:09.206009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:48.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216794+0000","last_change":"2024-04-25T13:57:12.157950+0000","last_active":"2024-04-25T13:58:09.216794+0000","last_peered":"2024-04-25T13:58:09.216794+0000","last_clean":"2024-04-25T13:58:09.216794+0000","last_became_active":"2024-04-25T13:57:12.157612+0000","last_became_peered":"2024-04-25T13:57:12.157612+0000","last_unstale":"2024-04-25T13:58:09.216794+0000","last_undegraded":"2024-04-25T13:58:09.216794+0000","last_fullsized":"2024-04-25T13:58:09.216794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:55.521381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.222728+0000","last_change":"2024-04-25T13:58:09.222728+0000","last_active":"2024-04-25T13:58:09.222728+0000","last_peered":"2024-04-25T13:58:09.222728+0000","last_clean":"2024-04-25T13:58:09.222728+0000","last_became_active":"2024-04-25T13:58:09.210819+0000","last_became_peered":"2024-04-25T13:58:09.210819+0000","last_unstale":"2024-04-25T13:58:09.222728+0000","last_undegraded":"2024-04-25T13:58:09.222728+0000","last_fullsized":"2024-04-25T13:58:09.222728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:00.606838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212961+0000","last_change":"2024-04-25T13:57:12.151301+0000","last_active":"2024-04-25T13:58:09.212961+0000","last_peered":"2024-04-25T13:58:09.212961+0000","last_clean":"2024-04-25T13:58:09.212961+0000","last_became_active":"2024-04-25T13:57:12.150941+0000","last_became_peered":"2024-04-25T13:57:12.150941+0000","last_unstale":"2024-04-25T13:58:09.212961+0000","last_undegraded":"2024-04-25T13:58:09.212961+0000","last_fullsized":"2024-04-25T13:58:09.212961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:22.259683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213261+0000","last_change":"2024-04-25T13:57:11.147314+0000","last_active":"2024-04-25T13:58:09.213261+0000","last_peered":"2024-04-25T13:58:09.213261+0000","last_clean":"2024-04-25T13:58:09.213261+0000","last_became_active":"2024-04-25T13:57:11.146894+0000","last_became_peered":"2024-04-25T13:57:11.146894+0000","last_unstale":"2024-04-25T13:58:09.213261+0000","last_undegraded":"2024-04-25T13:58:09.213261+0000","last_fullsized":"2024-04-25T13:58:09.213261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:55.927007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210152+0000","last_change":"2024-04-25T13:57:12.149186+0000","last_active":"2024-04-25T13:58:09.210152+0000","last_peered":"2024-04-25T13:58:09.210152+0000","last_clean":"2024-04-25T13:58:09.210152+0000","last_became_active":"2024-04-25T13:57:12.148155+0000","last_became_peered":"2024-04-25T13:57:12.148155+0000","last_unstale":"2024-04-25T13:58:09.210152+0000","last_undegraded":"2024-04-25T13:58:09.210152+0000","last_fullsized":"2024-04-25T13:58:09.210152+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209681+0000","last_change":"2024-04-25T13:57:11.140086+0000","last_active":"2024-04-25T13:58:09.209681+0000","last_peered":"2024-04-25T13:58:09.209681+0000","last_clean":"2024-04-25T13:58:09.209681+0000","last_became_active":"2024-04-25T13:57:11.139751+0000","last_became_peered":"2024-04-25T13:57:11.139751+0000","last_unstale":"2024-04-25T13:58:09.209681+0000","last_undegraded":"2024-04-25T13:58:09.209681+0000","last_fullsized":"2024-04-25T13:58:09.209681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.552503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325309+0000","last_change":"2024-04-25T13:57:12.157029+0000","last_active":"2024-04-25T13:58:09.325309+0000","last_peered":"2024-04-25T13:58:09.325309+0000","last_clean":"2024-04-25T13:58:09.325309+0000","last_became_active":"2024-04-25T13:57:12.156668+0000","last_became_peered":"2024-04-25T13:57:12.156668+0000","last_unstale":"2024-04-25T13:58:09.325309+0000","last_undegraded":"2024-04-25T13:58:09.325309+0000","last_fullsized":"2024-04-25T13:58:09.325309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208828+0000","last_change":"2024-04-25T13:57:11.140592+0000","last_active":"2024-04-25T13:58:09.208828+0000","last_peered":"2024-04-25T13:58:09.208828+0000","last_clean":"2024-04-25T13:58:09.208828+0000","last_became_active":"2024-04-25T13:57:11.140244+0000","last_became_peered":"2024-04-25T13:57:11.140244+0000","last_unstale":"2024-04-25T13:58:09.208828+0000","last_undegraded":"2024-04-25T13:58:09.208828+0000","last_fullsized":"2024-04-25T13:58:09.208828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:53.071092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.326770+0000","last_change":"2024-04-25T13:58:09.326284+0000","last_active":"2024-04-25T13:58:09.326770+0000","last_peered":"2024-04-25T13:58:09.326770+0000","last_clean":"2024-04-25T13:58:09.326770+0000","last_became_active":"2024-04-25T13:58:09.325888+0000","last_became_peered":"2024-04-25T13:58:09.325888+0000","last_unstale":"2024-04-25T13:58:09.326770+0000","last_undegraded":"2024-04-25T13:58:09.326770+0000","last_fullsized":"2024-04-25T13:58:09.326770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:39.151793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":113,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:26.245669+0000","last_change":"2024-04-25T13:58:09.221709+0000","last_active":"2024-04-25T13:58:26.245669+0000","last_peered":"2024-04-25T13:58:26.245669+0000","last_clean":"2024-04-25T13:58:26.245669+0000","last_became_active":"2024-04-25T13:58:09.215262+0000","last_became_peered":"2024-04-25T13:58:09.215262+0000","last_unstale":"2024-04-25T13:58:26.245669+0000","last_undegraded":"2024-04-25T13:58:26.245669+0000","last_fullsized":"2024-04-25T13:58:26.245669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:05.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208574+0000","last_change":"2024-04-25T13:57:12.148787+0000","last_active":"2024-04-25T13:58:09.208574+0000","last_peered":"2024-04-25T13:58:09.208574+0000","last_clean":"2024-04-25T13:58:09.208574+0000","last_became_active":"2024-04-25T13:57:12.148424+0000","last_became_peered":"2024-04-25T13:57:12.148424+0000","last_unstale":"2024-04-25T13:58:09.208574+0000","last_undegraded":"2024-04-25T13:58:09.208574+0000","last_fullsized":"2024-04-25T13:58:09.208574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:11.921873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324717+0000","last_change":"2024-04-25T13:57:11.142424+0000","last_active":"2024-04-25T13:58:09.324717+0000","last_peered":"2024-04-25T13:58:09.324717+0000","last_clean":"2024-04-25T13:58:09.324717+0000","last_became_active":"2024-04-25T13:57:11.142047+0000","last_became_peered":"2024-04-25T13:57:11.142047+0000","last_unstale":"2024-04-25T13:58:09.324717+0000","last_undegraded":"2024-04-25T13:58:09.324717+0000","last_fullsized":"2024-04-25T13:58:09.324717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:13.455149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214205+0000","last_change":"2024-04-25T13:57:12.155940+0000","last_active":"2024-04-25T13:58:09.214205+0000","last_peered":"2024-04-25T13:58:09.214205+0000","last_clean":"2024-04-25T13:58:09.214205+0000","last_became_active":"2024-04-25T13:57:12.155503+0000","last_became_peered":"2024-04-25T13:57:12.155503+0000","last_unstale":"2024-04-25T13:58:09.214205+0000","last_undegraded":"2024-04-25T13:58:09.214205+0000","last_fullsized":"2024-04-25T13:58:09.214205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:48.345437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209853+0000","last_change":"2024-04-25T13:57:11.131160+0000","last_active":"2024-04-25T13:58:09.209853+0000","last_peered":"2024-04-25T13:58:09.209853+0000","last_clean":"2024-04-25T13:58:09.209853+0000","last_became_active":"2024-04-25T13:57:11.130753+0000","last_became_peered":"2024-04-25T13:57:11.130753+0000","last_unstale":"2024-04-25T13:58:09.209853+0000","last_undegraded":"2024-04-25T13:58:09.209853+0000","last_fullsized":"2024-04-25T13:58:09.209853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:48.297546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212990+0000","last_change":"2024-04-25T13:57:12.155779+0000","last_active":"2024-04-25T13:58:09.212990+0000","last_peered":"2024-04-25T13:58:09.212990+0000","last_clean":"2024-04-25T13:58:09.212990+0000","last_became_active":"2024-04-25T13:57:12.154682+0000","last_became_peered":"2024-04-25T13:57:12.154682+0000","last_unstale":"2024-04-25T13:58:09.212990+0000","last_undegraded":"2024-04-25T13:58:09.212990+0000","last_fullsized":"2024-04-25T13:58:09.212990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:20.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211839+0000","last_change":"2024-04-25T13:57:11.140211+0000","last_active":"2024-04-25T13:58:09.211839+0000","last_peered":"2024-04-25T13:58:09.211839+0000","last_clean":"2024-04-25T13:58:09.211839+0000","last_became_active":"2024-04-25T13:57:11.139891+0000","last_became_peered":"2024-04-25T13:57:11.139891+0000","last_unstale":"2024-04-25T13:58:09.211839+0000","last_undegraded":"2024-04-25T13:58:09.211839+0000","last_fullsized":"2024-04-25T13:58:09.211839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:21.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.205883+0000","last_change":"2024-04-25T13:57:12.153886+0000","last_active":"2024-04-25T13:58:09.205883+0000","last_peered":"2024-04-25T13:58:09.205883+0000","last_clean":"2024-04-25T13:58:09.205883+0000","last_became_active":"2024-04-25T13:57:12.153500+0000","last_became_peered":"2024-04-25T13:57:12.153500+0000","last_unstale":"2024-04-25T13:58:09.205883+0000","last_undegraded":"2024-04-25T13:58:09.205883+0000","last_fullsized":"2024-04-25T13:58:09.205883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:53.549745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'3","reported_seq":98,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:29.249701+0000","last_change":"2024-04-25T13:57:11.138829+0000","last_active":"2024-04-25T13:58:29.249701+0000","last_peered":"2024-04-25T13:58:29.249701+0000","last_clean":"2024-04-25T13:58:29.249701+0000","last_became_active":"2024-04-25T13:57:11.138461+0000","last_became_peered":"2024-04-25T13:57:11.138461+0000","last_unstale":"2024-04-25T13:58:29.249701+0000","last_undegraded":"2024-04-25T13:58:29.249701+0000","last_fullsized":"2024-04-25T13:58:29.249701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.134061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325531+0000","last_change":"2024-04-25T13:57:12.157394+0000","last_active":"2024-04-25T13:58:09.325531+0000","last_peered":"2024-04-25T13:58:09.325531+0000","last_clean":"2024-04-25T13:58:09.325531+0000","last_became_active":"2024-04-25T13:57:12.156408+0000","last_became_peered":"2024-04-25T13:57:12.156408+0000","last_unstale":"2024-04-25T13:58:09.325531+0000","last_undegraded":"2024-04-25T13:58:09.325531+0000","last_fullsized":"2024-04-25T13:58:09.325531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:36.741367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208928+0000","last_change":"2024-04-25T13:57:11.142846+0000","last_active":"2024-04-25T13:58:09.208928+0000","last_peered":"2024-04-25T13:58:09.208928+0000","last_clean":"2024-04-25T13:58:09.208928+0000","last_became_active":"2024-04-25T13:57:11.142387+0000","last_became_peered":"2024-04-25T13:57:11.142387+0000","last_unstale":"2024-04-25T13:58:09.208928+0000","last_undegraded":"2024-04-25T13:58:09.208928+0000","last_fullsized":"2024-04-25T13:58:09.208928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.164994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210009+0000","last_change":"2024-04-25T13:57:12.145617+0000","last_active":"2024-04-25T13:58:09.210009+0000","last_peered":"2024-04-25T13:58:09.210009+0000","last_clean":"2024-04-25T13:58:09.210009+0000","last_became_active":"2024-04-25T13:57:12.145263+0000","last_became_peered":"2024-04-25T13:57:12.145263+0000","last_unstale":"2024-04-25T13:58:09.210009+0000","last_undegraded":"2024-04-25T13:58:09.210009+0000","last_fullsized":"2024-04-25T13:58:09.210009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.886538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212101+0000","last_change":"2024-04-25T13:57:11.136972+0000","last_active":"2024-04-25T13:58:09.212101+0000","last_peered":"2024-04-25T13:58:09.212101+0000","last_clean":"2024-04-25T13:58:09.212101+0000","last_became_active":"2024-04-25T13:57:11.136392+0000","last_became_peered":"2024-04-25T13:57:11.136392+0000","last_unstale":"2024-04-25T13:58:09.212101+0000","last_undegraded":"2024-04-25T13:58:09.212101+0000","last_fullsized":"2024-04-25T13:58:09.212101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:45.841670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325608+0000","last_change":"2024-04-25T13:57:12.144905+0000","last_active":"2024-04-25T13:58:09.325608+0000","last_peered":"2024-04-25T13:58:09.325608+0000","last_clean":"2024-04-25T13:58:09.325608+0000","last_became_active":"2024-04-25T13:57:12.144404+0000","last_became_peered":"2024-04-25T13:57:12.144404+0000","last_unstale":"2024-04-25T13:58:09.325608+0000","last_undegraded":"2024-04-25T13:58:09.325608+0000","last_fullsized":"2024-04-25T13:58:09.325608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.186492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'28","reported_seq":207,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:30.246680+0000","last_change":"2024-04-25T13:57:11.139041+0000","last_active":"2024-04-25T13:58:30.246680+0000","last_peered":"2024-04-25T13:58:30.246680+0000","last_clean":"2024-04-25T13:58:30.246680+0000","last_became_active":"2024-04-25T13:57:11.138629+0000","last_became_peered":"2024-04-25T13:57:11.138629+0000","last_unstale":"2024-04-25T13:58:30.246680+0000","last_undegraded":"2024-04-25T13:58:30.246680+0000","last_fullsized":"2024-04-25T13:58:30.246680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:04.863015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209260+0000","last_change":"2024-04-25T13:57:12.160897+0000","last_active":"2024-04-25T13:58:09.209260+0000","last_peered":"2024-04-25T13:58:09.209260+0000","last_clean":"2024-04-25T13:58:09.209260+0000","last_became_active":"2024-04-25T13:57:12.160322+0000","last_became_peered":"2024-04-25T13:57:12.160322+0000","last_unstale":"2024-04-25T13:58:09.209260+0000","last_undegraded":"2024-04-25T13:58:09.209260+0000","last_fullsized":"2024-04-25T13:58:09.209260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:26.180324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216914+0000","last_change":"2024-04-25T13:57:11.137104+0000","last_active":"2024-04-25T13:58:09.216914+0000","last_peered":"2024-04-25T13:58:09.216914+0000","last_clean":"2024-04-25T13:58:09.216914+0000","last_became_active":"2024-04-25T13:57:11.136720+0000","last_became_peered":"2024-04-25T13:57:11.136720+0000","last_unstale":"2024-04-25T13:58:09.216914+0000","last_undegraded":"2024-04-25T13:58:09.216914+0000","last_fullsized":"2024-04-25T13:58:09.216914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:41.262776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210048+0000","last_change":"2024-04-25T13:57:12.147831+0000","last_active":"2024-04-25T13:58:09.210048+0000","last_peered":"2024-04-25T13:58:09.210048+0000","last_clean":"2024-04-25T13:58:09.210048+0000","last_became_active":"2024-04-25T13:57:12.147270+0000","last_became_peered":"2024-04-25T13:57:12.147270+0000","last_unstale":"2024-04-25T13:58:09.210048+0000","last_undegraded":"2024-04-25T13:58:09.210048+0000","last_fullsized":"2024-04-25T13:58:09.210048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:31.574661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211663+0000","last_change":"2024-04-25T13:57:11.138254+0000","last_active":"2024-04-25T13:58:09.211663+0000","last_peered":"2024-04-25T13:58:09.211663+0000","last_clean":"2024-04-25T13:58:09.211663+0000","last_became_active":"2024-04-25T13:57:11.137924+0000","last_became_peered":"2024-04-25T13:57:11.137924+0000","last_unstale":"2024-04-25T13:58:09.211663+0000","last_undegraded":"2024-04-25T13:58:09.211663+0000","last_fullsized":"2024-04-25T13:58:09.211663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:27.746203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208758+0000","last_change":"2024-04-25T13:57:12.145743+0000","last_active":"2024-04-25T13:58:09.208758+0000","last_peered":"2024-04-25T13:58:09.208758+0000","last_clean":"2024-04-25T13:58:09.208758+0000","last_became_active":"2024-04-25T13:57:12.145337+0000","last_became_peered":"2024-04-25T13:57:12.145337+0000","last_unstale":"2024-04-25T13:58:09.208758+0000","last_undegraded":"2024-04-25T13:58:09.208758+0000","last_fullsized":"2024-04-25T13:58:09.208758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.692900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'4","reported_seq":102,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:30.205010+0000","last_change":"2024-04-25T13:57:11.138059+0000","last_active":"2024-04-25T13:58:30.205010+0000","last_peered":"2024-04-25T13:58:30.205010+0000","last_clean":"2024-04-25T13:58:30.205010+0000","last_became_active":"2024-04-25T13:57:11.137698+0000","last_became_peered":"2024-04-25T13:57:11.137698+0000","last_unstale":"2024-04-25T13:58:30.205010+0000","last_undegraded":"2024-04-25T13:58:30.205010+0000","last_fullsized":"2024-04-25T13:58:30.205010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:03.595126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208160+0000","last_change":"2024-04-25T13:57:12.155507+0000","last_active":"2024-04-25T13:58:09.208160+0000","last_peered":"2024-04-25T13:58:09.208160+0000","last_clean":"2024-04-25T13:58:09.208160+0000","last_became_active":"2024-04-25T13:57:12.155175+0000","last_became_peered":"2024-04-25T13:57:12.155175+0000","last_unstale":"2024-04-25T13:58:09.208160+0000","last_undegraded":"2024-04-25T13:58:09.208160+0000","last_fullsized":"2024-04-25T13:58:09.208160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:21.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'15","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208247+0000","last_change":"2024-04-25T13:57:11.141672+0000","last_active":"2024-04-25T13:58:09.208247+0000","last_peered":"2024-04-25T13:58:09.208247+0000","last_clean":"2024-04-25T13:58:09.208247+0000","last_became_active":"2024-04-25T13:57:11.140957+0000","last_became_peered":"2024-04-25T13:57:11.140957+0000","last_unstale":"2024-04-25T13:58:09.208247+0000","last_undegraded":"2024-04-25T13:58:09.208247+0000","last_fullsized":"2024-04-25T13:58:09.208247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:47.362949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211503+0000","last_change":"2024-04-25T13:57:12.147881+0000","last_active":"2024-04-25T13:58:09.211503+0000","last_peered":"2024-04-25T13:58:09.211503+0000","last_clean":"2024-04-25T13:58:09.211503+0000","last_became_active":"2024-04-25T13:57:12.146200+0000","last_became_peered":"2024-04-25T13:57:12.146200+0000","last_unstale":"2024-04-25T13:58:09.211503+0000","last_undegraded":"2024-04-25T13:58:09.211503+0000","last_fullsized":"2024-04-25T13:58:09.211503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:51.711045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209516+0000","last_change":"2024-04-25T13:57:11.139574+0000","last_active":"2024-04-25T13:58:09.209516+0000","last_peered":"2024-04-25T13:58:09.209516+0000","last_clean":"2024-04-25T13:58:09.209516+0000","last_became_active":"2024-04-25T13:57:11.139232+0000","last_became_peered":"2024-04-25T13:57:11.139232+0000","last_unstale":"2024-04-25T13:58:09.209516+0000","last_undegraded":"2024-04-25T13:58:09.209516+0000","last_fullsized":"2024-04-25T13:58:09.209516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:08.590270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209697+0000","last_change":"2024-04-25T13:57:12.147122+0000","last_active":"2024-04-25T13:58:09.209697+0000","last_peered":"2024-04-25T13:58:09.209697+0000","last_clean":"2024-04-25T13:58:09.209697+0000","last_became_active":"2024-04-25T13:57:12.146708+0000","last_became_peered":"2024-04-25T13:57:12.146708+0000","last_unstale":"2024-04-25T13:58:09.209697+0000","last_undegraded":"2024-04-25T13:58:09.209697+0000","last_fullsized":"2024-04-25T13:58:09.209697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:47.858290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.205903+0000","last_change":"2024-04-25T13:57:11.146288+0000","last_active":"2024-04-25T13:58:09.205903+0000","last_peered":"2024-04-25T13:58:09.205903+0000","last_clean":"2024-04-25T13:58:09.205903+0000","last_became_active":"2024-04-25T13:57:11.145851+0000","last_became_peered":"2024-04-25T13:57:11.145851+0000","last_unstale":"2024-04-25T13:58:09.205903+0000","last_undegraded":"2024-04-25T13:58:09.205903+0000","last_fullsized":"2024-04-25T13:58:09.205903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325156+0000","last_change":"2024-04-25T13:57:12.154576+0000","last_active":"2024-04-25T13:58:09.325156+0000","last_peered":"2024-04-25T13:58:09.325156+0000","last_clean":"2024-04-25T13:58:09.325156+0000","last_became_active":"2024-04-25T13:57:12.154254+0000","last_became_peered":"2024-04-25T13:57:12.154254+0000","last_unstale":"2024-04-25T13:58:09.325156+0000","last_undegraded":"2024-04-25T13:58:09.325156+0000","last_fullsized":"2024-04-25T13:58:09.325156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:33.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208334+0000","last_change":"2024-04-25T13:57:11.139941+0000","last_active":"2024-04-25T13:58:09.208334+0000","last_peered":"2024-04-25T13:58:09.208334+0000","last_clean":"2024-04-25T13:58:09.208334+0000","last_became_active":"2024-04-25T13:57:11.139614+0000","last_became_peered":"2024-04-25T13:57:11.139614+0000","last_unstale":"2024-04-25T13:58:09.208334+0000","last_undegraded":"2024-04-25T13:58:09.208334+0000","last_fullsized":"2024-04-25T13:58:09.208334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:39.656085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325028+0000","last_change":"2024-04-25T13:57:12.155935+0000","last_active":"2024-04-25T13:58:09.325028+0000","last_peered":"2024-04-25T13:58:09.325028+0000","last_clean":"2024-04-25T13:58:09.325028+0000","last_became_active":"2024-04-25T13:57:12.155261+0000","last_became_peered":"2024-04-25T13:57:12.155261+0000","last_unstale":"2024-04-25T13:58:09.325028+0000","last_undegraded":"2024-04-25T13:58:09.325028+0000","last_fullsized":"2024-04-25T13:58:09.325028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:54.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217792+0000","last_change":"2024-04-25T13:58:09.217792+0000","last_active":"2024-04-25T13:58:09.217792+0000","last_peered":"2024-04-25T13:58:09.217792+0000","last_clean":"2024-04-25T13:58:09.217792+0000","last_became_active":"2024-04-25T13:58:09.217386+0000","last_became_peered":"2024-04-25T13:58:09.217386+0000","last_unstale":"2024-04-25T13:58:09.217792+0000","last_undegraded":"2024-04-25T13:58:09.217792+0000","last_fullsized":"2024-04-25T13:58:09.217792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:13.833587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'4","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:26.198771+0000","last_change":"2024-04-25T13:57:11.135226+0000","last_active":"2024-04-25T13:58:26.198771+0000","last_peered":"2024-04-25T13:58:26.198771+0000","last_clean":"2024-04-25T13:58:26.198771+0000","last_became_active":"2024-04-25T13:57:11.134886+0000","last_became_peered":"2024-04-25T13:57:11.134886+0000","last_unstale":"2024-04-25T13:58:26.198771+0000","last_undegraded":"2024-04-25T13:58:26.198771+0000","last_fullsized":"2024-04-25T13:58:26.198771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:00.153597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213327+0000","last_change":"2024-04-25T13:57:12.149758+0000","last_active":"2024-04-25T13:58:09.213327+0000","last_peered":"2024-04-25T13:58:09.213327+0000","last_clean":"2024-04-25T13:58:09.213327+0000","last_became_active":"2024-04-25T13:57:12.149428+0000","last_became_peered":"2024-04-25T13:57:12.149428+0000","last_unstale":"2024-04-25T13:58:09.213327+0000","last_undegraded":"2024-04-25T13:58:09.213327+0000","last_fullsized":"2024-04-25T13:58:09.213327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.739074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207722+0000","last_change":"2024-04-25T13:57:11.135968+0000","last_active":"2024-04-25T13:58:09.207722+0000","last_peered":"2024-04-25T13:58:09.207722+0000","last_clean":"2024-04-25T13:58:09.207722+0000","last_became_active":"2024-04-25T13:57:11.135577+0000","last_became_peered":"2024-04-25T13:57:11.135577+0000","last_unstale":"2024-04-25T13:58:09.207722+0000","last_undegraded":"2024-04-25T13:58:09.207722+0000","last_fullsized":"2024-04-25T13:58:09.207722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:00.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208081+0000","last_change":"2024-04-25T13:55:57.666399+0000","last_active":"2024-04-25T13:58:09.208081+0000","last_peered":"2024-04-25T13:58:09.208081+0000","last_clean":"2024-04-25T13:58:09.208081+0000","last_became_active":"2024-04-25T13:55:57.660470+0000","last_became_peered":"2024-04-25T13:55:57.660470+0000","last_unstale":"2024-04-25T13:58:09.208081+0000","last_undegraded":"2024-04-25T13:58:09.208081+0000","last_fullsized":"2024-04-25T13:58:09.208081+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_clean_scrub_stamp":"2024-04-25T13:52:33.081441+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:22.788632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209521+0000","last_change":"2024-04-25T13:57:12.141923+0000","last_active":"2024-04-25T13:58:09.209521+0000","last_peered":"2024-04-25T13:58:09.209521+0000","last_clean":"2024-04-25T13:58:09.209521+0000","last_became_active":"2024-04-25T13:57:12.140946+0000","last_became_peered":"2024-04-25T13:57:12.140946+0000","last_unstale":"2024-04-25T13:58:09.209521+0000","last_undegraded":"2024-04-25T13:58:09.209521+0000","last_fullsized":"2024-04-25T13:58:09.209521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:49.294598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208387+0000","last_change":"2024-04-25T13:57:11.139851+0000","last_active":"2024-04-25T13:58:09.208387+0000","last_peered":"2024-04-25T13:58:09.208387+0000","last_clean":"2024-04-25T13:58:09.208387+0000","last_became_active":"2024-04-25T13:57:11.139476+0000","last_became_peered":"2024-04-25T13:57:11.139476+0000","last_unstale":"2024-04-25T13:58:09.208387+0000","last_undegraded":"2024-04-25T13:58:09.208387+0000","last_fullsized":"2024-04-25T13:58:09.208387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:29.435917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.220107+0000","last_change":"2024-04-25T13:58:09.219619+0000","last_active":"2024-04-25T13:58:09.220107+0000","last_peered":"2024-04-25T13:58:09.220107+0000","last_clean":"2024-04-25T13:58:09.220107+0000","last_became_active":"2024-04-25T13:58:09.211808+0000","last_became_peered":"2024-04-25T13:58:09.211808+0000","last_unstale":"2024-04-25T13:58:09.220107+0000","last_undegraded":"2024-04-25T13:58:09.220107+0000","last_fullsized":"2024-04-25T13:58:09.220107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:20.208191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210754+0000","last_change":"2024-04-25T13:57:12.136204+0000","last_active":"2024-04-25T13:58:09.210754+0000","last_peered":"2024-04-25T13:58:09.210754+0000","last_clean":"2024-04-25T13:58:09.210754+0000","last_became_active":"2024-04-25T13:57:12.135798+0000","last_became_peered":"2024-04-25T13:57:12.135798+0000","last_unstale":"2024-04-25T13:58:09.210754+0000","last_undegraded":"2024-04-25T13:58:09.210754+0000","last_fullsized":"2024-04-25T13:58:09.210754+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:36.894774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.206095+0000","last_change":"2024-04-25T13:57:11.138699+0000","last_active":"2024-04-25T13:58:09.206095+0000","last_peered":"2024-04-25T13:58:09.206095+0000","last_clean":"2024-04-25T13:58:09.206095+0000","last_became_active":"2024-04-25T13:57:11.138335+0000","last_became_peered":"2024-04-25T13:57:11.138335+0000","last_unstale":"2024-04-25T13:58:09.206095+0000","last_undegraded":"2024-04-25T13:58:09.206095+0000","last_fullsized":"2024-04-25T13:58:09.206095+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:44.413228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214134+0000","last_change":"2024-04-25T13:57:12.155851+0000","last_active":"2024-04-25T13:58:09.214134+0000","last_peered":"2024-04-25T13:58:09.214134+0000","last_clean":"2024-04-25T13:58:09.214134+0000","last_became_active":"2024-04-25T13:57:12.155416+0000","last_became_peered":"2024-04-25T13:57:12.155416+0000","last_unstale":"2024-04-25T13:58:09.214134+0000","last_undegraded":"2024-04-25T13:58:09.214134+0000","last_fullsized":"2024-04-25T13:58:09.214134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:22.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207744+0000","last_change":"2024-04-25T13:57:11.147465+0000","last_active":"2024-04-25T13:58:09.207744+0000","last_peered":"2024-04-25T13:58:09.207744+0000","last_clean":"2024-04-25T13:58:09.207744+0000","last_became_active":"2024-04-25T13:57:11.147096+0000","last_became_peered":"2024-04-25T13:57:11.147096+0000","last_unstale":"2024-04-25T13:58:09.207744+0000","last_undegraded":"2024-04-25T13:58:09.207744+0000","last_fullsized":"2024-04-25T13:58:09.207744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.229356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210528+0000","last_change":"2024-04-25T13:57:12.146446+0000","last_active":"2024-04-25T13:58:09.210528+0000","last_peered":"2024-04-25T13:58:09.210528+0000","last_clean":"2024-04-25T13:58:09.210528+0000","last_became_active":"2024-04-25T13:57:12.146129+0000","last_became_peered":"2024-04-25T13:57:12.146129+0000","last_unstale":"2024-04-25T13:58:09.210528+0000","last_undegraded":"2024-04-25T13:58:09.210528+0000","last_fullsized":"2024-04-25T13:58:09.210528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.977663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325014+0000","last_change":"2024-04-25T13:57:11.145127+0000","last_active":"2024-04-25T13:58:09.325014+0000","last_peered":"2024-04-25T13:58:09.325014+0000","last_clean":"2024-04-25T13:58:09.325014+0000","last_became_active":"2024-04-25T13:57:11.144822+0000","last_became_peered":"2024-04-25T13:57:11.144822+0000","last_unstale":"2024-04-25T13:58:09.325014+0000","last_undegraded":"2024-04-25T13:58:09.325014+0000","last_fullsized":"2024-04-25T13:58:09.325014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:00.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208402+0000","last_change":"2024-04-25T13:57:12.156184+0000","last_active":"2024-04-25T13:58:09.208402+0000","last_peered":"2024-04-25T13:58:09.208402+0000","last_clean":"2024-04-25T13:58:09.208402+0000","last_became_active":"2024-04-25T13:57:12.155855+0000","last_became_peered":"2024-04-25T13:57:12.155855+0000","last_unstale":"2024-04-25T13:58:09.208402+0000","last_undegraded":"2024-04-25T13:58:09.208402+0000","last_fullsized":"2024-04-25T13:58:09.208402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213952+0000","last_change":"2024-04-25T13:57:11.137616+0000","last_active":"2024-04-25T13:58:09.213952+0000","last_peered":"2024-04-25T13:58:09.213952+0000","last_clean":"2024-04-25T13:58:09.213952+0000","last_became_active":"2024-04-25T13:57:11.137041+0000","last_became_peered":"2024-04-25T13:57:11.137041+0000","last_unstale":"2024-04-25T13:58:09.213952+0000","last_undegraded":"2024-04-25T13:58:09.213952+0000","last_fullsized":"2024-04-25T13:58:09.213952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:54.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208531+0000","last_change":"2024-04-25T13:57:12.151223+0000","last_active":"2024-04-25T13:58:09.208531+0000","last_peered":"2024-04-25T13:58:09.208531+0000","last_clean":"2024-04-25T13:58:09.208531+0000","last_became_active":"2024-04-25T13:57:12.150871+0000","last_became_peered":"2024-04-25T13:57:12.150871+0000","last_unstale":"2024-04-25T13:58:09.208531+0000","last_undegraded":"2024-04-25T13:58:09.208531+0000","last_fullsized":"2024-04-25T13:58:09.208531+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:17.957728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216414+0000","last_change":"2024-04-25T13:57:11.136524+0000","last_active":"2024-04-25T13:58:09.216414+0000","last_peered":"2024-04-25T13:58:09.216414+0000","last_clean":"2024-04-25T13:58:09.216414+0000","last_became_active":"2024-04-25T13:57:11.136152+0000","last_became_peered":"2024-04-25T13:57:11.136152+0000","last_unstale":"2024-04-25T13:58:09.216414+0000","last_undegraded":"2024-04-25T13:58:09.216414+0000","last_fullsized":"2024-04-25T13:58:09.216414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:23.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214204+0000","last_change":"2024-04-25T13:57:12.157641+0000","last_active":"2024-04-25T13:58:09.214204+0000","last_peered":"2024-04-25T13:58:09.214204+0000","last_clean":"2024-04-25T13:58:09.214204+0000","last_became_active":"2024-04-25T13:57:12.157281+0000","last_became_peered":"2024-04-25T13:57:12.157281+0000","last_unstale":"2024-04-25T13:58:09.214204+0000","last_undegraded":"2024-04-25T13:58:09.214204+0000","last_fullsized":"2024-04-25T13:58:09.214204+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:28.245071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208565+0000","last_change":"2024-04-25T13:57:11.139036+0000","last_active":"2024-04-25T13:58:09.208565+0000","last_peered":"2024-04-25T13:58:09.208565+0000","last_clean":"2024-04-25T13:58:09.208565+0000","last_became_active":"2024-04-25T13:57:11.138681+0000","last_became_peered":"2024-04-25T13:57:11.138681+0000","last_unstale":"2024-04-25T13:58:09.208565+0000","last_undegraded":"2024-04-25T13:58:09.208565+0000","last_fullsized":"2024-04-25T13:58:09.208565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:48.584313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208577+0000","last_change":"2024-04-25T13:57:12.153072+0000","last_active":"2024-04-25T13:58:09.208577+0000","last_peered":"2024-04-25T13:58:09.208577+0000","last_clean":"2024-04-25T13:58:09.208577+0000","last_became_active":"2024-04-25T13:57:12.151556+0000","last_became_peered":"2024-04-25T13:57:12.151556+0000","last_unstale":"2024-04-25T13:58:09.208577+0000","last_undegraded":"2024-04-25T13:58:09.208577+0000","last_fullsized":"2024-04-25T13:58:09.208577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:20.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217008+0000","last_change":"2024-04-25T13:57:11.148823+0000","last_active":"2024-04-25T13:58:09.217008+0000","last_peered":"2024-04-25T13:58:09.217008+0000","last_clean":"2024-04-25T13:58:09.217008+0000","last_became_active":"2024-04-25T13:57:11.148436+0000","last_became_peered":"2024-04-25T13:57:11.148436+0000","last_unstale":"2024-04-25T13:58:09.217008+0000","last_undegraded":"2024-04-25T13:58:09.217008+0000","last_fullsized":"2024-04-25T13:58:09.217008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:28.496184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208931+0000","last_change":"2024-04-25T13:57:12.149082+0000","last_active":"2024-04-25T13:58:09.208931+0000","last_peered":"2024-04-25T13:58:09.208931+0000","last_clean":"2024-04-25T13:58:09.208931+0000","last_became_active":"2024-04-25T13:57:12.148260+0000","last_became_peered":"2024-04-25T13:57:12.148260+0000","last_unstale":"2024-04-25T13:58:09.208931+0000","last_undegraded":"2024-04-25T13:58:09.208931+0000","last_fullsized":"2024-04-25T13:58:09.208931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:49.733114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211832+0000","last_change":"2024-04-25T13:57:11.136736+0000","last_active":"2024-04-25T13:58:09.211832+0000","last_peered":"2024-04-25T13:58:09.211832+0000","last_clean":"2024-04-25T13:58:09.211832+0000","last_became_active":"2024-04-25T13:57:11.135831+0000","last_became_peered":"2024-04-25T13:57:11.135831+0000","last_unstale":"2024-04-25T13:58:09.211832+0000","last_undegraded":"2024-04-25T13:58:09.211832+0000","last_fullsized":"2024-04-25T13:58:09.211832+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:49.654642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208827+0000","last_change":"2024-04-25T13:57:12.149797+0000","last_active":"2024-04-25T13:58:09.208827+0000","last_peered":"2024-04-25T13:58:09.208827+0000","last_clean":"2024-04-25T13:58:09.208827+0000","last_became_active":"2024-04-25T13:57:12.149413+0000","last_became_peered":"2024-04-25T13:57:12.149413+0000","last_unstale":"2024-04-25T13:58:09.208827+0000","last_undegraded":"2024-04-25T13:58:09.208827+0000","last_fullsized":"2024-04-25T13:58:09.208827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:58.517208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.328809+0000","last_change":"2024-04-25T13:58:09.328809+0000","last_active":"2024-04-25T13:58:09.328809+0000","last_peered":"2024-04-25T13:58:09.328809+0000","last_clean":"2024-04-25T13:58:09.328809+0000","last_became_active":"2024-04-25T13:58:09.326068+0000","last_became_peered":"2024-04-25T13:58:09.326068+0000","last_unstale":"2024-04-25T13:58:09.328809+0000","last_undegraded":"2024-04-25T13:58:09.328809+0000","last_fullsized":"2024-04-25T13:58:09.328809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:14.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209112+0000","last_change":"2024-04-25T13:57:12.150247+0000","last_active":"2024-04-25T13:58:09.209112+0000","last_peered":"2024-04-25T13:58:09.209112+0000","last_clean":"2024-04-25T13:58:09.209112+0000","last_became_active":"2024-04-25T13:57:12.149882+0000","last_became_peered":"2024-04-25T13:57:12.149882+0000","last_unstale":"2024-04-25T13:58:09.209112+0000","last_undegraded":"2024-04-25T13:58:09.209112+0000","last_fullsized":"2024-04-25T13:58:09.209112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.624928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208702+0000","last_change":"2024-04-25T13:57:12.154618+0000","last_active":"2024-04-25T13:58:09.208702+0000","last_peered":"2024-04-25T13:58:09.208702+0000","last_clean":"2024-04-25T13:58:09.208702+0000","last_became_active":"2024-04-25T13:57:12.153648+0000","last_became_peered":"2024-04-25T13:57:12.153648+0000","last_unstale":"2024-04-25T13:58:09.208702+0000","last_undegraded":"2024-04-25T13:58:09.208702+0000","last_fullsized":"2024-04-25T13:58:09.208702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:24.010980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216677+0000","last_change":"2024-04-25T13:57:12.143097+0000","last_active":"2024-04-25T13:58:09.216677+0000","last_peered":"2024-04-25T13:58:09.216677+0000","last_clean":"2024-04-25T13:58:09.216677+0000","last_became_active":"2024-04-25T13:57:12.142295+0000","last_became_peered":"2024-04-25T13:57:12.142295+0000","last_unstale":"2024-04-25T13:58:09.216677+0000","last_undegraded":"2024-04-25T13:58:09.216677+0000","last_fullsized":"2024-04-25T13:58:09.216677+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.907643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216299+0000","last_change":"2024-04-25T13:57:12.155659+0000","last_active":"2024-04-25T13:58:09.216299+0000","last_peered":"2024-04-25T13:58:09.216299+0000","last_clean":"2024-04-25T13:58:09.216299+0000","last_became_active":"2024-04-25T13:57:12.155308+0000","last_became_peered":"2024-04-25T13:57:12.155308+0000","last_unstale":"2024-04-25T13:58:09.216299+0000","last_undegraded":"2024-04-25T13:58:09.216299+0000","last_fullsized":"2024-04-25T13:58:09.216299+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:12.361660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212066+0000","last_change":"2024-04-25T13:58:09.212066+0000","last_active":"2024-04-25T13:58:09.212066+0000","last_peered":"2024-04-25T13:58:09.212066+0000","last_clean":"2024-04-25T13:58:09.212066+0000","last_became_active":"2024-04-25T13:58:09.211708+0000","last_became_peered":"2024-04-25T13:58:09.211708+0000","last_unstale":"2024-04-25T13:58:09.212066+0000","last_undegraded":"2024-04-25T13:58:09.212066+0000","last_fullsized":"2024-04-25T13:58:09.212066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:34.585999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210854+0000","last_change":"2024-04-25T13:57:12.142134+0000","last_active":"2024-04-25T13:58:09.210854+0000","last_peered":"2024-04-25T13:58:09.210854+0000","last_clean":"2024-04-25T13:58:09.210854+0000","last_became_active":"2024-04-25T13:57:12.141776+0000","last_became_peered":"2024-04-25T13:57:12.141776+0000","last_unstale":"2024-04-25T13:58:09.210854+0000","last_undegraded":"2024-04-25T13:58:09.210854+0000","last_fullsized":"2024-04-25T13:58:09.210854+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:07.621761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213725+0000","last_change":"2024-04-25T13:58:09.213725+0000","last_active":"2024-04-25T13:58:09.213725+0000","last_peered":"2024-04-25T13:58:09.213725+0000","last_clean":"2024-04-25T13:58:09.213725+0000","last_became_active":"2024-04-25T13:58:09.213229+0000","last_became_peered":"2024-04-25T13:58:09.213229+0000","last_unstale":"2024-04-25T13:58:09.213725+0000","last_undegraded":"2024-04-25T13:58:09.213725+0000","last_fullsized":"2024-04-25T13:58:09.213725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:14.728468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217140+0000","last_change":"2024-04-25T13:57:12.147818+0000","last_active":"2024-04-25T13:58:09.217140+0000","last_peered":"2024-04-25T13:58:09.217140+0000","last_clean":"2024-04-25T13:58:09.217140+0000","last_became_active":"2024-04-25T13:57:12.146764+0000","last_became_peered":"2024-04-25T13:57:12.146764+0000","last_unstale":"2024-04-25T13:58:09.217140+0000","last_undegraded":"2024-04-25T13:58:09.217140+0000","last_fullsized":"2024-04-25T13:58:09.217140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.750115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.214031+0000","last_change":"2024-04-25T13:57:12.152591+0000","last_active":"2024-04-25T13:58:09.214031+0000","last_peered":"2024-04-25T13:58:09.214031+0000","last_clean":"2024-04-25T13:58:09.214031+0000","last_became_active":"2024-04-25T13:57:12.152198+0000","last_became_peered":"2024-04-25T13:57:12.152198+0000","last_unstale":"2024-04-25T13:58:09.214031+0000","last_undegraded":"2024-04-25T13:58:09.214031+0000","last_fullsized":"2024-04-25T13:58:09.214031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:05.868774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211990+0000","last_change":"2024-04-25T13:57:12.154924+0000","last_active":"2024-04-25T13:58:09.211990+0000","last_peered":"2024-04-25T13:58:09.211990+0000","last_clean":"2024-04-25T13:58:09.211990+0000","last_became_active":"2024-04-25T13:57:12.154549+0000","last_became_peered":"2024-04-25T13:57:12.154549+0000","last_unstale":"2024-04-25T13:58:09.211990+0000","last_undegraded":"2024-04-25T13:58:09.211990+0000","last_fullsized":"2024-04-25T13:58:09.211990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:49.418436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217391+0000","last_change":"2024-04-25T13:58:09.216933+0000","last_active":"2024-04-25T13:58:09.217391+0000","last_peered":"2024-04-25T13:58:09.217391+0000","last_clean":"2024-04-25T13:58:09.217391+0000","last_became_active":"2024-04-25T13:58:09.216615+0000","last_became_peered":"2024-04-25T13:58:09.216615+0000","last_unstale":"2024-04-25T13:58:09.217391+0000","last_undegraded":"2024-04-25T13:58:09.217391+0000","last_fullsized":"2024-04-25T13:58:09.217391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:25.597023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325474+0000","last_change":"2024-04-25T13:57:12.158672+0000","last_active":"2024-04-25T13:58:09.325474+0000","last_peered":"2024-04-25T13:58:09.325474+0000","last_clean":"2024-04-25T13:58:09.325474+0000","last_became_active":"2024-04-25T13:57:12.158391+0000","last_became_peered":"2024-04-25T13:57:12.158391+0000","last_unstale":"2024-04-25T13:58:09.325474+0000","last_undegraded":"2024-04-25T13:58:09.325474+0000","last_fullsized":"2024-04-25T13:58:09.325474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.369907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.210535+0000","last_change":"2024-04-25T13:57:12.139310+0000","last_active":"2024-04-25T13:58:09.210535+0000","last_peered":"2024-04-25T13:58:09.210535+0000","last_clean":"2024-04-25T13:58:09.210535+0000","last_became_active":"2024-04-25T13:57:12.138942+0000","last_became_peered":"2024-04-25T13:57:12.138942+0000","last_unstale":"2024-04-25T13:58:09.210535+0000","last_undegraded":"2024-04-25T13:58:09.210535+0000","last_fullsized":"2024-04-25T13:58:09.210535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:01.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209279+0000","last_change":"2024-04-25T13:57:12.147451+0000","last_active":"2024-04-25T13:58:09.209279+0000","last_peered":"2024-04-25T13:58:09.209279+0000","last_clean":"2024-04-25T13:58:09.209279+0000","last_became_active":"2024-04-25T13:57:12.146746+0000","last_became_peered":"2024-04-25T13:57:12.146746+0000","last_unstale":"2024-04-25T13:58:09.209279+0000","last_undegraded":"2024-04-25T13:58:09.209279+0000","last_fullsized":"2024-04-25T13:58:09.209279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247568,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":474,"num_read_kb":306,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1617,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196314,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710749,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":402153,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906979,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1054467,"data_compressed":43506,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":401785,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266733,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":839680,"data_stored":401683,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462962,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":993444,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626487,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":831488,"data_stored":401174,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":462013,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":402445,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":905216,"data_stored":462741,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":401485,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986242,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":843776,"data_stored":403015,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822716,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":993319,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61774,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63015,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:58:33.842 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T13:58:33.842 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:58:33.842 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:58:33.842 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T13:58:34.081 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:34 smithi078 ceph-mon[29163]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:58:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:34 smithi112 ceph-mon[32617]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:58:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:34 smithi129 ceph-mon[32569]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:58:35.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:35 smithi078 ceph-mon[29163]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:58:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:35 smithi112 ceph-mon[32617]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:58:35.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:35 smithi129 ceph-mon[32569]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:58:35.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 -- 172.21.15.78:0/1443126100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790107c90 msgr2=0x7f57901080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:35.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 --2- 172.21.15.78:0/1443126100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790107c90 0x7f57901080a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57800099c0 tx=0x7f5780009cd0 comp rx=0 tx=0).stop 2024-04-25T13:58:35.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 -- 172.21.15.78:0/1443126100 shutdown_connections 2024-04-25T13:58:35.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 --2- 172.21.15.78:0/1443126100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57901085e0 0x7f579010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:35.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 --2- 172.21.15.78:0/1443126100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5790103360 0x7f5790103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:35.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 --2- 172.21.15.78:0/1443126100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5790107c90 0x7f57901080a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:35.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 -- 172.21.15.78:0/1443126100 >> 172.21.15.78:0/1443126100 conn(0x7f57900fecb0 msgr2=0x7f57901010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:35.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 -- 172.21.15.78:0/1443126100 shutdown_connections 2024-04-25T13:58:35.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.788+0000 7f57986e5700 1 -- 172.21.15.78:0/1443126100 wait complete. 2024-04-25T13:58:35.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 Processor -- start 2024-04-25T13:58:35.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 -- start start 2024-04-25T13:58:35.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5790103360 0x7f579019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:35.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5790107c90 0x7f579019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:35.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57901085e0 0x7f57901a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:35.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5790113260 con 0x7f57901085e0 2024-04-25T13:58:35.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57901130e0 con 0x7f5790107c90 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f57986e5700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57901133e0 con 0x7f5790103360 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f5796c82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57901085e0 0x7f57901a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f5796c82700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57901085e0 0x7f57901a6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49070/0 (socket says 172.21.15.78:49070) 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f5796c82700 1 -- 172.21.15.78:0/2630516745 learned_addr learned my addr 172.21.15.78:0/2630516745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.789+0000 7f5795c80700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5790107c90 0x7f579019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f5796481700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5790103360 0x7f579019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f5796c82700 1 -- 172.21.15.78:0/2630516745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5790103360 msgr2=0x7f579019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f5796c82700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5790103360 0x7f579019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f5796c82700 1 -- 172.21.15.78:0/2630516745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5790107c90 msgr2=0x7f579019c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f5796c82700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5790107c90 0x7f579019c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f5796c82700 1 -- 172.21.15.78:0/2630516745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579019c860 con 0x7f57901085e0 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f5796c82700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57901085e0 0x7f57901a6b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f578800b970 tx=0x7f578800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f57877fe700 1 -- 172.21.15.78:0/2630516745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788019070 con 0x7f57901085e0 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57901a7120 con 0x7f57901085e0 2024-04-25T13:58:35.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f57877fe700 1 -- 172.21.15.78:0/2630516745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f578800cde0 con 0x7f57901085e0 2024-04-25T13:58:35.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f57877fe700 1 -- 172.21.15.78:0/2630516745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788007370 con 0x7f57901085e0 2024-04-25T13:58:35.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.790+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57901a7630 con 0x7f57901085e0 2024-04-25T13:58:35.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.791+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5790051190 con 0x7f57901085e0 2024-04-25T13:58:35.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.794+0000 7f57877fe700 1 -- 172.21.15.78:0/2630516745 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f578801c070 con 0x7f57901085e0 2024-04-25T13:58:35.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.794+0000 7f57877fe700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f577c0afc90 0x7f577c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:35.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.794+0000 7f5796481700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f577c0afc90 0x7f577c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:35.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.794+0000 7f57877fe700 1 -- 172.21.15.78:0/2630516745 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f57880de8b0 con 0x7f57901085e0 2024-04-25T13:58:35.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.795+0000 7f5796481700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f577c0afc90 0x7f577c0b21a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f578c005ea0 tx=0x7f578c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:35.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:35.798+0000 7f57877fe700 1 -- 172.21.15.78:0/2630516745 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5788023070 con 0x7f57901085e0 2024-04-25T13:58:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.045+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f579002e510 con 0x7f57901085e0 2024-04-25T13:58:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.045+0000 7f57877fe700 1 -- 172.21.15.78:0/2630516745 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f57880a74b0 con 0x7f57901085e0 2024-04-25T13:58:36.045 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:36.045 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f577c0afc90 msgr2=0x7f577c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f577c0afc90 0x7f577c0b21a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f578c005ea0 tx=0x7f578c005e30 comp rx=0 tx=0).stop 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57901085e0 msgr2=0x7f57901a6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57901085e0 0x7f57901a6b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f578800b970 tx=0x7f578800bc80 comp rx=0 tx=0).stop 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 shutdown_connections 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f577c0afc90 0x7f577c0b21a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57901085e0 0x7f57901a6b80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5790103360 0x7f579019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 --2- 172.21.15.78:0/2630516745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5790107c90 0x7f579019c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 >> 172.21.15.78:0/2630516745 conn(0x7f57900fecb0 msgr2=0x7f5790107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 shutdown_connections 2024-04-25T13:58:36.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.047+0000 7f57986e5700 1 -- 172.21.15.78:0/2630516745 wait complete. 2024-04-25T13:58:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:36 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2630516745' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:58:36.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:36 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2630516745' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:58:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:36 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2630516745' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:58:36.771 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:58:36.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:58:36.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 -- 172.21.15.78:0/2373409211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 msgr2=0x7f485c10bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:36.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 --2- 172.21.15.78:0/2373409211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c10bcb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4854009ca0 tx=0x7f485402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 -- 172.21.15.78:0/2373409211 shutdown_connections 2024-04-25T13:58:36.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 --2- 172.21.15.78:0/2373409211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c10bcb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 --2- 172.21.15.78:0/2373409211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f485c101750 0x7f485c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 --2- 172.21.15.78:0/2373409211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c1050a0 0x7f485c1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 -- 172.21.15.78:0/2373409211 >> 172.21.15.78:0/2373409211 conn(0x7f485c0fd370 msgr2=0x7f485c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 -- 172.21.15.78:0/2373409211 shutdown_connections 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 -- 172.21.15.78:0/2373409211 wait complete. 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 Processor -- start 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.882+0000 7f4863f82640 1 -- start start 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f485c101750 0x7f485c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c1050a0 0x7f485c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f485c10f630 con 0x7f485c102250 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f485c10f4b0 con 0x7f485c1050a0 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f485c10f7b0 con 0x7f485c101750 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56882/0 (socket says 172.21.15.78:56882) 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 -- 172.21.15.78:0/1573303687 learned_addr learned my addr 172.21.15.78:0/1573303687 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:36.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4861d1e640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f485c101750 0x7f485c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486251f640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c1050a0 0x7f485c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 -- 172.21.15.78:0/1573303687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f485c101750 msgr2=0x7f485c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f485c101750 0x7f485c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 -- 172.21.15.78:0/1573303687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c1050a0 msgr2=0x7f485c1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c1050a0 0x7f485c1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 -- 172.21.15.78:0/1573303687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c1a1cb0 con 0x7f485c102250 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4861d1e640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f485c101750 0x7f485c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486251f640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c1050a0 0x7f485c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f486151d640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c19a6b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c970 tx=0x7f484c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4852ffd640 1 -- 172.21.15.78:0/1573303687 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c007bb0 con 0x7f485c102250 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4852ffd640 1 -- 172.21.15.78:0/1573303687 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f484c007d50 con 0x7f485c102250 2024-04-25T13:58:36.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c1a1fa0 con 0x7f485c102250 2024-04-25T13:58:36.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4852ffd640 1 -- 172.21.15.78:0/1573303687 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c00f450 con 0x7f485c102250 2024-04-25T13:58:36.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.883+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1a2410 con 0x7f485c102250 2024-04-25T13:58:36.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.884+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485c10aa40 con 0x7f485c102250 2024-04-25T13:58:36.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.886+0000 7f4852ffd640 1 -- 172.21.15.78:0/1573303687 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f484c004f00 con 0x7f485c102250 2024-04-25T13:58:36.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.886+0000 7f4852ffd640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48480afc00 0x7f48480b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:36.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.886+0000 7f4861d1e640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48480afc00 0x7f48480b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:36.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.886+0000 7f4852ffd640 1 -- 172.21.15.78:0/1573303687 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f484c0def60 con 0x7f485c102250 2024-04-25T13:58:36.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.887+0000 7f4861d1e640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48480afc00 0x7f48480b2110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f485c06b320 tx=0x7f4858005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:36.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:36.890+0000 7f4852ffd640 1 -- 172.21.15.78:0/1573303687 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f484c014030 con 0x7f485c102250 2024-04-25T13:58:37.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.070+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f485c02dd20 con 0x7f485c102250 2024-04-25T13:58:37.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.071+0000 7f4852ffd640 1 -- 172.21.15.78:0/1573303687 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f484c0a7a50 con 0x7f485c102250 2024-04-25T13:58:37.071 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:37.072 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:58:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48480afc00 msgr2=0x7f48480b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48480afc00 0x7f48480b2110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f485c06b320 tx=0x7f4858005d30 comp rx=0 tx=0).stop 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 msgr2=0x7f485c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c19a6b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c970 tx=0x7f484c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 shutdown_connections 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f485c1050a0 0x7f485c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48480afc00 0x7f48480b2110 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485c102250 0x7f485c19a6b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 --2- 172.21.15.78:0/1573303687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f485c101750 0x7f485c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 >> 172.21.15.78:0/1573303687 conn(0x7f485c0fd370 msgr2=0x7f485c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 shutdown_connections 2024-04-25T13:58:37.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.073+0000 7f4863f82640 1 -- 172.21.15.78:0/1573303687 wait complete. 2024-04-25T13:58:37.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:58:37.085 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 -- 172.21.15.78:0/1525690724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd0108930 msgr2=0x7ffbd010ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1525690724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd0108930 0x7ffbd010ad50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009ca0 tx=0x7ffbc802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 -- 172.21.15.78:0/1525690724 shutdown_connections 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1525690724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd0108930 0x7ffbd010ad50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1525690724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd0103c30 0x7ffbd0104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1525690724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd01029f0 0x7ffbd01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 -- 172.21.15.78:0/1525690724 >> 172.21.15.78:0/1525690724 conn(0x7ffbd00fdbc0 msgr2=0x7ffbd0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:37.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 -- 172.21.15.78:0/1525690724 shutdown_connections 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.197+0000 7ffbd732e640 1 -- 172.21.15.78:0/1525690724 wait complete. 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 Processor -- start 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 -- start start 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd01029f0 0x7ffbd0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd0103c30 0x7ffbd01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd0108930 0x7ffbd01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd010e520 con 0x7ffbd01029f0 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd010e3a0 con 0x7ffbd0108930 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd732e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd010e6a0 con 0x7ffbd0103c30 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd01029f0 0x7ffbd0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd01029f0 0x7ffbd0199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56900/0 (socket says 172.21.15.78:56900) 2024-04-25T13:58:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 -- 172.21.15.78:0/1217156954 learned_addr learned my addr 172.21.15.78:0/1217156954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd48c9640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd0103c30 0x7ffbd01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd58cb640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd0108930 0x7ffbd01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 -- 172.21.15.78:0/1217156954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd0103c30 msgr2=0x7ffbd01996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd0103c30 0x7ffbd01996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 -- 172.21.15.78:0/1217156954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd0108930 msgr2=0x7ffbd01a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd0108930 0x7ffbd01a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd50ca640 1 -- 172.21.15.78:0/1217156954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd01a0c50 con 0x7ffbd01029f0 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.198+0000 7ffbd48c9640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd0103c30 0x7ffbd01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.199+0000 7ffbd50ca640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd01029f0 0x7ffbd0199180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc005d00 tx=0x7ffbcc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.199+0000 7ffbc67fc640 1 -- 172.21.15.78:0/1217156954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc017070 con 0x7ffbd01029f0 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.199+0000 7ffbc67fc640 1 -- 172.21.15.78:0/1217156954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbcc009c30 con 0x7ffbd01029f0 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.199+0000 7ffbc67fc640 1 -- 172.21.15.78:0/1217156954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00f620 con 0x7ffbd01029f0 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.199+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd01a0f40 con 0x7ffbd01029f0 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.199+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd01a1350 con 0x7ffbd01029f0 2024-04-25T13:58:37.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.200+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd006a160 con 0x7ffbd01029f0 2024-04-25T13:58:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.201+0000 7ffbc67fc640 1 -- 172.21.15.78:0/1217156954 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ffbcc01a070 con 0x7ffbd01029f0 2024-04-25T13:58:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.202+0000 7ffbc67fc640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ffbbc0afb80 0x7ffbbc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.202+0000 7ffbd48c9640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ffbbc0afb80 0x7ffbbc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.202+0000 7ffbc67fc640 1 -- 172.21.15.78:0/1217156954 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ffbcc0de8d0 con 0x7ffbd01029f0 2024-04-25T13:58:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.202+0000 7ffbd48c9640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ffbbc0afb80 0x7ffbbc0b2090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0005fd0 tx=0x7ffbc0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:37.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.205+0000 7ffbc67fc640 1 -- 172.21.15.78:0/1217156954 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbcc009de0 con 0x7ffbd01029f0 2024-04-25T13:58:37.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.383+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffbd002dbf0 con 0x7ffbd01029f0 2024-04-25T13:58:37.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.383+0000 7ffbc67fc640 1 -- 172.21.15.78:0/1217156954 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7ffbcc0a7340 con 0x7ffbd01029f0 2024-04-25T13:58:37.384 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:58:37.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.385+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ffbbc0afb80 msgr2=0x7ffbbc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.385+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ffbbc0afb80 0x7ffbbc0b2090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0005fd0 tx=0x7ffbc0005850 comp rx=0 tx=0).stop 2024-04-25T13:58:37.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.385+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd01029f0 msgr2=0x7ffbd0199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.385+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd01029f0 0x7ffbd0199180 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc005d00 tx=0x7ffbcc00dde0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.385+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 shutdown_connections 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.385+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbd0108930 0x7ffbd01a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.386+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd0103c30 0x7ffbd01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.386+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ffbbc0afb80 0x7ffbbc0b2090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.386+0000 7ffbd732e640 1 --2- 172.21.15.78:0/1217156954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbd01029f0 0x7ffbd0199180 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.386+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 >> 172.21.15.78:0/1217156954 conn(0x7ffbd00fdbc0 msgr2=0x7ffbd0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.386+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 shutdown_connections 2024-04-25T13:58:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.386+0000 7ffbd732e640 1 -- 172.21.15.78:0/1217156954 wait complete. 2024-04-25T13:58:37.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:37 smithi078 ceph-mon[29163]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:37.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:37 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1573303687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:37.397 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 -- 172.21.15.78:0/1156256868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 msgr2=0x7f482c106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 --2- 172.21.15.78:0/1156256868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c106b20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f481c009900 tx=0x7f481c02a410 comp rx=0 tx=0).stop 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 -- 172.21.15.78:0/1156256868 shutdown_connections 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 --2- 172.21.15.78:0/1156256868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f482c109780 0x7f482c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 --2- 172.21.15.78:0/1156256868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f482c102360 0x7f482c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 --2- 172.21.15.78:0/1156256868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c106b20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 -- 172.21.15.78:0/1156256868 >> 172.21.15.78:0/1156256868 conn(0x7f482c0fd350 msgr2=0x7f482c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 -- 172.21.15.78:0/1156256868 shutdown_connections 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.507+0000 7f4833f57640 1 -- 172.21.15.78:0/1156256868 wait complete. 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 Processor -- start 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 -- start start 2024-04-25T13:58:37.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f482c102360 0x7f482c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f482c109780 0x7f482c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10f5e0 con 0x7f482c101e60 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10f460 con 0x7f482c102360 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4833f57640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10f760 con 0x7f482c109780 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56916/0 (socket says 172.21.15.78:56916) 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 -- 172.21.15.78:0/3665508692 learned_addr learned my addr 172.21.15.78:0/3665508692 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f48314f2640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f482c102360 0x7f482c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f48324f4640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f482c109780 0x7f482c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 -- 172.21.15.78:0/3665508692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f482c109780 msgr2=0x7f482c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f482c109780 0x7f482c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 -- 172.21.15.78:0/3665508692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f482c102360 msgr2=0x7f482c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f482c102360 0x7f482c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 -- 172.21.15.78:0/3665508692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c1a1d20 con 0x7f482c101e60 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.508+0000 7f4831cf3640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c19a1e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f481c006010 tx=0x7f481c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.509+0000 7f4822ffd640 1 -- 172.21.15.78:0/3665508692 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c03f030 con 0x7f482c101e60 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.509+0000 7f4822ffd640 1 -- 172.21.15.78:0/3665508692 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f481c003cb0 con 0x7f482c101e60 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.509+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c1a1fb0 con 0x7f482c101e60 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.509+0000 7f4822ffd640 1 -- 172.21.15.78:0/3665508692 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c035070 con 0x7f482c101e60 2024-04-25T13:58:37.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.509+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c1a23c0 con 0x7f482c101e60 2024-04-25T13:58:37.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.510+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f482c0fee70 con 0x7f482c101e60 2024-04-25T13:58:37.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.511+0000 7f4822ffd640 1 -- 172.21.15.78:0/3665508692 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f481c042030 con 0x7f482c101e60 2024-04-25T13:58:37.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.512+0000 7f4822ffd640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48180afb80 0x7f48180b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.512+0000 7f48314f2640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48180afb80 0x7f48180b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.512+0000 7f4822ffd640 1 -- 172.21.15.78:0/3665508692 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f481c0ffa30 con 0x7f482c101e60 2024-04-25T13:58:37.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.513+0000 7f48314f2640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48180afb80 0x7f48180b2090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f482c19b280 tx=0x7f482800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:37.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.516+0000 7f4822ffd640 1 -- 172.21.15.78:0/3665508692 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f481c003e60 con 0x7f482c101e60 2024-04-25T13:58:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:37 smithi112 ceph-mon[32617]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:37 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1573303687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:37.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:37 smithi129 ceph-mon[32569]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:37.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:37 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1573303687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:37.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.695+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f482c02dda0 con 0x7f482c101e60 2024-04-25T13:58:37.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.696+0000 7f4822ffd640 1 -- 172.21.15.78:0/3665508692 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f481c0c7e50 con 0x7f482c101e60 2024-04-25T13:58:37.696 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T13:58:37.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48180afb80 msgr2=0x7f48180b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48180afb80 0x7f48180b2090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f482c19b280 tx=0x7f482800b390 comp rx=0 tx=0).stop 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 msgr2=0x7f482c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c19a1e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f481c006010 tx=0x7f481c004340 comp rx=0 tx=0).stop 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 shutdown_connections 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f482c109780 0x7f482c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f482c102360 0x7f482c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f48180afb80 0x7f48180b2090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 --2- 172.21.15.78:0/3665508692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f482c101e60 0x7f482c19a1e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 >> 172.21.15.78:0/3665508692 conn(0x7f482c0fd350 msgr2=0x7f482c1053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 shutdown_connections 2024-04-25T13:58:37.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.698+0000 7f4833f57640 1 -- 172.21.15.78:0/3665508692 wait complete. 2024-04-25T13:58:37.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 -- 172.21.15.78:0/667925405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1804103c30 msgr2=0x7f1804104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 --2- 172.21.15.78:0/667925405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1804103c30 0x7f1804104040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009900 tx=0x7f17f802a410 comp rx=0 tx=0).stop 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 -- 172.21.15.78:0/667925405 shutdown_connections 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 --2- 172.21.15.78:0/667925405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804108930 0x7f180410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 --2- 172.21.15.78:0/667925405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1804103c30 0x7f1804104040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 --2- 172.21.15.78:0/667925405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18041029f0 0x7f18041036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 -- 172.21.15.78:0/667925405 >> 172.21.15.78:0/667925405 conn(0x7f18040fdc00 msgr2=0x7f1804100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 -- 172.21.15.78:0/667925405 shutdown_connections 2024-04-25T13:58:37.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 -- 172.21.15.78:0/667925405 wait complete. 2024-04-25T13:58:37.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 Processor -- start 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.820+0000 7f1808bf8640 1 -- start start 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18041029f0 0x7f180419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804103c30 0x7f180419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1804108930 0x7f18041a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f180410e570 con 0x7f18041029f0 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f180410e3f0 con 0x7f1804108930 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f180410e6f0 con 0x7f1804103c30 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18041029f0 0x7f180419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18041029f0 0x7f180419a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56936/0 (socket says 172.21.15.78:56936) 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 -- 172.21.15.78:0/3203147759 learned_addr learned my addr 172.21.15.78:0/3203147759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1802d9d640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1804108930 0x7f18041a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 -- 172.21.15.78:0/3203147759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804103c30 msgr2=0x7f180419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804103c30 0x7f180419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 -- 172.21.15.78:0/3203147759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1804108930 msgr2=0x7f18041a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1804108930 0x7f18041a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 -- 172.21.15.78:0/3203147759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18041a1db0 con 0x7f18041029f0 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1802d9d640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1804108930 0x7f18041a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f180259c640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18041029f0 0x7f180419a270 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f17f400ead0 tx=0x7f17f400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f17f37fe640 1 -- 172.21.15.78:0/3203147759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f400ce40 con 0x7f18041029f0 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f17f37fe640 1 -- 172.21.15.78:0/3203147759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f4011920 con 0x7f18041029f0 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18041a20a0 con 0x7f18041029f0 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f17f37fe640 1 -- 172.21.15.78:0/3203147759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f401aae0 con 0x7f18041029f0 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.821+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18041a25f0 con 0x7f18041029f0 2024-04-25T13:58:37.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.822+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18041a28b0 con 0x7f18041029f0 2024-04-25T13:58:37.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.824+0000 7f17f37fe640 1 -- 172.21.15.78:0/3203147759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f17f401ac80 con 0x7f18041029f0 2024-04-25T13:58:37.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.824+0000 7f17f37fe640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17ec0afb80 0x7f17ec0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:37.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.825+0000 7f1801d9b640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17ec0afb80 0x7f17ec0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:37.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.825+0000 7f17f37fe640 1 -- 172.21.15.78:0/3203147759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f17f40df5b0 con 0x7f18041029f0 2024-04-25T13:58:37.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.825+0000 7f1801d9b640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17ec0afb80 0x7f17ec0b2090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f17f8031500 tx=0x7f17f8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:37.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:37.828+0000 7f17f37fe640 1 -- 172.21.15.78:0/3203147759 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17f4055ff0 con 0x7f18041029f0 2024-04-25T13:58:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.007+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18041a2a80 con 0x7f18041029f0 2024-04-25T13:58:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.008+0000 7f17f37fe640 1 -- 172.21.15.78:0/3203147759 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f17f4016030 con 0x7f18041029f0 2024-04-25T13:58:38.008 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T13:58:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17ec0afb80 msgr2=0x7f17ec0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17ec0afb80 0x7f17ec0b2090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f17f8031500 tx=0x7f17f8031400 comp rx=0 tx=0).stop 2024-04-25T13:58:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18041029f0 msgr2=0x7f180419a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18041029f0 0x7f180419a270 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f17f400ead0 tx=0x7f17f400ef90 comp rx=0 tx=0).stop 2024-04-25T13:58:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 shutdown_connections 2024-04-25T13:58:38.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1804108930 0x7f18041a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1804103c30 0x7f180419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17ec0afb80 0x7f17ec0b2090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 --2- 172.21.15.78:0/3203147759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18041029f0 0x7f180419a270 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 >> 172.21.15.78:0/3203147759 conn(0x7f18040fdc00 msgr2=0x7f180406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 shutdown_connections 2024-04-25T13:58:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.010+0000 7f1808bf8640 1 -- 172.21.15.78:0/3203147759 wait complete. 2024-04-25T13:58:38.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:58:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.155+0000 7f8b8f25b640 1 -- 172.21.15.78:0/125657929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810b190 msgr2=0x7f8b8810b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.155+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/125657929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810b190 0x7f8b8810b5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009900 tx=0x7f8b7802a440 comp rx=0 tx=0).stop 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.155+0000 7f8b8f25b640 1 -- 172.21.15.78:0/125657929 shutdown_connections 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.155+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/125657929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8810fe90 0x7f8b881102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.155+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/125657929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810b190 0x7f8b8810b5a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.155+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/125657929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b8810a340 0x7f8b8810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.155+0000 7f8b8f25b640 1 -- 172.21.15.78:0/125657929 >> 172.21.15.78:0/125657929 conn(0x7f8b88070320 msgr2=0x7f8b880727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.156+0000 7f8b8f25b640 1 -- 172.21.15.78:0/125657929 shutdown_connections 2024-04-25T13:58:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.156+0000 7f8b8f25b640 1 -- 172.21.15.78:0/125657929 wait complete. 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 Processor -- start 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 -- start start 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b8810a340 0x7f8b881a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8810b190 0x7f8b881a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810fe90 0x7f8b881b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8811e360 con 0x7f8b8810b190 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8811e1e0 con 0x7f8b8810a340 2024-04-25T13:58:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.157+0000 7f8b8f25b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8811e4e0 con 0x7f8b8810fe90 2024-04-25T13:58:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.158+0000 7f8b8ea5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810fe90 0x7f8b881b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.158+0000 7f8b8ea5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810fe90 0x7f8b881b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:51138/0 (socket says 172.21.15.78:51138) 2024-04-25T13:58:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.158+0000 7f8b8ea5a640 1 -- 172.21.15.78:0/847770023 learned_addr learned my addr 172.21.15.78:0/847770023 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.158+0000 7f8b8da58640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8810b190 0x7f8b881a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.158+0000 7f8b8e259640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b8810a340 0x7f8b881a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.159+0000 7f8b8da58640 1 -- 172.21.15.78:0/847770023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810fe90 msgr2=0x7f8b881b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.159+0000 7f8b8da58640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810fe90 0x7f8b881b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.159+0000 7f8b8da58640 1 -- 172.21.15.78:0/847770023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b8810a340 msgr2=0x7f8b881a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.159+0000 7f8b8da58640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b8810a340 0x7f8b881a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.159+0000 7f8b8da58640 1 -- 172.21.15.78:0/847770023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b881b0aa0 con 0x7f8b8810b190 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.159+0000 7f8b8da58640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8810b190 0x7f8b881a94a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009850 tx=0x7f8b78003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.159+0000 7f8b8ea5a640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810fe90 0x7f8b881b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.160+0000 7f8b7f7fe640 1 -- 172.21.15.78:0/847770023 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7803f070 con 0x7f8b8810b190 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.160+0000 7f8b7f7fe640 1 -- 172.21.15.78:0/847770023 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b78031dd0 con 0x7f8b8810b190 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.160+0000 7f8b7f7fe640 1 -- 172.21.15.78:0/847770023 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78035030 con 0x7f8b8810b190 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.160+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b881b0d30 con 0x7f8b8810b190 2024-04-25T13:58:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.160+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b881b11a0 con 0x7f8b8810b190 2024-04-25T13:58:38.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.161+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8810eca0 con 0x7f8b8810b190 2024-04-25T13:58:38.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.163+0000 7f8b7f7fe640 1 -- 172.21.15.78:0/847770023 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8b78042070 con 0x7f8b8810b190 2024-04-25T13:58:38.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.163+0000 7f8b7f7fe640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b740afcc0 0x7f8b740b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.164+0000 7f8b8e259640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b740afcc0 0x7f8b740b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.164+0000 7f8b7f7fe640 1 -- 172.21.15.78:0/847770023 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8b780fffb0 con 0x7f8b8810b190 2024-04-25T13:58:38.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.164+0000 7f8b8e259640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b740afcc0 0x7f8b740b21d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b88109c10 tx=0x7f8b800094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:38.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.167+0000 7f8b7f7fe640 1 -- 172.21.15.78:0/847770023 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b780c89c0 con 0x7f8b8810b190 2024-04-25T13:58:38.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.390+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b8802db50 con 0x7f8b8810b190 2024-04-25T13:58:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.390+0000 7f8b7f7fe640 1 -- 172.21.15.78:0/847770023 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f8b780c82c0 con 0x7f8b8810b190 2024-04-25T13:58:38.390 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:38.390 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:57:16.081259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24512,24484]},"id":1}]} 2024-04-25T13:58:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 21 2024-04-25T13:58:38.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b740afcc0 msgr2=0x7f8b740b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b740afcc0 0x7f8b740b21d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b88109c10 tx=0x7f8b800094f0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8810b190 msgr2=0x7f8b881a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8810b190 0x7f8b881a94a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009850 tx=0x7f8b78003cf0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 shutdown_connections 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b8810fe90 0x7f8b881b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8b740afcc0 0x7f8b740b21d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b8810b190 0x7f8b881a94a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 --2- 172.21.15.78:0/847770023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b8810a340 0x7f8b881a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 >> 172.21.15.78:0/847770023 conn(0x7f8b88070320 msgr2=0x7f8b8810be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 shutdown_connections 2024-04-25T13:58:38.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.392+0000 7f8b8f25b640 1 -- 172.21.15.78:0/847770023 wait complete. 2024-04-25T13:58:38.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:38 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1217156954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:38 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3665508692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:38 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3203147759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:58:38.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.512+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2451059025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28102230 msgr2=0x7fec2810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.512+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2451059025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28102230 0x7fec2810bc90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec18009900 tx=0x7fec1802a410 comp rx=0 tx=0).stop 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2451059025 shutdown_connections 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2451059025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28102230 0x7fec2810bc90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2451059025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28101730 0x7fec28101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2451059025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec28105080 0x7fec28105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2451059025 >> 172.21.15.78:0/2451059025 conn(0x7fec280fd330 msgr2=0x7fec280ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2451059025 shutdown_connections 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2451059025 wait complete. 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 Processor -- start 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 -- start start 2024-04-25T13:58:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28101730 0x7fec2819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec28102230 0x7fec2819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28105080 0x7fec281a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.513+0000 7fec2f8f9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec2810f610 con 0x7fec28105080 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2f8f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec2810f490 con 0x7fec28101730 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2f8f9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec2810f790 con 0x7fec28102230 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28105080 0x7fec281a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28105080 0x7fec281a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56980/0 (socket says 172.21.15.78:56980) 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 -- 172.21.15.78:0/2131399987 learned_addr learned my addr 172.21.15.78:0/2131399987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2ce94640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec28102230 0x7fec2819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2d695640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28101730 0x7fec2819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 -- 172.21.15.78:0/2131399987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec28102230 msgr2=0x7fec2819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec28102230 0x7fec2819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 -- 172.21.15.78:0/2131399987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28101730 msgr2=0x7fec2819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28101730 0x7fec2819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 -- 172.21.15.78:0/2131399987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec281a1cf0 con 0x7fec28105080 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2d695640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28101730 0x7fec2819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2de96640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28105080 0x7fec281a1680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec18005ab0 tx=0x7fec18004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec267fc640 1 -- 172.21.15.78:0/2131399987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1803f030 con 0x7fec28105080 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec267fc640 1 -- 172.21.15.78:0/2131399987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec18003cb0 con 0x7fec28105080 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec281a1f80 con 0x7fec28105080 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec267fc640 1 -- 172.21.15.78:0/2131399987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18035070 con 0x7fec28105080 2024-04-25T13:58:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.514+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec281a2390 con 0x7fec28105080 2024-04-25T13:58:38.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:38 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1217156954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:38 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3665508692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:38 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3203147759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.515+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2810aa20 con 0x7fec28105080 2024-04-25T13:58:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.517+0000 7fec267fc640 1 -- 172.21.15.78:0/2131399987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fec18042030 con 0x7fec28105080 2024-04-25T13:58:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.517+0000 7fec267fc640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec100afb80 0x7fec100b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.517+0000 7fec2d695640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec100afb80 0x7fec100b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.517+0000 7fec267fc640 1 -- 172.21.15.78:0/2131399987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fec180ffa10 con 0x7fec28105080 2024-04-25T13:58:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.518+0000 7fec2d695640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec100afb80 0x7fec100b2090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec20005c40 tx=0x7fec2000a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.521+0000 7fec267fc640 1 -- 172.21.15.78:0/2131399987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec180c8480 con 0x7fec28105080 2024-04-25T13:58:38.666 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:38 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1217156954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.666 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:38 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3665508692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.666 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:38 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3203147759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:58:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.698+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec2802dd20 con 0x7fec28105080 2024-04-25T13:58:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.699+0000 7fec267fc640 1 -- 172.21.15.78:0/2131399987 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7fec180c7d80 con 0x7fec28105080 2024-04-25T13:58:38.700 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:38.700 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:58:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec100afb80 msgr2=0x7fec100b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec100afb80 0x7fec100b2090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec20005c40 tx=0x7fec2000a430 comp rx=0 tx=0).stop 2024-04-25T13:58:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28105080 msgr2=0x7fec281a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28105080 0x7fec281a1680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec18005ab0 tx=0x7fec18004360 comp rx=0 tx=0).stop 2024-04-25T13:58:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 shutdown_connections 2024-04-25T13:58:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fec100afb80 0x7fec100b2090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec28105080 0x7fec281a1680 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec28102230 0x7fec2819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 --2- 172.21.15.78:0/2131399987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec28101730 0x7fec2819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 >> 172.21.15.78:0/2131399987 conn(0x7fec280fd330 msgr2=0x7fec28107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:38.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 shutdown_connections 2024-04-25T13:58:38.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.701+0000 7fec2f8f9640 1 -- 172.21.15.78:0/2131399987 wait complete. 2024-04-25T13:58:38.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:58:38.715 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:58:38.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 -- 172.21.15.78:0/3014659281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 msgr2=0x7f09c4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/3014659281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c4109360 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402a410 comp rx=0 tx=0).stop 2024-04-25T13:58:38.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 -- 172.21.15.78:0/3014659281 shutdown_connections 2024-04-25T13:58:38.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/3014659281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09c41098a0 0x7f09c410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/3014659281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c4109360 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/3014659281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c41045e0 0x7f09c4106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 -- 172.21.15.78:0/3014659281 >> 172.21.15.78:0/3014659281 conn(0x7f09c40fdbc0 msgr2=0x7f09c4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 -- 172.21.15.78:0/3014659281 shutdown_connections 2024-04-25T13:58:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.826+0000 7f09cb7b2640 1 -- 172.21.15.78:0/3014659281 wait complete. 2024-04-25T13:58:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 Processor -- start 2024-04-25T13:58:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 -- start start 2024-04-25T13:58:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09c41045e0 0x7f09c419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c41098a0 0x7f09c41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09c410f620 con 0x7f09c4106f40 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09c410f4a0 con 0x7f09c41098a0 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09cb7b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f09c410f7a0 con 0x7f09c41045e0 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56992/0 (socket says 172.21.15.78:56992) 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 -- 172.21.15.78:0/83783983 learned_addr learned my addr 172.21.15.78:0/83783983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:38.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c9d4f640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c41098a0 0x7f09c41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c954e640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09c41045e0 0x7f09c419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 -- 172.21.15.78:0/83783983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09c41045e0 msgr2=0x7f09c419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09c41045e0 0x7f09c419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 -- 172.21.15.78:0/83783983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c41098a0 msgr2=0x7f09c41a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c41098a0 0x7f09c41a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c8d4d640 1 -- 172.21.15.78:0/83783983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c41a1d90 con 0x7f09c4106f40 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.827+0000 7f09c954e640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09c41045e0 0x7f09c419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:38.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.828+0000 7f09c8d4d640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c419a790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f09b4006010 tx=0x7f09b40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:38.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.828+0000 7f09ba7fc640 1 -- 172.21.15.78:0/83783983 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b403f030 con 0x7f09c4106f40 2024-04-25T13:58:38.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.828+0000 7f09ba7fc640 1 -- 172.21.15.78:0/83783983 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b4003cb0 con 0x7f09c4106f40 2024-04-25T13:58:38.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.828+0000 7f09ba7fc640 1 -- 172.21.15.78:0/83783983 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4035070 con 0x7f09c4106f40 2024-04-25T13:58:38.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.828+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c41a2020 con 0x7f09c4106f40 2024-04-25T13:58:38.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.828+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c41a2510 con 0x7f09c4106f40 2024-04-25T13:58:38.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.829+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c40ff6e0 con 0x7f09c4106f40 2024-04-25T13:58:38.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.830+0000 7f09ba7fc640 1 -- 172.21.15.78:0/83783983 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f09b4042030 con 0x7f09c4106f40 2024-04-25T13:58:38.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.831+0000 7f09ba7fc640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09b00afb80 0x7f09b00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:38.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.831+0000 7f09c954e640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09b00afb80 0x7f09b00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:38.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.831+0000 7f09ba7fc640 1 -- 172.21.15.78:0/83783983 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f09b40ff600 con 0x7f09c4106f40 2024-04-25T13:58:38.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.831+0000 7f09c954e640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09b00afb80 0x7f09b00b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f09c00058c0 tx=0x7f09c0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:38.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:38.835+0000 7f09ba7fc640 1 -- 172.21.15.78:0/83783983 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09b40c8070 con 0x7f09c4106f40 2024-04-25T13:58:39.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.010+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09c40637a0 con 0x7f09c4106f40 2024-04-25T13:58:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.011+0000 7f09ba7fc640 1 -- 172.21.15.78:0/83783983 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f09b40d00f0 con 0x7f09c4106f40 2024-04-25T13:58:39.011 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:58:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09b00afb80 msgr2=0x7f09b00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09b00afb80 0x7f09b00b2090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f09c00058c0 tx=0x7f09c0005d30 comp rx=0 tx=0).stop 2024-04-25T13:58:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 msgr2=0x7f09c419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c419a790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f09b4006010 tx=0x7f09b40043b0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 shutdown_connections 2024-04-25T13:58:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c41098a0 0x7f09c41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f09b00afb80 0x7f09b00b2090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09c4106f40 0x7f09c419a790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 --2- 172.21.15.78:0/83783983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09c41045e0 0x7f09c419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 >> 172.21.15.78:0/83783983 conn(0x7f09c40fdbc0 msgr2=0x7f09c41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 shutdown_connections 2024-04-25T13:58:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.013+0000 7f09cb7b2640 1 -- 172.21.15.78:0/83783983 wait complete. 2024-04-25T13:58:39.024 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:58:39.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.135+0000 7f2a1286e640 1 -- 172.21.15.78:0/2659204824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 msgr2=0x7f2a0c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.135+0000 7f2a1286e640 1 --2- 172.21.15.78:0/2659204824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c1090f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009900 tx=0x7f2a0002a410 comp rx=0 tx=0).stop 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.135+0000 7f2a1286e640 1 -- 172.21.15.78:0/2659204824 shutdown_connections 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.135+0000 7f2a1286e640 1 --2- 172.21.15.78:0/2659204824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c109780 0x7f2a0c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.135+0000 7f2a1286e640 1 --2- 172.21.15.78:0/2659204824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c1090f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.136+0000 7f2a1286e640 1 --2- 172.21.15.78:0/2659204824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c101e10 0x7f2a0c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.136+0000 7f2a1286e640 1 -- 172.21.15.78:0/2659204824 >> 172.21.15.78:0/2659204824 conn(0x7f2a0c0fd340 msgr2=0x7f2a0c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.136+0000 7f2a1286e640 1 -- 172.21.15.78:0/2659204824 shutdown_connections 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.136+0000 7f2a1286e640 1 -- 172.21.15.78:0/2659204824 wait complete. 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.136+0000 7f2a1286e640 1 Processor -- start 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.136+0000 7f2a1286e640 1 -- start start 2024-04-25T13:58:39.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a1286e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c101e10 0x7f2a0c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a1286e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a1286e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c109780 0x7f2a0c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a1286e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10f610 con 0x7f2a0c106c10 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a1286e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10f490 con 0x7f2a0c101e10 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a1286e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c10f790 con 0x7f2a0c109780 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57022/0 (socket says 172.21.15.78:57022) 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 -- 172.21.15.78:0/1286321763 learned_addr learned my addr 172.21.15.78:0/1286321763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0bfff640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c101e10 0x7f2a0c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a10e0b640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c109780 0x7f2a0c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 -- 172.21.15.78:0/1286321763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c109780 msgr2=0x7f2a0c1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c109780 0x7f2a0c1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 -- 172.21.15.78:0/1286321763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c101e10 msgr2=0x7f2a0c19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c101e10 0x7f2a0c19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 -- 172.21.15.78:0/1286321763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c1a1d30 con 0x7f2a0c106c10 2024-04-25T13:58:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.137+0000 7f2a0b7fe640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c19a730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2a00006010 tx=0x7f2a00031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.138+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0003f030 con 0x7f2a0c106c10 2024-04-25T13:58:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.138+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a00003aa0 con 0x7f2a0c106c10 2024-04-25T13:58:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.138+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c1a1fc0 con 0x7f2a0c106c10 2024-04-25T13:58:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.138+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00035070 con 0x7f2a0c106c10 2024-04-25T13:58:39.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.138+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a2420 con 0x7f2a0c106c10 2024-04-25T13:58:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.139+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c0fee60 con 0x7f2a0c106c10 2024-04-25T13:58:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.140+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2a00042030 con 0x7f2a0c106c10 2024-04-25T13:58:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.141+0000 7f2a097fa640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc00 0x7f29f40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.141+0000 7f2a0bfff640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc00 0x7f29f40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.141+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2a000ffa90 con 0x7f2a0c106c10 2024-04-25T13:58:39.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.142+0000 7f2a0bfff640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc00 0x7f29f40b2110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c104ae0 tx=0x7f29fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:39.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.145+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a000c8500 con 0x7f2a0c106c10 2024-04-25T13:58:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.214+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2a000c4090 con 0x7f2a0c106c10 2024-04-25T13:58:39.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.330+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2a0c02dce0 con 0x7f2a0c106c10 2024-04-25T13:58:39.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.331+0000 7f2a097fa640 1 -- 172.21.15.78:0/1286321763 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f2a000f5b90 con 0x7f2a0c106c10 2024-04-25T13:58:39.331 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T13:58:39.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.332+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc00 msgr2=0x7f29f40b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.332+0000 7f2a1286e640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc00 0x7f29f40b2110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c104ae0 tx=0x7f29fc005f90 comp rx=0 tx=0).stop 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.332+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 msgr2=0x7f2a0c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.332+0000 7f2a1286e640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c19a730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2a00006010 tx=0x7f2a00031a50 comp rx=0 tx=0).stop 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 shutdown_connections 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a0c109780 0x7f2a0c1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29f40afc00 0x7f29f40b2110 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a0c106c10 0x7f2a0c19a730 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 --2- 172.21.15.78:0/1286321763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a0c101e10 0x7f2a0c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 >> 172.21.15.78:0/1286321763 conn(0x7f2a0c0fd340 msgr2=0x7f2a0c107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 shutdown_connections 2024-04-25T13:58:39.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.333+0000 7f2a1286e640 1 -- 172.21.15.78:0/1286321763 wait complete. 2024-04-25T13:58:39.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:39 smithi078 ceph-mon[29163]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:39.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:39 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/847770023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:39.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:39 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2131399987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:39.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:39 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/83783983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:58:39.344 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:58:39.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 -- 172.21.15.78:0/1482071445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 msgr2=0x7f43f0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 --2- 172.21.15.78:0/1482071445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f0102e80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009900 tx=0x7f43e402a410 comp rx=0 tx=0).stop 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 -- 172.21.15.78:0/1482071445 shutdown_connections 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 --2- 172.21.15.78:0/1482071445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f01080c0 0x7f43f010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 --2- 172.21.15.78:0/1482071445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43f01033c0 0x7f43f01037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 --2- 172.21.15.78:0/1482071445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f0102e80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 -- 172.21.15.78:0/1482071445 >> 172.21.15.78:0/1482071445 conn(0x7f43f00fd350 msgr2=0x7f43f00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 -- 172.21.15.78:0/1482071445 shutdown_connections 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.454+0000 7f43f7034640 1 -- 172.21.15.78:0/1482071445 wait complete. 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 Processor -- start 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 -- start start 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f00ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43f01033c0 0x7f43f00fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f01080c0 0x7f43f00fddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010dc70 con 0x7f43f0102180 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010daf0 con 0x7f43f01080c0 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f7034640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010ddf0 con 0x7f43f01033c0 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f4dd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f00ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f4dd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f00ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57040/0 (socket says 172.21.15.78:57040) 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f4dd0640 1 -- 172.21.15.78:0/4063928731 learned_addr learned my addr 172.21.15.78:0/4063928731 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.455+0000 7f43f55d1640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f01080c0 0x7f43f00fddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f4dd0640 1 -- 172.21.15.78:0/4063928731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43f01033c0 msgr2=0x7f43f00fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f4dd0640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43f01033c0 0x7f43f00fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f4dd0640 1 -- 172.21.15.78:0/4063928731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f01080c0 msgr2=0x7f43f00fddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f4dd0640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f01080c0 0x7f43f00fddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f4dd0640 1 -- 172.21.15.78:0/4063928731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f00fe4f0 con 0x7f43f0102180 2024-04-25T13:58:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f4dd0640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f00ff1e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43e4006010 tx=0x7f43e4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43edffb640 1 -- 172.21.15.78:0/4063928731 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e403f030 con 0x7f43f0102180 2024-04-25T13:58:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43edffb640 1 -- 172.21.15.78:0/4063928731 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e4003aa0 con 0x7f43f0102180 2024-04-25T13:58:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f01a42e0 con 0x7f43f0102180 2024-04-25T13:58:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43edffb640 1 -- 172.21.15.78:0/4063928731 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4035070 con 0x7f43f0102180 2024-04-25T13:58:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.456+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01a46e0 con 0x7f43f0102180 2024-04-25T13:58:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.457+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f006a160 con 0x7f43f0102180 2024-04-25T13:58:39.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.459+0000 7f43edffb640 1 -- 172.21.15.78:0/4063928731 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f43e4042030 con 0x7f43f0102180 2024-04-25T13:58:39.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.460+0000 7f43edffb640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43e00afaf0 0x7f43e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.460+0000 7f43effff640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43e00afaf0 0x7f43e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.460+0000 7f43edffb640 1 -- 172.21.15.78:0/4063928731 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f43e40ff670 con 0x7f43f0102180 2024-04-25T13:58:39.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.461+0000 7f43effff640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43e00afaf0 0x7f43e00b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43f00feb80 tx=0x7f43dc005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:39.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.464+0000 7f43edffb640 1 -- 172.21.15.78:0/4063928731 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43e40c8240 con 0x7f43f0102180 2024-04-25T13:58:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:39 smithi112 ceph-mon[32617]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:39 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/847770023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:39 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2131399987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:39 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/83783983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:58:39.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.637+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43f01a49e0 con 0x7f43f0102180 2024-04-25T13:58:39.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.638+0000 7f43edffb640 1 -- 172.21.15.78:0/4063928731 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f43e40c7b40 con 0x7f43f0102180 2024-04-25T13:58:39.638 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T13:58:39.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43e00afaf0 msgr2=0x7f43e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43e00afaf0 0x7f43e00b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43f00feb80 tx=0x7f43dc005bf0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 msgr2=0x7f43f00ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f00ff1e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43e4006010 tx=0x7f43e4031a50 comp rx=0 tx=0).stop 2024-04-25T13:58:39.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 shutdown_connections 2024-04-25T13:58:39.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43f01080c0 0x7f43f00fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43f01033c0 0x7f43f00fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43e00afaf0 0x7f43e00b2000 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 --2- 172.21.15.78:0/4063928731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43f0102180 0x7f43f00ff1e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 >> 172.21.15.78:0/4063928731 conn(0x7f43f00fd350 msgr2=0x7f43f01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:39.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 shutdown_connections 2024-04-25T13:58:39.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.640+0000 7f43f7034640 1 -- 172.21.15.78:0/4063928731 wait complete. 2024-04-25T13:58:39.651 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:58:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:39 smithi129 ceph-mon[32569]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:39 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/847770023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:39 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2131399987' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:39 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/83783983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.758+0000 7fdaebfff640 1 -- 172.21.15.78:0/734296802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 msgr2=0x7fdaec1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.758+0000 7fdaebfff640 1 --2- 172.21.15.78:0/734296802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec1019f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009900 tx=0x7fdae002a410 comp rx=0 tx=0).stop 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.758+0000 7fdaebfff640 1 -- 172.21.15.78:0/734296802 shutdown_connections 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.758+0000 7fdaebfff640 1 --2- 172.21.15.78:0/734296802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaec102080 0x7fdaec10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.758+0000 7fdaebfff640 1 --2- 172.21.15.78:0/734296802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec1019f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.758+0000 7fdaebfff640 1 --2- 172.21.15.78:0/734296802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaec104ed0 0x7fdaec1052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.758+0000 7fdaebfff640 1 -- 172.21.15.78:0/734296802 >> 172.21.15.78:0/734296802 conn(0x7fdaec0fd1a0 msgr2=0x7fdaec0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.759+0000 7fdaebfff640 1 -- 172.21.15.78:0/734296802 shutdown_connections 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.759+0000 7fdaebfff640 1 -- 172.21.15.78:0/734296802 wait complete. 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.759+0000 7fdaebfff640 1 Processor -- start 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.759+0000 7fdaebfff640 1 -- start start 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaec102080 0x7fdaec102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaebfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaec104ed0 0x7fdaec102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaebfff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec10f460 con 0x7fdaec101580 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaebfff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec10f2e0 con 0x7fdaec102080 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaebfff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec10f5e0 con 0x7fdaec104ed0 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57062/0 (socket says 172.21.15.78:57062) 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 -- 172.21.15.78:0/179353571 learned_addr learned my addr 172.21.15.78:0/179353571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeb7fe640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaec104ed0 0x7fdaec102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaea7fc640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaec102080 0x7fdaec102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 -- 172.21.15.78:0/179353571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaec104ed0 msgr2=0x7fdaec102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaec104ed0 0x7fdaec102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 -- 172.21.15.78:0/179353571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaec102080 msgr2=0x7fdaec102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaec102080 0x7fdaec102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 -- 172.21.15.78:0/179353571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec103390 con 0x7fdaec101580 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaeaffd640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec104080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00ea50 tx=0x7fdadc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdad3fff640 1 -- 172.21.15.78:0/179353571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00cdc0 con 0x7fdaec101580 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec1a40d0 con 0x7fdaec101580 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdad3fff640 1 -- 172.21.15.78:0/179353571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdadc011920 con 0x7fdaec101580 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.760+0000 7fdad3fff640 1 -- 172.21.15.78:0/179353571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc01aae0 con 0x7fdaec101580 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.761+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec1a4570 con 0x7fdaec101580 2024-04-25T13:58:39.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.761+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec108860 con 0x7fdaec101580 2024-04-25T13:58:39.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.763+0000 7fdad3fff640 1 -- 172.21.15.78:0/179353571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdadc01ac80 con 0x7fdaec101580 2024-04-25T13:58:39.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.764+0000 7fdad3fff640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdad40afae0 0x7fdad40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.764+0000 7fdad3fff640 1 -- 172.21.15.78:0/179353571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdadc0df400 con 0x7fdaec101580 2024-04-25T13:58:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.764+0000 7fdaea7fc640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdad40afae0 0x7fdad40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.765+0000 7fdaea7fc640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdad40afae0 0x7fdad40b1ff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdae002a930 tx=0x7fdae0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.768+0000 7fdad3fff640 1 -- 172.21.15.78:0/179353571 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdadc055ff0 con 0x7fdaec101580 2024-04-25T13:58:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.983+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaec02db50 con 0x7fdaec101580 2024-04-25T13:58:39.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.984+0000 7fdad3fff640 1 -- 172.21.15.78:0/179353571 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7fdadc016030 con 0x7fdaec101580 2024-04-25T13:58:39.984 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:39.984 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:57:16.081259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24512,24484]},"id":1}]} 2024-04-25T13:58:39.985 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 21 2024-04-25T13:58:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdad40afae0 msgr2=0x7fdad40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdad40afae0 0x7fdad40b1ff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdae002a930 tx=0x7fdae0000bc0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 msgr2=0x7fdaec104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec104080 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00ea50 tx=0x7fdadc00ef10 comp rx=0 tx=0).stop 2024-04-25T13:58:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 shutdown_connections 2024-04-25T13:58:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaec104ed0 0x7fdaec102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaec102080 0x7fdaec102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdad40afae0 0x7fdad40b1ff0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 --2- 172.21.15.78:0/179353571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaec101580 0x7fdaec104080 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 >> 172.21.15.78:0/179353571 conn(0x7fdaec0fd1a0 msgr2=0x7fdaec105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 shutdown_connections 2024-04-25T13:58:39.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:39.986+0000 7fdaebfff640 1 -- 172.21.15.78:0/179353571 wait complete. 2024-04-25T13:58:39.998 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:58:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 -- 172.21.15.78:0/3986893220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 msgr2=0x7fbee010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3986893220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee010ad60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009ca0 tx=0x7fbed802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 -- 172.21.15.78:0/3986893220 shutdown_connections 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3986893220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee010ad60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3986893220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbee0103c40 0x7fbee0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3986893220 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee0102a00 0x7fbee0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 -- 172.21.15.78:0/3986893220 >> 172.21.15.78:0/3986893220 conn(0x7fbee00fdbf0 msgr2=0x7fbee01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 -- 172.21.15.78:0/3986893220 shutdown_connections 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.108+0000 7fbee82ee640 1 -- 172.21.15.78:0/3986893220 wait complete. 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 Processor -- start 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 -- start start 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbee0102a00 0x7fbee0075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee0103c40 0x7fbee019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee01a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010e530 con 0x7fbee0108940 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010e3b0 con 0x7fbee0103c40 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee82ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010e6b0 con 0x7fbee0102a00 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee01a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee01a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57072/0 (socket says 172.21.15.78:57072) 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 -- 172.21.15.78:0/3696278188 learned_addr learned my addr 172.21.15.78:0/3696278188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee608a640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbee0102a00 0x7fbee0075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 -- 172.21.15.78:0/3696278188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbee0102a00 msgr2=0x7fbee0075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbee0102a00 0x7fbee0075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee5889640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee0103c40 0x7fbee019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 -- 172.21.15.78:0/3696278188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee0103c40 msgr2=0x7fbee019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee0103c40 0x7fbee019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 -- 172.21.15.78:0/3696278188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee01a1c90 con 0x7fbee0108940 2024-04-25T13:58:40.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee608a640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbee0102a00 0x7fbee0075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.109+0000 7fbee688b640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee01a1470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbed80097b0 tx=0x7fbed8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.110+0000 7fbed77fe640 1 -- 172.21.15.78:0/3696278188 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8004240 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.110+0000 7fbed77fe640 1 -- 172.21.15.78:0/3696278188 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed802f380 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.110+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee01a1f20 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.110+0000 7fbed77fe640 1 -- 172.21.15.78:0/3696278188 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed802fc30 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.110+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a2380 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.111+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee006a190 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.112+0000 7fbed77fe640 1 -- 172.21.15.78:0/3696278188 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbed80043e0 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.113+0000 7fbed77fe640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbecc0afaf0 0x7fbecc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.113+0000 7fbee608a640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbecc0afaf0 0x7fbecc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.113+0000 7fbed77fe640 1 -- 172.21.15.78:0/3696278188 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbed8100580 con 0x7fbee0108940 2024-04-25T13:58:40.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.114+0000 7fbee608a640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbecc0afaf0 0x7fbecc0b2000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbed0005d00 tx=0x7fbed0005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.117+0000 7fbed77fe640 1 -- 172.21.15.78:0/3696278188 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbed80c9150 con 0x7fbee0108940 2024-04-25T13:58:40.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.303+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbee002dc10 con 0x7fbee0108940 2024-04-25T13:58:40.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.305+0000 7fbed77fe640 1 -- 172.21.15.78:0/3696278188 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7fbed80c8a50 con 0x7fbee0108940 2024-04-25T13:58:40.307 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:40.307 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:58:40.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbecc0afaf0 msgr2=0x7fbecc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbecc0afaf0 0x7fbecc0b2000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbed0005d00 tx=0x7fbed0005c90 comp rx=0 tx=0).stop 2024-04-25T13:58:40.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 msgr2=0x7fbee01a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee01a1470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbed80097b0 tx=0x7fbed8003c20 comp rx=0 tx=0).stop 2024-04-25T13:58:40.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 shutdown_connections 2024-04-25T13:58:40.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbecc0afaf0 0x7fbecc0b2000 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee0108940 0x7fbee01a1470 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee0103c40 0x7fbee019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 --2- 172.21.15.78:0/3696278188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbee0102a00 0x7fbee0075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 >> 172.21.15.78:0/3696278188 conn(0x7fbee00fdbf0 msgr2=0x7fbee0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 shutdown_connections 2024-04-25T13:58:40.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.307+0000 7fbee82ee640 1 -- 172.21.15.78:0/3696278188 wait complete. 2024-04-25T13:58:40.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:40 smithi078 ceph-mon[29163]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:58:40.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:40 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1286321763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:58:40.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:40 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4063928731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:58:40.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:40 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/179353571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:40.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 -- 172.21.15.78:0/1537879960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 msgr2=0x7f76ac10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1537879960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac10ad20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009ca0 tx=0x7f76a802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 -- 172.21.15.78:0/1537879960 shutdown_connections 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1537879960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac10ad20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1537879960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76ac103c00 0x7f76ac104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1537879960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76ac1029c0 0x7f76ac1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 -- 172.21.15.78:0/1537879960 >> 172.21.15.78:0/1537879960 conn(0x7f76ac0fdbd0 msgr2=0x7f76ac100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 -- 172.21.15.78:0/1537879960 shutdown_connections 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 -- 172.21.15.78:0/1537879960 wait complete. 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 Processor -- start 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.426+0000 7f76b3a92640 1 -- start start 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76ac1029c0 0x7f76ac19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76ac103c00 0x7f76ac19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac10f5f0 con 0x7f76ac108900 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac10f470 con 0x7f76ac103c00 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac10f770 con 0x7f76ac1029c0 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac1a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57092/0 (socket says 172.21.15.78:57092) 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 -- 172.21.15.78:0/1933374576 learned_addr learned my addr 172.21.15.78:0/1933374576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b102d640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76ac103c00 0x7f76ac19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 -- 172.21.15.78:0/1933374576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76ac1029c0 msgr2=0x7f76ac19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b182e640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76ac1029c0 0x7f76ac19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76ac1029c0 0x7f76ac19a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 -- 172.21.15.78:0/1933374576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76ac103c00 msgr2=0x7f76ac19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76ac103c00 0x7f76ac19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 -- 172.21.15.78:0/1933374576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac1a1c20 con 0x7f76ac108900 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b182e640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76ac1029c0 0x7f76ac19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b202f640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac1a1660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097b0 tx=0x7f76a8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76a2ffd640 1 -- 172.21.15.78:0/1933374576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004240 con 0x7f76ac108900 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76a2ffd640 1 -- 172.21.15.78:0/1933374576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a802f380 con 0x7f76ac108900 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76a2ffd640 1 -- 172.21.15.78:0/1933374576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a802fc30 con 0x7f76ac108900 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac1a1eb0 con 0x7f76ac108900 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.427+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac1a2310 con 0x7f76ac108900 2024-04-25T13:58:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.428+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac06afd0 con 0x7f76ac108900 2024-04-25T13:58:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.430+0000 7f76a2ffd640 1 -- 172.21.15.78:0/1933374576 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f76a80043e0 con 0x7f76ac108900 2024-04-25T13:58:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.431+0000 7f76a2ffd640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76980afaf0 0x7f76980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.431+0000 7f76b182e640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76980afaf0 0x7f76980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.431+0000 7f76a2ffd640 1 -- 172.21.15.78:0/1933374576 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f76a81005a0 con 0x7f76ac108900 2024-04-25T13:58:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.432+0000 7f76b182e640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76980afaf0 0x7f76980b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f76ac1035c0 tx=0x7f76a4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.435+0000 7f76a2ffd640 1 -- 172.21.15.78:0/1933374576 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76ac06afd0 con 0x7f76ac108900 2024-04-25T13:58:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:40 smithi112 ceph-mon[32617]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:58:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:40 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1286321763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:58:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:40 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4063928731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:58:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:40 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/179353571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:40.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.650+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76ac02dc10 con 0x7f76ac108900 2024-04-25T13:58:40.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.651+0000 7f76a2ffd640 1 -- 172.21.15.78:0/1933374576 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f76a80c9170 con 0x7f76ac108900 2024-04-25T13:58:40.651 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:40.651 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:57:16.081259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24512,24484]},"id":1}]} 2024-04-25T13:58:40.651 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 21 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76980afaf0 msgr2=0x7f76980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76980afaf0 0x7f76980b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f76ac1035c0 tx=0x7f76a4005db0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 msgr2=0x7f76ac1a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac1a1660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097b0 tx=0x7f76a8003c20 comp rx=0 tx=0).stop 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 shutdown_connections 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76980afaf0 0x7f76980b2000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76ac108900 0x7f76ac1a1660 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76ac103c00 0x7f76ac19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 --2- 172.21.15.78:0/1933374576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76ac1029c0 0x7f76ac19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 >> 172.21.15.78:0/1933374576 conn(0x7f76ac0fdbd0 msgr2=0x7f76ac1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 shutdown_connections 2024-04-25T13:58:40.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.653+0000 7f76b3a92640 1 -- 172.21.15.78:0/1933374576 wait complete. 2024-04-25T13:58:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:40 smithi129 ceph-mon[32569]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:58:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:40 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1286321763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:58:40.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:40 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4063928731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:58:40.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:40 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/179353571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:40.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:57:11.484343+0000', 'modified': '2024-04-25T13:57:16.081259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24484, 'mds_2': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/2622963442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 2622963442}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 2622963442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.129:6835/2365161980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 2365161980}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 2365161980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6837/750801105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 750801105}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 750801105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6837/4152465978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 4152465978}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 4152465978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.129:6839/1046146051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 1046146051}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 1046146051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/67760277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 67760277}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 67760277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24454, 24512, 24484]} 2024-04-25T13:58:40.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T13:58:40.665 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 -- 172.21.15.78:0/295906480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c103c20 msgr2=0x7f390c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 --2- 172.21.15.78:0/295906480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c103c20 0x7f390c104030 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3900009900 tx=0x7f390002a410 comp rx=0 tx=0).stop 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 -- 172.21.15.78:0/295906480 shutdown_connections 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 --2- 172.21.15.78:0/295906480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f390c108920 0x7f390c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 --2- 172.21.15.78:0/295906480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c103c20 0x7f390c104030 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 --2- 172.21.15.78:0/295906480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f390c1029e0 0x7f390c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 -- 172.21.15.78:0/295906480 >> 172.21.15.78:0/295906480 conn(0x7f390c0fdbd0 msgr2=0x7f390c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 -- 172.21.15.78:0/295906480 shutdown_connections 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 -- 172.21.15.78:0/295906480 wait complete. 2024-04-25T13:58:40.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 Processor -- start 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 -- start start 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f390c1029e0 0x7f390c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.774+0000 7f3912e84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f390c103c20 0x7f390c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3912e84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c108920 0x7f390c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3912e84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10e4e0 con 0x7f390c108920 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3912e84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10e360 con 0x7f390c1029e0 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3912e84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10e660 con 0x7f390c103c20 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c108920 0x7f390c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c108920 0x7f390c1a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57114/0 (socket says 172.21.15.78:57114) 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 -- 172.21.15.78:0/772128298 learned_addr learned my addr 172.21.15.78:0/772128298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3910c20640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f390c1029e0 0x7f390c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f390bfff640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f390c103c20 0x7f390c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 -- 172.21.15.78:0/772128298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f390c103c20 msgr2=0x7f390c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f390c103c20 0x7f390c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 -- 172.21.15.78:0/772128298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f390c1029e0 msgr2=0x7f390c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f390c1029e0 0x7f390c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 -- 172.21.15.78:0/772128298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c1a2070 con 0x7f390c108920 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3911421640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c108920 0x7f390c1a1a00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f390400cad0 tx=0x7f390400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3909ffb640 1 -- 172.21.15.78:0/772128298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390400ae20 con 0x7f390c108920 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3909ffb640 1 -- 172.21.15.78:0/772128298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3904005020 con 0x7f390c108920 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c1a2360 con 0x7f390c108920 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3909ffb640 1 -- 172.21.15.78:0/772128298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390400f6b0 con 0x7f390c108920 2024-04-25T13:58:40.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.775+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1a2860 con 0x7f390c108920 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.778+0000 7f3919e07740 1 Processor -- start 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.778+0000 7f3909ffb640 1 -- 172.21.15.78:0/772128298 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f39040051d0 con 0x7f390c108920 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3919e07740 1 -- start start 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3919e07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd7d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3919e07740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd7d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3919e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3919e07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560d8dce2740 con 0x560d8d9dc270 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3919e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d8dc7e3a0 con 0x560d8d9bb590 2024-04-25T13:58:40.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3919e07740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d8dcde700 con 0x560d8d9b33b0 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd7d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd7d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57126/0 (socket says 172.21.15.78:57126) 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 -- 172.21.15.78:0/1441953219 learned_addr learned my addr 172.21.15.78:0/1441953219 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3910c20640 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd7d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3909ffb640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38f40afaf0 0x7f38f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 -- 172.21.15.78:0/1441953219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 msgr2=0x560d8dd7d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd7d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 -- 172.21.15.78:0/1441953219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 msgr2=0x560d8dd847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 -- 172.21.15.78:0/1441953219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d8dc9a750 con 0x560d8d9dc270 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3910c20640 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd7d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3911421640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38f40afaf0 0x7f38f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f390bfff640 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd7d370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f390002a930 tx=0x7f3900005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d428071e0d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.779+0000 7f3909ffb640 1 -- 172.21.15.78:0/772128298 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f39040ded30 con 0x7f390c108920 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.780+0000 7f38f2ffd640 1 -- 172.21.15.78:0/1441953219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900041070 con 0x560d8d9dc270 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.780+0000 7f38f2ffd640 1 -- 172.21.15.78:0/1441953219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39000317a0 con 0x560d8d9dc270 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.780+0000 7f38f2ffd640 1 -- 172.21.15.78:0/1441953219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900035070 con 0x560d8d9dc270 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.780+0000 7f3919e07740 1 -- 172.21.15.78:0/1441953219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 msgr2=0x560d8dd7d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.780+0000 7f3911421640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38f40afaf0 0x7f38f40b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39040a4470 tx=0x7f3904075d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.780+0000 7f3919e07740 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd7d370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f390002a930 tx=0x7f3900005e50 comp rx=0 tx=0).stop 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- 172.21.15.78:0/1441953219 shutdown_connections 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd7d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 --2- 172.21.15.78:0/1441953219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd7d370 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- 172.21.15.78:0/1441953219 >> 172.21.15.78:0/1441953219 conn(0x560d8d9d03c0 msgr2=0x560d8dd78d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- 172.21.15.78:0/1441953219 shutdown_connections 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- 172.21.15.78:0/1441953219 wait complete. 2024-04-25T13:58:40.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- start start 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 Processor -- start 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9b33b0 0x560d8dd87290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9dc270 0x560d8dd8e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd851e0 con 0x560d8d9b33b0 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d8dc69fe0 con 0x560d8d9bb590 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.781+0000 7f3919e07740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd85360 con 0x560d8d9dc270 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f3919e07740 10 client.0 _ll_register_callbacks cb 0x560d8dc80150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9b33b0 0x560d8dd87290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9b33b0 0x560d8dd87290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57140/0 (socket says 172.21.15.78:57140) 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 -- 172.21.15.78:0/1835311903 learned_addr learned my addr 172.21.15.78:0/1835311903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f3910c20640 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9dc270 0x560d8dd8e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 -- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9dc270 msgr2=0x560d8dd8e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9dc270 0x560d8dd8e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 -- 172.21.15.78:0/1835311903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 msgr2=0x560d8dd877d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd877d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 -- 172.21.15.78:0/1835311903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d8d9f42d0 con 0x560d8d9b33b0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f390bfff640 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9b33b0 0x560d8dd87290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3900006010 tx=0x7f39000314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2852b7e3fc418f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f38ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f38ebfff640 1 -- 172.21.15.78:0/1835311903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900035040 con 0x560d8d9b33b0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f38ebfff640 1 -- 172.21.15.78:0/1835311903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3900004130 con 0x560d8d9b33b0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39000317a0 con 0x560d8d9b33b0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f38ebfff640 1 -- 172.21.15.78:0/1835311903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3900041040 con 0x560d8d9b33b0 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f3919e07740 10 client.15075 fetch_fsmap learned FSMap version 21 2024-04-25T13:58:40.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d8dabe6b0 con 0x560d8d9b33b0 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.782+0000 7f38ebfff640 1 -- 172.21.15.78:0/1835311903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39000317a0 con 0x560d8d9b33b0 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f38ebfff640 1 -- 172.21.15.78:0/1835311903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3900046450 con 0x560d8d9b33b0 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3919e07740 10 client.15075 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3919e07740 10 client.15075 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3919e07740 20 client.15075 populate_metadata read hostname 'smithi078' 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3919e07740 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d99ae40 0x560d8dd949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3919e07740 4 client.15075 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d8dcf0350 con 0x560d8d99ae40 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3910c20640 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d99ae40 0x560d8dd949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f38ebfff640 10 client.15075 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.783+0000 7f3910c20640 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d99ae40 0x560d8dd949a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e59d02f437e97431 server_cookie=1aed6def47e3b31 in_seq=0 out_seq=0 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f38ebfff640 1 -- 172.21.15.78:0/1835311903 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d8dcf0350 con 0x560d8d99ae40 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f38ebfff640 10 client.15075 handle_command_reply: tid=0 2024-04-25T13:58:40.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 client.15075 shutdown 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9b33b0 msgr2=0x560d8dd87290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9b33b0 0x560d8dd87290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3900006010 tx=0x7f39000314a0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 shutdown_connections 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d99ae40 0x560d8dd949a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9dc270 0x560d8dd8e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd877d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f38ebfff640 0 client.15075 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 --2- 172.21.15.78:0/1835311903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9b33b0 0x560d8dd87290 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.784+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 >> 172.21.15.78:0/1835311903 conn(0x560d8d9d03c0 msgr2=0x560d8dd77d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.785+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 shutdown_connections 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.785+0000 7f3919e07740 1 -- 172.21.15.78:0/1835311903 wait complete. 2024-04-25T13:58:40.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.785+0000 7f3919e07740 20 client.15075 trim_cache size 0 max 16384 2024-04-25T13:58:40.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 Processor -- start 2024-04-25T13:58:40.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 -- start start 2024-04-25T13:58:40.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd87570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd92ce0 con 0x560d8d9dc270 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd933b0 con 0x560d8d9bb590 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f3919e07740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd93a80 con 0x560d8d9b33b0 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57146/0 (socket says 172.21.15.78:57146) 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 -- 172.21.15.78:0/3136646272 learned_addr learned my addr 172.21.15.78:0/3136646272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd87570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 -- 172.21.15.78:0/3136646272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 msgr2=0x560d8dd87570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd87570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 -- 172.21.15.78:0/3136646272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 msgr2=0x560d8dd8e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 -- 172.21.15.78:0/3136646272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d8dd82170 con 0x560d8d9dc270 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.803+0000 7f390bfff640 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8ec00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f39000096c0 tx=0x7f390002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65270902371a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f38f27fc640 1 -- 172.21.15.78:0/3136646272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900035070 con 0x560d8d9dc270 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f38f27fc640 1 -- 172.21.15.78:0/3136646272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3900060070 con 0x560d8d9dc270 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f38f27fc640 1 -- 172.21.15.78:0/3136646272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900059470 con 0x560d8d9dc270 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 -- 172.21.15.78:0/3136646272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 msgr2=0x560d8dd8ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8ec00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f39000096c0 tx=0x7f390002a410 comp rx=0 tx=0).stop 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 -- 172.21.15.78:0/3136646272 shutdown_connections 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8ec00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd8e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 --2- 172.21.15.78:0/3136646272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd87570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 -- 172.21.15.78:0/3136646272 >> 172.21.15.78:0/3136646272 conn(0x560d8d9d03c0 msgr2=0x560d8dd86e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 -- 172.21.15.78:0/3136646272 shutdown_connections 2024-04-25T13:58:40.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.804+0000 7f3919e07740 1 -- 172.21.15.78:0/3136646272 wait complete. 2024-04-25T13:58:40.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 -- start start 2024-04-25T13:58:40.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 Processor -- start 2024-04-25T13:58:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd8c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd94820 con 0x560d8d9dc270 2024-04-25T13:58:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd94150 con 0x560d8d9bb590 2024-04-25T13:58:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d8dd94ef0 con 0x560d8d9b33b0 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f3919e07740 10 client.0 _ll_register_callbacks cb 0x7f3900031d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57156/0 (socket says 172.21.15.78:57156) 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 -- 172.21.15.78:0/3553551328 learned_addr learned my addr 172.21.15.78:0/3553551328 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd8c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 -- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 msgr2=0x560d8dd8c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd8c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 -- 172.21.15.78:0/3553551328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 msgr2=0x560d8dd8cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.805+0000 7f390bfff640 1 -- 172.21.15.78:0/3553551328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d8dd8dff0 con 0x560d8d9dc270 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f390bfff640 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8da30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3900031500 tx=0x7f39000058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2088d4b2ec6577c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f38ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:58:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f38ebfff640 1 -- 172.21.15.78:0/3553551328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900035040 con 0x560d8d9dc270 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f38ebfff640 1 -- 172.21.15.78:0/3553551328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3900004130 con 0x560d8d9dc270 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3900060070 con 0x560d8d9dc270 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f38ebfff640 1 -- 172.21.15.78:0/3553551328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3900009520 con 0x560d8d9dc270 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 10 client.15081 fetch_fsmap learned FSMap version 21 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d8dd8f270 con 0x560d8d9dc270 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f38ebfff640 1 -- 172.21.15.78:0/3553551328 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3900060070 con 0x560d8d9dc270 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f38ebfff640 1 -- 172.21.15.78:0/3553551328 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3900059c50 con 0x560d8d9dc270 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 10 client.15081 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 10 client.15081 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 20 client.15081 populate_metadata read hostname 'smithi078' 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d970280 0x560d8ddfe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 4 client.15081 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.806+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "status"}) -- 0x560d8dcea1c0 con 0x560d8d970280 2024-04-25T13:58:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f3910c20640 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d970280 0x560d8ddfe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f38ebfff640 10 client.15081 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f3910c20640 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d970280 0x560d8ddfe4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdf0e63cad779b40 server_cookie=2040826a55629505 in_seq=0 out_seq=0 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f38ebfff640 1 -- 172.21.15.78:0/3553551328 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560d8dcea1c0 con 0x560d8d970280 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f38ebfff640 10 client.15081 handle_command_reply: tid=0 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f3919e07740 1 client.15081 shutdown 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 msgr2=0x560d8dd8da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.807+0000 7f3919e07740 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8da30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3900031500 tx=0x7f39000058b0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 shutdown_connections 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x560d8d970280 0x560d8ddfe4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560d8d9dc270 0x560d8dd8da30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d8d9bb590 0x560d8dd8cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 --2- 172.21.15.78:0/3553551328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d8d9b33b0 0x560d8dd8c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f38ebfff640 0 client.15081 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 >> 172.21.15.78:0/3553551328 conn(0x560d8d9d03c0 msgr2=0x560d8dd7bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 shutdown_connections 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 1 -- 172.21.15.78:0/3553551328 wait complete. 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3919e07740 20 client.15081 trim_cache size 0 max 16384 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24454, 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T13:58:40.814 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T13:58:40.815 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T13:58:40.815 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 85.732725039000002, 2024-04-25T13:58:40.815 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 21, 2024-04-25T13:58:40.815 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-04-25T13:58:40.815 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T13:58:40.815 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 112.06545814499999 2024-04-25T13:58:40.815 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38f40afaf0 msgr2=0x7f38f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38f40afaf0 0x7f38f40b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39040a4470 tx=0x7f3904075d50 comp rx=0 tx=0).stop 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c108920 msgr2=0x7f390c1a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c108920 0x7f390c1a1a00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f390400cad0 tx=0x7f390400cf90 comp rx=0 tx=0).stop 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 shutdown_connections 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38f40afaf0 0x7f38f40b2000 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f390c108920 0x7f390c1a1a00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f390c103c20 0x7f390c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 --2- 172.21.15.78:0/772128298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f390c1029e0 0x7f390c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 >> 172.21.15.78:0/772128298 conn(0x7f390c0fdbd0 msgr2=0x7f390c109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 shutdown_connections 2024-04-25T13:58:40.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.808+0000 7f3912e84640 1 -- 172.21.15.78:0/772128298 wait complete. 2024-04-25T13:58:40.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:58:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 -- 172.21.15.78:0/3780231837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c102fc0 msgr2=0x7f3f1c1033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/3780231837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c102fc0 0x7f3f1c1033d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009ca0 tx=0x7f3f1802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 -- 172.21.15.78:0/3780231837 shutdown_connections 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/3780231837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f1c104680 0x7f3f1c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/3780231837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c103b20 0x7f3f1c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/3780231837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c102fc0 0x7f3f1c1033d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 -- 172.21.15.78:0/3780231837 >> 172.21.15.78:0/3780231837 conn(0x7f3f1c0fd340 msgr2=0x7f3f1c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 -- 172.21.15.78:0/3780231837 shutdown_connections 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 -- 172.21.15.78:0/3780231837 wait complete. 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 Processor -- start 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.931+0000 7f3f23fb2640 1 -- start start 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f1c102fc0 0x7f3f1c10d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c103b20 0x7f3f1c10bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c10c0d0 0x7f3f1c10c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10f850 con 0x7f3f1c103b20 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10f6d0 con 0x7f3f1c10c0d0 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10f9d0 con 0x7f3f1c102fc0 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c103b20 0x7f3f1c10bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c103b20 0x7f3f1c10bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57176/0 (socket says 172.21.15.78:57176) 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 -- 172.21.15.78:0/1314805691 learned_addr learned my addr 172.21.15.78:0/1314805691 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2254f640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c10c0d0 0x7f3f1c10c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 -- 172.21.15.78:0/1314805691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f1c102fc0 msgr2=0x7f3f1c10d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f1c102fc0 0x7f3f1c10d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 -- 172.21.15.78:0/1314805691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c10c0d0 msgr2=0x7f3f1c10c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c10c0d0 0x7f3f1c10c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 -- 172.21.15.78:0/1314805691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c1a41d0 con 0x7f3f1c103b20 2024-04-25T13:58:40.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c103b20 0x7f3f1c10bb90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00d980 tx=0x7f3f0c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f12ffd640 1 -- 172.21.15.78:0/1314805691 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c017030 con 0x7f3f1c103b20 2024-04-25T13:58:40.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f12ffd640 1 -- 172.21.15.78:0/1314805691 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c004020 con 0x7f3f1c103b20 2024-04-25T13:58:40.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c1a44c0 con 0x7f3f1c103b20 2024-04-25T13:58:40.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f12ffd640 1 -- 172.21.15.78:0/1314805691 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c005630 con 0x7f3f1c103b20 2024-04-25T13:58:40.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.932+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c1a4a10 con 0x7f3f1c103b20 2024-04-25T13:58:40.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 Processor -- start 2024-04-25T13:58:40.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 -- start start 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f12ffd640 1 -- 172.21.15.78:0/1314805691 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3f0c0047a0 con 0x7f3f1c103b20 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d2f0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 0x56333d2f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56333d255740 con 0x56333cf263b0 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56333d1f13a0 con 0x56333cf2e590 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56333d251700 con 0x56333cf4f270 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f21d4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f21d4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57182/0 (socket says 172.21.15.78:57182) 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f21d4e640 1 -- 172.21.15.78:0/2289731706 learned_addr learned my addr 172.21.15.78:0/2289731706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2254f640 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d2f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f2154d640 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 0x56333d2f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f21d4e640 1 -- 172.21.15.78:0/2289731706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 msgr2=0x56333d2f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d2f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.935+0000 7f3f21d4e640 1 -- 172.21.15.78:0/2289731706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 msgr2=0x56333d2f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 0x56333d2f77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f21d4e640 1 -- 172.21.15.78:0/2289731706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333d20d750 con 0x56333cf263b0 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f12ffd640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3f080afaf0 0x7f3f080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f08f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f1802ade0 tx=0x7f3f1802f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df78b8a4df654469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f2254f640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3f080afaf0 0x7f3f080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f12ffd640 1 -- 172.21.15.78:0/1314805691 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3f0c0de820 con 0x7f3f1c103b20 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f07fff640 1 -- 172.21.15.78:0/2289731706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18031650 con 0x56333cf263b0 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f07fff640 1 -- 172.21.15.78:0/2289731706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f1803f070 con 0x56333cf263b0 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.936+0000 7f3f07fff640 1 -- 172.21.15.78:0/2289731706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f180046e0 con 0x56333cf263b0 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2289731706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 msgr2=0x56333d2f08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2254f640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3f080afaf0 0x7f3f080b2000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x56333d2f1540 tx=0x7f3f1400c350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f08f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f1802ade0 tx=0x7f3f1802f450 comp rx=0 tx=0).stop 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2289731706 shutdown_connections 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 0x56333d2f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f08f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2289731706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d2f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2289731706 >> 172.21.15.78:0/2289731706 conn(0x56333cf433c0 msgr2=0x56333d2ebd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2289731706 shutdown_connections 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2289731706 wait complete. 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 -- start start 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.937+0000 7f3f2ae5e740 1 Processor -- start 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf2e590 0x56333d2fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d3016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56333d2f81e0 con 0x56333cf2e590 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56333d1dcfe0 con 0x56333cf263b0 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56333d2f8360 con 0x56333cf4f270 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 10 client.0 _ll_register_callbacks cb 0x56333d1f3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf2e590 0x56333d2fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf2e590 0x56333d2fa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57190/0 (socket says 172.21.15.78:57190) 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 -- 172.21.15.78:0/1876554074 learned_addr learned my addr 172.21.15.78:0/1876554074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2254f640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d3016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 -- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 msgr2=0x56333d3016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d3016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 -- 172.21.15.78:0/1876554074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 msgr2=0x56333d2fa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 -- 172.21.15.78:0/1876554074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333cf672d0 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2254f640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d3016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf2e590 0x56333d2fa750 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00d950 tx=0x7f3f0c0a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4558d10e7ed4015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f057fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f057fa640 1 -- 172.21.15.78:0/1876554074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0a6cf0 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f057fa640 1 -- 172.21.15.78:0/1876554074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3f0c0a2f80 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.938+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f1803f070 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f057fa640 1 -- 172.21.15.78:0/1876554074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c0ab0a0 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 10 client.15093 fetch_fsmap learned FSMap version 21 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56333d0316b0 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f057fa640 1 -- 172.21.15.78:0/1876554074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f1803f070 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f057fa640 1 -- 172.21.15.78:0/1876554074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3f0c0a27f0 con 0x56333cf2e590 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 10 client.15093 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 20 client.15093 populate_metadata read hostname 'smithi078' 2024-04-25T13:58:40.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf0de40 0x56333d307ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 4 client.15093 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333d263350 con 0x56333cf0de40 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.939+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf0de40 0x56333d307ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.940+0000 7f3f057fa640 10 client.15093 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.940+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf0de40 0x56333d307ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73c7f6f44b2e3f2f server_cookie=f9fb41b5d544f32a in_seq=0 out_seq=0 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f057fa640 1 -- 172.21.15.78:0/1876554074 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56333d263350 con 0x56333cf0de40 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f057fa640 10 client.15093 handle_command_reply: tid=0 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 client.15093 shutdown 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf2e590 msgr2=0x56333d2fa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf2e590 0x56333d2fa750 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00d950 tx=0x7f3f0c0a6520 comp rx=0 tx=0).stop 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 shutdown_connections 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf0de40 0x56333d307ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf4f270 0x56333d3016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf2e590 0x56333d2fa750 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f057fa640 0 client.15093 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T13:58:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1876554074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 >> 172.21.15.78:0/1876554074 conn(0x56333cf433c0 msgr2=0x56333d2ead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 shutdown_connections 2024-04-25T13:58:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1876554074 wait complete. 2024-04-25T13:58:40.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.941+0000 7f3f2ae5e740 20 client.15093 trim_cache size 0 max 16384 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 Processor -- start 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 -- start start 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf0de40 0x56333d2e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 0x56333d2f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56333d306210 con 0x56333cf263b0 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56333d3068e0 con 0x56333cf2e590 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56333d306fb0 con 0x56333cf0de40 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57202/0 (socket says 172.21.15.78:57202) 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.959+0000 7f3f2154d640 1 -- 172.21.15.78:0/2555423041 learned_addr learned my addr 172.21.15.78:0/2555423041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f2154d640 1 -- 172.21.15.78:0/2555423041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf0de40 msgr2=0x56333d2e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f2154d640 1 --2- 172.21.15.78:0/2555423041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf0de40 0x56333d2e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f2154d640 1 -- 172.21.15.78:0/2555423041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 msgr2=0x56333d2f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f2154d640 1 --2- 172.21.15.78:0/2555423041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 0x56333d2f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f2154d640 1 -- 172.21.15.78:0/2555423041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333d2eff20 con 0x56333cf263b0 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f2154d640 1 --2- 172.21.15.78:0/2555423041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f8850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0a6c80 tx=0x7f3f0c0a6590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0ce1be09403791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f077fe640 1 -- 172.21.15.78:0/2555423041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0a6ac0 con 0x56333cf263b0 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f077fe640 1 -- 172.21.15.78:0/2555423041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c0a6cf0 con 0x56333cf263b0 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.960+0000 7f3f077fe640 1 -- 172.21.15.78:0/2555423041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0a2660 con 0x56333cf263b0 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2555423041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 msgr2=0x56333d2f8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2555423041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f8850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0a6c80 tx=0x7f3f0c0a6590 comp rx=0 tx=0).stop 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2555423041 shutdown_connections 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2555423041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf2e590 0x56333d2f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2555423041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf263b0 0x56333d2f8850 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/2555423041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf0de40 0x56333d2e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2555423041 >> 172.21.15.78:0/2555423041 conn(0x56333cf433c0 msgr2=0x56333d2f46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2555423041 shutdown_connections 2024-04-25T13:58:40.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/2555423041 wait complete. 2024-04-25T13:58:40.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 -- start start 2024-04-25T13:58:40.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.961+0000 7f3f2ae5e740 1 Processor -- start 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf0de40 0x56333d2fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf2e590 0x56333d300ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56333d307d50 con 0x56333cf0de40 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56333d307680 con 0x56333cf263b0 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56333d308420 con 0x56333cf2e590 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 10 client.0 _ll_register_callbacks cb 0x7f3f0c00df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf0de40 0x56333d2fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf0de40 0x56333d2fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57214/0 (socket says 172.21.15.78:57214) 2024-04-25T13:58:40.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 -- 172.21.15.78:0/1515661051 learned_addr learned my addr 172.21.15.78:0/1515661051 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 -- 172.21.15.78:0/1515661051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf2e590 msgr2=0x56333d300ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf2e590 0x56333d300ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 -- 172.21.15.78:0/1515661051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 msgr2=0x56333d2fddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 -- 172.21.15.78:0/1515661051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333d301260 con 0x56333cf0de40 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f21d4e640 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf0de40 0x56333d2fd870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f180097b0 tx=0x7f3f18009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990e5fa01918ecca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f057fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f057fa640 1 -- 172.21.15.78:0/1515661051 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18003b70 con 0x56333cf0de40 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f057fa640 1 -- 172.21.15.78:0/1515661051 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3f18003d10 con 0x56333cf0de40 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0c0a6cf0 con 0x56333cf0de40 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f057fa640 1 -- 172.21.15.78:0/1515661051 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3f18004040 con 0x56333cf0de40 2024-04-25T13:58:40.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 10 client.15099 fetch_fsmap learned FSMap version 21 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.962+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56333d370e80 con 0x56333cf0de40 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f057fa640 1 -- 172.21.15.78:0/1515661051 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c0a6cf0 con 0x56333cf0de40 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f057fa640 1 -- 172.21.15.78:0/1515661051 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3f1805a020 con 0x56333cf0de40 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2ae5e740 10 client.15099 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2ae5e740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2ae5e740 20 client.15099 populate_metadata read hostname 'smithi078' 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf4f270 0x56333d371a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2ae5e740 4 client.15099 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "status"}) -- 0x56333d25d1c0 con 0x56333cf4f270 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf4f270 0x56333d371a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f057fa640 10 client.15099 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.963+0000 7f3f2154d640 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf4f270 0x56333d371a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8886d225f4848892 server_cookie=40105a04e2cb9b1c in_seq=0 out_seq=0 2024-04-25T13:58:40.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f057fa640 1 -- 172.21.15.78:0/1515661051 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56333d25d1c0 con 0x56333cf4f270 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f057fa640 10 client.15099 handle_command_reply: tid=0 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 client.15099 shutdown 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf0de40 msgr2=0x56333d2fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf0de40 0x56333d2fd870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3f180097b0 tx=0x7f3f18009540 comp rx=0 tx=0).stop 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 shutdown_connections 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56333cf4f270 0x56333d371a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cf2e590 0x56333d300ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56333cf263b0 0x56333d2fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f057fa640 0 client.15099 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 --2- 172.21.15.78:0/1515661051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56333cf0de40 0x56333d2fd870 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 >> 172.21.15.78:0/1515661051 conn(0x56333cf433c0 msgr2=0x56333d2f8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 shutdown_connections 2024-04-25T13:58:40.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 1 -- 172.21.15.78:0/1515661051 wait complete. 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f2ae5e740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24484, 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 86.806439510999994, 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 21, 2024-04-25T13:58:40.971 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-04-25T13:58:40.972 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T13:58:40.972 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 103.87692472499999 2024-04-25T13:58:40.972 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:58:40.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3f080afaf0 msgr2=0x7f3f080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3f080afaf0 0x7f3f080b2000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x56333d2f1540 tx=0x7f3f1400c350 comp rx=0 tx=0).stop 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c103b20 msgr2=0x7f3f1c10bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c103b20 0x7f3f1c10bb90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00d980 tx=0x7f3f0c00de40 comp rx=0 tx=0).stop 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 shutdown_connections 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f1c10c0d0 0x7f3f1c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.964+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3f080afaf0 0x7f3f080b2000 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.965+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3f1c103b20 0x7f3f1c10bb90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.965+0000 7f3f23fb2640 1 --2- 172.21.15.78:0/1314805691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f1c102fc0 0x7f3f1c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.965+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 >> 172.21.15.78:0/1314805691 conn(0x7f3f1c0fd340 msgr2=0x7f3f1c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.965+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 shutdown_connections 2024-04-25T13:58:40.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:40.965+0000 7f3f23fb2640 1 -- 172.21.15.78:0/1314805691 wait complete. 2024-04-25T13:58:40.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:58:41.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 -- 172.21.15.78:0/192602755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341080c0 msgr2=0x7fc73410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 --2- 172.21.15.78:0/192602755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341080c0 0x7fc73410a4e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009ca0 tx=0x7fc72c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 -- 172.21.15.78:0/192602755 shutdown_connections 2024-04-25T13:58:41.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 --2- 172.21.15.78:0/192602755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341080c0 0x7fc73410a4e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 --2- 172.21.15.78:0/192602755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7341033c0 0x7fc7341037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 --2- 172.21.15.78:0/192602755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc734102180 0x7fc734102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 -- 172.21.15.78:0/192602755 >> 172.21.15.78:0/192602755 conn(0x7fc7340fd350 msgr2=0x7fc7340ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 -- 172.21.15.78:0/192602755 shutdown_connections 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 -- 172.21.15.78:0/192602755 wait complete. 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.085+0000 7fc73c092640 1 Processor -- start 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 -- start start 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc734102180 0x7fc734199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341033c0 0x7fc7341995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7341080c0 0x7fc7341a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410e510 con 0x7fc7341033c0 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410e390 con 0x7fc734102180 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410e690 con 0x7fc7341080c0 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341033c0 0x7fc7341995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341033c0 0x7fc7341995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57224/0 (socket says 172.21.15.78:57224) 2024-04-25T13:58:41.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 -- 172.21.15.78:0/2654752371 learned_addr learned my addr 172.21.15.78:0/2654752371 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73a62f640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7341080c0 0x7fc7341a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc739e2e640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc734102180 0x7fc734199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 -- 172.21.15.78:0/2654752371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7341080c0 msgr2=0x7fc7341a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7341080c0 0x7fc7341a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 -- 172.21.15.78:0/2654752371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc734102180 msgr2=0x7fc734199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc734102180 0x7fc734199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 -- 172.21.15.78:0/2654752371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7341a0b10 con 0x7fc7341033c0 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc739e2e640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc734102180 0x7fc734199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73962d640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341033c0 0x7fc7341995c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c920 tx=0x7fc73000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:41.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc72affd640 1 -- 172.21.15.78:0/2654752371 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730007c70 con 0x7fc7341033c0 2024-04-25T13:58:41.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc72affd640 1 -- 172.21.15.78:0/2654752371 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc730007e10 con 0x7fc7341033c0 2024-04-25T13:58:41.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341a0e00 con 0x7fc7341033c0 2024-04-25T13:58:41.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc72affd640 1 -- 172.21.15.78:0/2654752371 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730018820 con 0x7fc7341033c0 2024-04-25T13:58:41.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.086+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a1270 con 0x7fc7341033c0 2024-04-25T13:58:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc72affd640 1 -- 172.21.15.78:0/2654752371 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc73000f450 con 0x7fc7341033c0 2024-04-25T13:58:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 Processor -- start 2024-04-25T13:58:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 -- start start 2024-04-25T13:58:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce042740 con 0x55d8cdd1b590 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8cdfde3a0 con 0x55d8cdd3c270 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.089+0000 7fc743014740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce03e700 con 0x55d8cdd133b0 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc72affd640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc7200afaf0 0x7fc7200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57228/0 (socket says 172.21.15.78:57228) 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 -- 172.21.15.78:0/1185654811 learned_addr learned my addr 172.21.15.78:0/1185654811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc739e2e640 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73a62f640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc7200afaf0 0x7fc7200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc72affd640 1 -- 172.21.15.78:0/2654752371 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc7300dee30 con 0x7fc7341033c0 2024-04-25T13:58:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73a62f640 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 -- 172.21.15.78:0/1185654811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 msgr2=0x55d8ce0dfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0dfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 -- 172.21.15.78:0/1185654811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 msgr2=0x55d8ce0df580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0df580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 -- 172.21.15.78:0/1185654811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cdffa750 con 0x55d8cdd1b590 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73a62f640 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73962d640 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e6a00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc730000c00 tx=0x7fc73000fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f6fab731b3265c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.090+0000 7fc73a62f640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc7200afaf0 0x7fc7200b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc72c004af0 tx=0x7fc72c0099c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.091+0000 7fc71ffff640 1 -- 172.21.15.78:0/1185654811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7300a6e10 con 0x55d8cdd1b590 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.091+0000 7fc71ffff640 1 -- 172.21.15.78:0/1185654811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7300a7a00 con 0x55d8cdd1b590 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.091+0000 7fc71ffff640 1 -- 172.21.15.78:0/1185654811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73000f8a0 con 0x55d8cdd1b590 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 -- 172.21.15.78:0/1185654811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 msgr2=0x55d8ce0e6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e6a00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc730000c00 tx=0x7fc73000fe60 comp rx=0 tx=0).stop 2024-04-25T13:58:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 -- 172.21.15.78:0/1185654811 shutdown_connections 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e6a00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 --2- 172.21.15.78:0/1185654811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 -- 172.21.15.78:0/1185654811 >> 172.21.15.78:0/1185654811 conn(0x55d8cdd303c0 msgr2=0x55d8ce0d8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 -- 172.21.15.78:0/1185654811 shutdown_connections 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 -- 172.21.15.78:0/1185654811 wait complete. 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 -- start start 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.092+0000 7fc743014740 1 Processor -- start 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0e7450 con 0x55d8cdd3c270 2024-04-25T13:58:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8cdfc9fe0 con 0x55d8cdd1b590 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0e75d0 con 0x55d8cdd133b0 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 10 client.0 _ll_register_callbacks cb 0x55d8cdfe0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57236/0 (socket says 172.21.15.78:57236) 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 -- 172.21.15.78:0/163011024 learned_addr learned my addr 172.21.15.78:0/163011024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73962d640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc739e2e640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 -- 172.21.15.78:0/163011024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 msgr2=0x55d8ce0e9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 -- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 msgr2=0x55d8ce0e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 -- 172.21.15.78:0/163011024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cdd542d0 con 0x55d8cdd3c270 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc739e2e640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73962d640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc73a62f640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0f0a00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009ca0 tx=0x7fc72c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6deef3aa8dc467cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc71d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc71d7fa640 1 -- 172.21.15.78:0/163011024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c04d030 con 0x55d8cdd3c270 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc71d7fa640 1 -- 172.21.15.78:0/163011024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc72c044070 con 0x55d8cdd3c270 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.093+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7300a7a00 con 0x55d8cdd3c270 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc71d7fa640 1 -- 172.21.15.78:0/163011024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc72c003e50 con 0x55d8cdd3c270 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 10 client.15111 fetch_fsmap learned FSMap version 21 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d8cde1e6b0 con 0x55d8cdd3c270 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc71d7fa640 1 -- 172.21.15.78:0/163011024 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7300a7a00 con 0x55d8cdd3c270 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc71d7fa640 1 -- 172.21.15.78:0/163011024 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc72c047040 con 0x55d8cdd3c270 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 10 client.15111 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 10 client.15111 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:58:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 20 client.15111 populate_metadata read hostname 'smithi078' 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcfae40 0x55d8ce0f7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 4 client.15111 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8ce050350 con 0x55d8cdcfae40 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.094+0000 7fc739e2e640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcfae40 0x55d8ce0f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.095+0000 7fc739e2e640 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcfae40 0x55d8ce0f7290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f6e0ff42802297 server_cookie=811f15bb3b1d490b in_seq=0 out_seq=0 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.095+0000 7fc71d7fa640 10 client.15111 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc71d7fa640 1 -- 172.21.15.78:0/163011024 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d8ce050350 con 0x55d8cdcfae40 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc71d7fa640 10 client.15111 handle_command_reply: tid=0 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 client.15111 shutdown 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 msgr2=0x55d8ce0f0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0f0a00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009ca0 tx=0x7fc72c043000 comp rx=0 tx=0).stop 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 shutdown_connections 2024-04-25T13:58:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0f0a00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcfae40 0x55d8ce0f7290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 --2- 172.21.15.78:0/163011024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc71d7fa640 0 client.15111 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 >> 172.21.15.78:0/163011024 conn(0x55d8cdd303c0 msgr2=0x55d8ce0d7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 shutdown_connections 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 1 -- 172.21.15.78:0/163011024 wait complete. 2024-04-25T13:58:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.096+0000 7fc743014740 20 client.15111 trim_cache size 0 max 16384 2024-04-25T13:58:41.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.114+0000 7fc743014740 1 Processor -- start 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.114+0000 7fc743014740 1 -- start start 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc743014740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc743014740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc743014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc743014740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0f55d0 con 0x55d8cdd1b590 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc743014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0f5ca0 con 0x55d8cdd3c270 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc743014740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0f6370 con 0x55d8cdd133b0 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57252/0 (socket says 172.21.15.78:57252) 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 -- 172.21.15.78:0/1144255922 learned_addr learned my addr 172.21.15.78:0/1144255922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73a62f640 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 -- 172.21.15.78:0/1144255922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 msgr2=0x55d8ce0e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 -- 172.21.15.78:0/1144255922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 msgr2=0x55d8ce0e7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0e7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 -- 172.21.15.78:0/1144255922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ce0e7970 con 0x55d8cdd1b590 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73a62f640 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0e7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc73962d640 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e3f10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc7300a4570 tx=0x7fc7300a7200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d709dfb63661f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc71f7fe640 1 -- 172.21.15.78:0/1144255922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73006b070 con 0x55d8cdd1b590 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc71f7fe640 1 -- 172.21.15.78:0/1144255922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7300a7a00 con 0x55d8cdd1b590 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.115+0000 7fc71f7fe640 1 -- 172.21.15.78:0/1144255922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7300a7560 con 0x55d8cdd1b590 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 -- 172.21.15.78:0/1144255922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 msgr2=0x55d8ce0e3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e3f10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc7300a4570 tx=0x7fc7300a7200 comp rx=0 tx=0).stop 2024-04-25T13:58:41.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 -- 172.21.15.78:0/1144255922 shutdown_connections 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd3c270 0x55d8ce0e7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd1b590 0x55d8ce0e3f10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 --2- 172.21.15.78:0/1144255922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 -- 172.21.15.78:0/1144255922 >> 172.21.15.78:0/1144255922 conn(0x55d8cdd303c0 msgr2=0x55d8ce0d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 -- 172.21.15.78:0/1144255922 shutdown_connections 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 -- 172.21.15.78:0/1144255922 wait complete. 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.116+0000 7fc743014740 1 -- start start 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 1 Processor -- start 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0eb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0eb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0f7110 con 0x55d8cdd3c270 2024-04-25T13:58:41.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0f6a40 con 0x55d8cdd1b590 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8ce0f77e0 con 0x55d8cdd133b0 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc743014740 10 client.0 _ll_register_callbacks cb 0x7fc72c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57264/0 (socket says 172.21.15.78:57264) 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 -- 172.21.15.78:0/2325194110 learned_addr learned my addr 172.21.15.78:0/2325194110 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc739e2e640 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 -- 172.21.15.78:0/2325194110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 msgr2=0x55d8ce0eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0eb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 -- 172.21.15.78:0/2325194110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 msgr2=0x55d8ce0eb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.117+0000 7fc73a62f640 1 -- 172.21.15.78:0/2325194110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ce0eee40 con 0x55d8cdd3c270 2024-04-25T13:58:41.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc73a62f640 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0ee880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0094e0 tx=0x7fc72c030930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c091c6ec43170af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc71d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc71d7fa640 1 -- 172.21.15.78:0/2325194110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c030b20 con 0x55d8cdd3c270 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc71d7fa640 1 -- 172.21.15.78:0/2325194110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc72c044070 con 0x55d8cdd3c270 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7300a7a00 con 0x55d8cdd3c270 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc71d7fa640 1 -- 172.21.15.78:0/2325194110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc72c004620 con 0x55d8cdd3c270 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 10 client.15117 fetch_fsmap learned FSMap version 21 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d8ce0f0090 con 0x55d8cdd3c270 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc71d7fa640 1 -- 172.21.15.78:0/2325194110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7300a7a00 con 0x55d8cdd3c270 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc71d7fa640 1 -- 172.21.15.78:0/2325194110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc72c066050 con 0x55d8cdd3c270 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 10 client.15117 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 20 client.15117 populate_metadata read hostname 'smithi078' 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcd0280 0x55d8ce163540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:41.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 4 client.15117 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.118+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "status"}) -- 0x55d8ce04a1c0 con 0x55d8cdcd0280 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc739e2e640 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcd0280 0x55d8ce163540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc71d7fa640 10 client.15117 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc739e2e640 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcd0280 0x55d8ce163540 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588c450f5fe44668 server_cookie=d29562c7d0be4e4f in_seq=0 out_seq=0 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc71d7fa640 1 -- 172.21.15.78:0/2325194110 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d8ce04a1c0 con 0x55d8cdcd0280 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc71d7fa640 10 client.15117 handle_command_reply: tid=0 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 client.15117 shutdown 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 msgr2=0x55d8ce0ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0ee880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0094e0 tx=0x7fc72c030930 comp rx=0 tx=0).stop 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 shutdown_connections 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cdd3c270 0x55d8ce0ee880 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d8cdd1b590 0x55d8ce0eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d8cdcd0280 0x55d8ce163540 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 --2- 172.21.15.78:0/2325194110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8cdd133b0 0x55d8ce0eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc71d7fa640 0 client.15117 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 >> 172.21.15.78:0/2325194110 conn(0x55d8cdd303c0 msgr2=0x55d8ce0e39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 shutdown_connections 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.119+0000 7fc743014740 1 -- 172.21.15.78:0/2325194110 wait complete. 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc743014740 20 client.15117 trim_cache size 0 max 16384 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24512, 2024-04-25T13:58:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 88.995454674000001, 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 21, 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 95.218758976999993 2024-04-25T13:58:41.128 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc7200afaf0 msgr2=0x7fc7200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc7200afaf0 0x7fc7200b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc72c004af0 tx=0x7fc72c0099c0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341033c0 msgr2=0x7fc7341995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341033c0 0x7fc7341995c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c920 tx=0x7fc73000cde0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 shutdown_connections 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7341080c0 0x7fc7341a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc7200afaf0 0x7fc7200b2000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7341033c0 0x7fc7341995c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 --2- 172.21.15.78:0/2654752371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc734102180 0x7fc734199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 >> 172.21.15.78:0/2654752371 conn(0x7fc7340fd350 msgr2=0x7fc7341040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 shutdown_connections 2024-04-25T13:58:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:41.120+0000 7fc73c092640 1 -- 172.21.15.78:0/2654752371 wait complete. 2024-04-25T13:58:41.131 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T13:58:41.131 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:58:41.141 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T13:58:41.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T13:58:41.380 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:41 smithi078 ceph-mon[29163]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:58:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:41 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3696278188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:41 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1933374576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:41 smithi112 ceph-mon[32617]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:58:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:41 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3696278188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:41 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1933374576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:41.565 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:41 smithi129 ceph-mon[32569]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:58:41.565 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:41 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3696278188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:58:41.565 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:41 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1933374576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:58:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:42 smithi078 ceph-mon[29163]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:58:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:42 smithi112 ceph-mon[32617]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:58:42.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:42 smithi129 ceph-mon[32569]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:58:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 -- 172.21.15.78:0/354812756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104240 msgr2=0x7fab84104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 --2- 172.21.15.78:0/354812756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104240 0x7fab84104650 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009990 tx=0x7fab6c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 -- 172.21.15.78:0/354812756 shutdown_connections 2024-04-25T13:58:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 --2- 172.21.15.78:0/354812756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab84104be0 0x7fab8410b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 --2- 172.21.15.78:0/354812756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab84103110 0x7fab84103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 --2- 172.21.15.78:0/354812756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104240 0x7fab84104650 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 -- 172.21.15.78:0/354812756 >> 172.21.15.78:0/354812756 conn(0x7fab840fe750 msgr2=0x7fab84100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 -- 172.21.15.78:0/354812756 shutdown_connections 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.109+0000 7fab89692700 1 -- 172.21.15.78:0/354812756 wait complete. 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 Processor -- start 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 -- start start 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab84103110 0x7fab8406bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab84104240 0x7fab8419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104be0 0x7fab841a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab8410ec00 con 0x7fab84104be0 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab8410ea80 con 0x7fab84104240 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab89692700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab8410ed80 con 0x7fab84103110 2024-04-25T13:58:43.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab837fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104be0 0x7fab841a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab82ffd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab84103110 0x7fab8406bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab837fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104be0 0x7fab841a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57294/0 (socket says 172.21.15.78:57294) 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab837fe700 1 -- 172.21.15.78:0/510699964 learned_addr learned my addr 172.21.15.78:0/510699964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.110+0000 7fab7a7fc700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab84104240 0x7fab8419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab837fe700 1 -- 172.21.15.78:0/510699964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab84103110 msgr2=0x7fab8406bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab837fe700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab84103110 0x7fab8406bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab837fe700 1 -- 172.21.15.78:0/510699964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab84104240 msgr2=0x7fab8419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab837fe700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab84104240 0x7fab8419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab837fe700 1 -- 172.21.15.78:0/510699964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab841a2310 con 0x7fab84104be0 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab837fe700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104be0 0x7fab841a1b70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b970 tx=0x7fab7c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:43.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab80ff9700 1 -- 172.21.15.78:0/510699964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c019070 con 0x7fab84104be0 2024-04-25T13:58:43.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab80ff9700 1 -- 172.21.15.78:0/510699964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7c00cde0 con 0x7fab84104be0 2024-04-25T13:58:43.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab841a2600 con 0x7fab84104be0 2024-04-25T13:58:43.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab80ff9700 1 -- 172.21.15.78:0/510699964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c007370 con 0x7fab84104be0 2024-04-25T13:58:43.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.111+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab841a2b70 con 0x7fab84104be0 2024-04-25T13:58:43.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.112+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab84051190 con 0x7fab84104be0 2024-04-25T13:58:43.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.113+0000 7fab80ff9700 1 -- 172.21.15.78:0/510699964 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fab7c01c070 con 0x7fab84104be0 2024-04-25T13:58:43.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.114+0000 7fab80ff9700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab700afb80 0x7fab700b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:43.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.114+0000 7fab82ffd700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab700afb80 0x7fab700b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:43.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.114+0000 7fab80ff9700 1 -- 172.21.15.78:0/510699964 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fab7c0de880 con 0x7fab84104be0 2024-04-25T13:58:43.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.115+0000 7fab82ffd700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab700afb80 0x7fab700b2090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fab84104140 tx=0x7fab7400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:43.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.118+0000 7fab80ff9700 1 -- 172.21.15.78:0/510699964 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab7c023070 con 0x7fab84104be0 2024-04-25T13:58:43.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.340+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fab84063ad0 con 0x7fab700afb80 2024-04-25T13:58:43.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.341+0000 7fab80ff9700 1 -- 172.21.15.78:0/510699964 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fab84063ad0 con 0x7fab700afb80 2024-04-25T13:58:43.341 INFO:teuthology.orchestra.run.smithi078.stdout:Backend: cephadm 2024-04-25T13:58:43.341 INFO:teuthology.orchestra.run.smithi078.stdout:Available: Yes 2024-04-25T13:58:43.341 INFO:teuthology.orchestra.run.smithi078.stdout:Paused: No 2024-04-25T13:58:43.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.343+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab700afb80 msgr2=0x7fab700b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.343+0000 7fab89692700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab700afb80 0x7fab700b2090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fab84104140 tx=0x7fab7400a3b0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.343+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104be0 msgr2=0x7fab841a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.343+0000 7fab89692700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104be0 0x7fab841a1b70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b970 tx=0x7fab7c00bc80 comp rx=0 tx=0).stop 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 shutdown_connections 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab700afb80 0x7fab700b2090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab84104be0 0x7fab841a1b70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab84103110 0x7fab8406bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 --2- 172.21.15.78:0/510699964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab84104240 0x7fab8419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 >> 172.21.15.78:0/510699964 conn(0x7fab840fe750 msgr2=0x7fab84107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 shutdown_connections 2024-04-25T13:58:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:43.344+0000 7fab89692700 1 -- 172.21.15.78:0/510699964 wait complete. 2024-04-25T13:58:44.048 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T13:58:44.295 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:45 smithi078 ceph-mon[29163]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:45 smithi078 ceph-mon[29163]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:45 smithi112 ceph-mon[32617]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:45 smithi112 ceph-mon[32617]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:45.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:45 smithi129 ceph-mon[32569]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:45.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:45 smithi129 ceph-mon[32569]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 -- 172.21.15.78:0/2505411004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104f50 msgr2=0x7f8f2410b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 --2- 172.21.15.78:0/2505411004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104f50 0x7f8f2410b850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00b0d0 tx=0x7f8f1c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 -- 172.21.15.78:0/2505411004 shutdown_connections 2024-04-25T13:58:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 --2- 172.21.15.78:0/2505411004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104f50 0x7f8f2410b850 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 --2- 172.21.15.78:0/2505411004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f24103700 0x7f8f24103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 --2- 172.21.15.78:0/2505411004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f24104600 0x7f8f24104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 -- 172.21.15.78:0/2505411004 >> 172.21.15.78:0/2505411004 conn(0x7f8f240fdef0 msgr2=0x7f8f241003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 -- 172.21.15.78:0/2505411004 shutdown_connections 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.120+0000 7f8f2a946700 1 -- 172.21.15.78:0/2505411004 wait complete. 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 Processor -- start 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 -- start start 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f24103700 0x7f8f2419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104600 0x7f8f2419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f24104f50 0x7f8f241a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410edc0 con 0x7f8f24104600 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410ec40 con 0x7f8f24104f50 2024-04-25T13:58:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f2a946700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410ef40 con 0x7f8f24103700 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f237fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104600 0x7f8f2419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f237fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104600 0x7f8f2419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57310/0 (socket says 172.21.15.78:57310) 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f237fe700 1 -- 172.21.15.78:0/804811907 learned_addr learned my addr 172.21.15.78:0/804811907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.121+0000 7f8f23fff700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f24103700 0x7f8f2419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f28ee3700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f24104f50 0x7f8f241a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f237fe700 1 -- 172.21.15.78:0/804811907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f24103700 msgr2=0x7f8f2419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f237fe700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f24103700 0x7f8f2419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f237fe700 1 -- 172.21.15.78:0/804811907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f24104f50 msgr2=0x7f8f241a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f237fe700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f24104f50 0x7f8f241a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f237fe700 1 -- 172.21.15.78:0/804811907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f241a24d0 con 0x7f8f24104600 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f28ee3700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f24104f50 0x7f8f241a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:58:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f237fe700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104600 0x7f8f2419aeb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800d980 tx=0x7f8f1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f217fa700 1 -- 172.21.15.78:0/804811907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18016070 con 0x7f8f24104600 2024-04-25T13:58:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f217fa700 1 -- 172.21.15.78:0/804811907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f18004830 con 0x7f8f24104600 2024-04-25T13:58:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f241a27c0 con 0x7f8f24104600 2024-04-25T13:58:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f217fa700 1 -- 172.21.15.78:0/804811907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f180054c0 con 0x7f8f24104600 2024-04-25T13:58:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.122+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a2d00 con 0x7f8f24104600 2024-04-25T13:58:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.123+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24051190 con 0x7f8f24104600 2024-04-25T13:58:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.124+0000 7f8f217fa700 1 -- 172.21.15.78:0/804811907 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8f18004220 con 0x7f8f24104600 2024-04-25T13:58:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.125+0000 7f8f217fa700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8f0c0afb80 0x7f8f0c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.125+0000 7f8f23fff700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8f0c0afb80 0x7f8f0c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.125+0000 7f8f217fa700 1 -- 172.21.15.78:0/804811907 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8f180def50 con 0x7f8f24104600 2024-04-25T13:58:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.126+0000 7f8f23fff700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8f0c0afb80 0x7f8f0c0b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f24104590 tx=0x7f8f14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:46.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.129+0000 7f8f217fa700 1 -- 172.21.15.78:0/804811907 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f1801f020 con 0x7f8f24104600 2024-04-25T13:58:46.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.336+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8f240ffe70 con 0x7f8f0c0afb80 2024-04-25T13:58:46.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.350+0000 7f8f217fa700 1 -- 172.21.15.78:0/804811907 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f8f240ffe70 con 0x7f8f0c0afb80 2024-04-25T13:58:46.349 INFO:teuthology.orchestra.run.smithi078.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.a smithi078 running (103s) 16s ago 103s 84.1M - 19.0.0-3291-g994e2f20 d53c96779a29 7ead8345f63b 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.b smithi112 running (100s) 34s ago 100s 96.0M - 19.0.0-3291-g994e2f20 d53c96779a29 832616ae2e58 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.c smithi129 running (115s) 34s ago 114s 86.0M - 19.0.0-3291-g994e2f20 d53c96779a29 62bd797bd311 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.d smithi078 running (2m) 16s ago 2m 81.1M - 19.0.0-3291-g994e2f20 d53c96779a29 6a0ff26c9602 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.e smithi112 running (2m) 34s ago 2m 84.4M - 19.0.0-3291-g994e2f20 d53c96779a29 94e0751fdcd2 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.f smithi129 running (118s) 34s ago 117s 84.4M - 19.0.0-3291-g994e2f20 d53c96779a29 c887c63f1db7 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.g smithi078 running (106s) 16s ago 106s 83.6M - 19.0.0-3291-g994e2f20 d53c96779a29 8a2adb1f743e 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.h smithi112 running (112s) 34s ago 111s 85.4M - 19.0.0-3291-g994e2f20 d53c96779a29 7871954402fe 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.i smithi129 running (109s) 34s ago 109s 84.9M - 19.0.0-3291-g994e2f20 d53c96779a29 48581fb53716 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.j smithi078 running (2m) 16s ago 2m 81.4M - 19.0.0-3291-g994e2f20 d53c96779a29 d98f796c6f60 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.k smithi112 running (2m) 34s ago 2m 83.9M - 19.0.0-3291-g994e2f20 d53c96779a29 e2927c9c209b 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mds.l smithi129 running (2m) 34s ago 2m 81.5M - 19.0.0-3291-g994e2f20 d53c96779a29 659f89faefcb 2024-04-25T13:58:46.350 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.x smithi078 *:9283,8765 running (9m) 16s ago 9m 583M - 19.0.0-3291-g994e2f20 d53c96779a29 b162b50cfff9 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.y smithi112 *:8443,8765 running (7m) 34s ago 7m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 93c7a9abd610 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.z smithi129 *:8443,8765 running (7m) 34s ago 7m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 5a240cc8953e 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:mon.a smithi078 running (9m) 16s ago 9m 115M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 70d96a743c37 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:mon.b smithi112 running (7m) 34s ago 7m 106M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 874de5220127 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:mon.c smithi129 running (7m) 34s ago 7m 109M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 004acab6f631 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 smithi078 running (7m) 16s ago 7m 117M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 17e7001d85f5 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 smithi112 running (6m) 34s ago 6m 123M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 86eff2d8504d 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 smithi129 running (6m) 34s ago 6m 124M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 8ce11e25e84b 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 smithi078 running (5m) 16s ago 5m 123M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 7caab565cf93 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 smithi112 running (5m) 34s ago 5m 119M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 f5253fa989a5 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 smithi129 running (5m) 34s ago 5m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 37b05d34c6fc 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 smithi078 running (4m) 16s ago 4m 108M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 d70e3f488e01 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 smithi112 running (4m) 34s ago 4m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 b8ea8386350a 2024-04-25T13:58:46.351 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 smithi129 running (3m) 34s ago 3m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 49d499df7f57 2024-04-25T13:58:46.352 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 smithi078 running (3m) 16s ago 3m 122M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 ad91ba42e188 2024-04-25T13:58:46.352 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 smithi112 running (2m) 34s ago 2m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 bd356e625fee 2024-04-25T13:58:46.352 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 smithi129 running (2m) 34s ago 2m 110M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 f4776edfcf71 2024-04-25T13:58:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8f0c0afb80 msgr2=0x7f8f0c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8f0c0afb80 0x7f8f0c0b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f24104590 tx=0x7f8f14009260 comp rx=0 tx=0).stop 2024-04-25T13:58:46.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104600 msgr2=0x7f8f2419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104600 0x7f8f2419aeb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800d980 tx=0x7f8f1800dc90 comp rx=0 tx=0).stop 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 shutdown_connections 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8f0c0afb80 0x7f8f0c0b2090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f24104f50 0x7f8f241a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f24103700 0x7f8f2419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 --2- 172.21.15.78:0/804811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f24104600 0x7f8f2419aeb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 >> 172.21.15.78:0/804811907 conn(0x7f8f240fdef0 msgr2=0x7f8f240ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 shutdown_connections 2024-04-25T13:58:46.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:46.352+0000 7f8f2a946700 1 -- 172.21.15.78:0/804811907 wait complete. 2024-04-25T13:58:47.170 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T13:58:47.411 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:47 smithi078 ceph-mon[29163]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:58:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:47 smithi112 ceph-mon[32617]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:58:47.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:47 smithi129 ceph-mon[32569]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T13:58:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:48 smithi078 ceph-mon[29163]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:48 smithi112 ceph-mon[32617]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:48 smithi129 ceph-mon[32569]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:49.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 -- 172.21.15.78:0/2735876607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 msgr2=0x7f66ec10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:49.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 --2- 172.21.15.78:0/2735876607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec10c580 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009990 tx=0x7f66dc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 -- 172.21.15.78:0/2735876607 shutdown_connections 2024-04-25T13:58:49.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 --2- 172.21.15.78:0/2735876607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec10c580 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 --2- 172.21.15.78:0/2735876607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec104200 0x7f66ec104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 --2- 172.21.15.78:0/2735876607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66ec105330 0x7f66ec105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 -- 172.21.15.78:0/2735876607 >> 172.21.15.78:0/2735876607 conn(0x7f66ec0ff840 msgr2=0x7f66ec101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 -- 172.21.15.78:0/2735876607 shutdown_connections 2024-04-25T13:58:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.252+0000 7f66f1716700 1 -- 172.21.15.78:0/2735876607 wait complete. 2024-04-25T13:58:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.253+0000 7f66f1716700 1 Processor -- start 2024-04-25T13:58:49.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.253+0000 7f66f1716700 1 -- start start 2024-04-25T13:58:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.253+0000 7f66f1716700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec104200 0x7f66ec19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66f1716700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66ec105330 0x7f66ec19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66f1716700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec1a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66f1716700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec10fc60 con 0x7f66ec105cd0 2024-04-25T13:58:49.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66f1716700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec10fae0 con 0x7f66ec104200 2024-04-25T13:58:49.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66f1716700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec10fde0 con 0x7f66ec105330 2024-04-25T13:58:49.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec1a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:49.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec1a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33468/0 (socket says 172.21.15.78:33468) 2024-04-25T13:58:49.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 -- 172.21.15.78:0/2256630441 learned_addr learned my addr 172.21.15.78:0/2256630441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:49.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eaffd700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec104200 0x7f66ec19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:49.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66ea7fc700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66ec105330 0x7f66ec19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:49.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 -- 172.21.15.78:0/2256630441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66ec105330 msgr2=0x7f66ec19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:49.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66ec105330 0x7f66ec19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 -- 172.21.15.78:0/2256630441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec104200 msgr2=0x7f66ec19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:49.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec104200 0x7f66ec19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 -- 172.21.15.78:0/2256630441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec1a2250 con 0x7f66ec105cd0 2024-04-25T13:58:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66ea7fc700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66ec105330 0x7f66ec19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:58:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66eb7fe700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec1a1bd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0038a0 tx=0x7f66dc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66d3fff700 1 -- 172.21.15.78:0/2256630441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc027070 con 0x7f66ec105cd0 2024-04-25T13:58:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec1a24e0 con 0x7f66ec105cd0 2024-04-25T13:58:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66d3fff700 1 -- 172.21.15.78:0/2256630441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66dc004000 con 0x7f66ec105cd0 2024-04-25T13:58:49.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66d3fff700 1 -- 172.21.15.78:0/2256630441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc022da0 con 0x7f66ec105cd0 2024-04-25T13:58:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.254+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec1a2990 con 0x7f66ec105cd0 2024-04-25T13:58:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.255+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ec051190 con 0x7f66ec105cd0 2024-04-25T13:58:49.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.257+0000 7f66d3fff700 1 -- 172.21.15.78:0/2256630441 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f66dc02a070 con 0x7f66ec105cd0 2024-04-25T13:58:49.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.257+0000 7f66d3fff700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66d40afb80 0x7f66d40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:49.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.257+0000 7f66eaffd700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66d40afb80 0x7f66d40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:49.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.258+0000 7f66d3fff700 1 -- 172.21.15.78:0/2256630441 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f66dc0dffb0 con 0x7f66ec105cd0 2024-04-25T13:58:49.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.258+0000 7f66eaffd700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66d40afb80 0x7f66d40b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009550 tx=0x7f66e40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:49.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.261+0000 7f66d3fff700 1 -- 172.21.15.78:0/2256630441 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66dc031020 con 0x7f66ec105cd0 2024-04-25T13:58:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.469+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f66ec063ad0 con 0x7f66d40afb80 2024-04-25T13:58:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:49 smithi078 ceph-mon[29163]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.475+0000 7f66d3fff700 1 -- 172.21.15.78:0/2256630441 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f66ec063ad0 con 0x7f66d40afb80 2024-04-25T13:58:49.475 INFO:teuthology.orchestra.run.smithi078.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T13:58:49.475 INFO:teuthology.orchestra.run.smithi078.stdout:mds.all 12/12 37s ago 2m smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi112=b;smithi112=e;smithi112=h;smithi112=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-04-25T13:58:49.475 INFO:teuthology.orchestra.run.smithi078.stdout:mgr 3/3 37s ago 7m smithi078=x;smithi112=y;smithi129=z;count:3 2024-04-25T13:58:49.475 INFO:teuthology.orchestra.run.smithi078.stdout:mon 3/3 37s ago 8m smithi078:172.21.15.78=a;smithi112:172.21.15.112=b;smithi129:172.21.15.129=c;count:3 2024-04-25T13:58:49.475 INFO:teuthology.orchestra.run.smithi078.stdout:osd 12 37s ago - 2024-04-25T13:58:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66d40afb80 msgr2=0x7f66d40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66d40afb80 0x7f66d40b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009550 tx=0x7f66e40091b0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 msgr2=0x7f66ec1a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec1a1bd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0038a0 tx=0x7f66dc005b00 comp rx=0 tx=0).stop 2024-04-25T13:58:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 shutdown_connections 2024-04-25T13:58:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66d40afb80 0x7f66d40b2090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ec105cd0 0x7f66ec1a1bd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec104200 0x7f66ec19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 --2- 172.21.15.78:0/2256630441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66ec105330 0x7f66ec19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 >> 172.21.15.78:0/2256630441 conn(0x7f66ec0ff840 msgr2=0x7f66ec108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 shutdown_connections 2024-04-25T13:58:49.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:49.477+0000 7f66f1716700 1 -- 172.21.15.78:0/2256630441 wait complete. 2024-04-25T13:58:49.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:49 smithi112 ceph-mon[32617]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:49.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:49 smithi129 ceph-mon[32569]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:50.144 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T13:58:50.383 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:50 smithi078 ceph-mon[29163]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:50 smithi078 ceph-mon[29163]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:50 smithi112 ceph-mon[32617]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:50 smithi112 ceph-mon[32617]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:50 smithi129 ceph-mon[32569]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:50 smithi129 ceph-mon[32569]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T13:58:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.216+0000 7ff070d9c700 1 -- 172.21.15.78:0/855647702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 msgr2=0x7ff06c10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.216+0000 7ff070d9c700 1 --2- 172.21.15.78:0/855647702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c10b460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff06400b0a0 tx=0x7ff06400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.216+0000 7ff070d9c700 1 -- 172.21.15.78:0/855647702 shutdown_connections 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.216+0000 7ff070d9c700 1 --2- 172.21.15.78:0/855647702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c10b460 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.216+0000 7ff070d9c700 1 --2- 172.21.15.78:0/855647702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c1030e0 0x7ff06c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.216+0000 7ff070d9c700 1 --2- 172.21.15.78:0/855647702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff06c104210 0x7ff06c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.216+0000 7ff070d9c700 1 -- 172.21.15.78:0/855647702 >> 172.21.15.78:0/855647702 conn(0x7ff06c0fe740 msgr2=0x7ff06c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.217+0000 7ff070d9c700 1 -- 172.21.15.78:0/855647702 shutdown_connections 2024-04-25T13:58:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.217+0000 7ff070d9c700 1 -- 172.21.15.78:0/855647702 wait complete. 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.217+0000 7ff070d9c700 1 Processor -- start 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.217+0000 7ff070d9c700 1 -- start start 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.217+0000 7ff070d9c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff06c1030e0 0x7ff06c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff070d9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c104210 0x7ff06c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff070d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff070d9c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10eb60 con 0x7ff06c104bb0 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff070d9c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10e9e0 con 0x7ff06c104210 2024-04-25T13:58:52.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff070d9c700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10ece0 con 0x7ff06c1030e0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c1a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33486/0 (socket says 172.21.15.78:33486) 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 -- 172.21.15.78:0/1572560591 learned_addr learned my addr 172.21.15.78:0/1572560591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06a59c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff06c1030e0 0x7ff06c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff069d9b700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c104210 0x7ff06c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 -- 172.21.15.78:0/1572560591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff06c1030e0 msgr2=0x7ff06c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff06c1030e0 0x7ff06c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 -- 172.21.15.78:0/1572560591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c104210 msgr2=0x7ff06c19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c104210 0x7ff06c19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 -- 172.21.15.78:0/1572560591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c1a2200 con 0x7ff06c104bb0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff06ad9d700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c1a1c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff0640062a0 tx=0x7ff064004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff05b7fe700 1 -- 172.21.15.78:0/1572560591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064021070 con 0x7ff06c104bb0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff05b7fe700 1 -- 172.21.15.78:0/1572560591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff064007c80 con 0x7ff06c104bb0 2024-04-25T13:58:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c1a2490 con 0x7ff06c104bb0 2024-04-25T13:58:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.218+0000 7ff05b7fe700 1 -- 172.21.15.78:0/1572560591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064004850 con 0x7ff06c104bb0 2024-04-25T13:58:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.219+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c1a29d0 con 0x7ff06c104bb0 2024-04-25T13:58:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.220+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c051190 con 0x7ff06c104bb0 2024-04-25T13:58:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.221+0000 7ff05b7fe700 1 -- 172.21.15.78:0/1572560591 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff064003a00 con 0x7ff06c104bb0 2024-04-25T13:58:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.221+0000 7ff05b7fe700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff0540afb80 0x7ff0540b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.221+0000 7ff06a59c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff0540afb80 0x7ff0540b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.222+0000 7ff05b7fe700 1 -- 172.21.15.78:0/1572560591 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff06400f020 con 0x7ff06c104bb0 2024-04-25T13:58:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.222+0000 7ff06a59c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff0540afb80 0x7ff0540b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff06c104110 tx=0x7ff05c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.226+0000 7ff05b7fe700 1 -- 172.21.15.78:0/1572560591 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff06402a020 con 0x7ff06c104bb0 2024-04-25T13:58:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.437+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ff06c06bbf0 con 0x7ff0540afb80 2024-04-25T13:58:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.438+0000 7ff05b7fe700 1 -- 172.21.15.78:0/1572560591 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7ff06c06bbf0 con 0x7ff0540afb80 2024-04-25T13:58:52.438 INFO:teuthology.orchestra.run.smithi078.stdout:HOST ADDR LABELS STATUS 2024-04-25T13:58:52.438 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 172.21.15.78 2024-04-25T13:58:52.438 INFO:teuthology.orchestra.run.smithi078.stdout:smithi112 172.21.15.112 2024-04-25T13:58:52.438 INFO:teuthology.orchestra.run.smithi078.stdout:smithi129 172.21.15.129 2024-04-25T13:58:52.438 INFO:teuthology.orchestra.run.smithi078.stdout:3 hosts in cluster 2024-04-25T13:58:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff0540afb80 msgr2=0x7ff0540b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff0540afb80 0x7ff0540b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff06c104110 tx=0x7ff05c00a3b0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 msgr2=0x7ff06c1a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c1a1c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff0640062a0 tx=0x7ff064004e40 comp rx=0 tx=0).stop 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 shutdown_connections 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff0540afb80 0x7ff0540b2090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff06c104bb0 0x7ff06c1a1c10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff06c1030e0 0x7ff06c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 --2- 172.21.15.78:0/1572560591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c104210 0x7ff06c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 >> 172.21.15.78:0/1572560591 conn(0x7ff06c0fe740 msgr2=0x7ff06c06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 shutdown_connections 2024-04-25T13:58:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:52.440+0000 7ff070d9c700 1 -- 172.21.15.78:0/1572560591 wait complete. 2024-04-25T13:58:53.151 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T13:58:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:53 smithi078 ceph-mon[29163]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:58:53.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:53 smithi112 ceph-mon[32617]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:58:53.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:53 smithi129 ceph-mon[32569]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:58:54.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:54 smithi078 ceph-mon[29163]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:54.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:54 smithi112 ceph-mon[32617]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:54 smithi129 ceph-mon[32569]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:55.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 -- 172.21.15.78:0/3953623595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 msgr2=0x7fef0410b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3953623595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef0410b4a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feefc00b0a0 tx=0x7feefc00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 -- 172.21.15.78:0/3953623595 shutdown_connections 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3953623595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef0410b4a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3953623595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04103120 0x7fef04103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3953623595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef04104250 0x7fef04104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 -- 172.21.15.78:0/3953623595 >> 172.21.15.78:0/3953623595 conn(0x7fef040fe760 msgr2=0x7fef04100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 -- 172.21.15.78:0/3953623595 shutdown_connections 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 -- 172.21.15.78:0/3953623595 wait complete. 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 Processor -- start 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.111+0000 7fef0aefc700 1 -- start start 2024-04-25T13:58:55.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef0aefc700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef04103120 0x7fef041997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef0aefc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04104250 0x7fef04199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef0aefc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef041a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef0aefc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410ec80 con 0x7fef04104bf0 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef0aefc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410eb00 con 0x7fef04104250 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef0aefc700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410ee00 con 0x7fef04103120 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef041a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef041a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33492/0 (socket says 172.21.15.78:33492) 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 -- 172.21.15.78:0/3826402125 learned_addr learned my addr 172.21.15.78:0/3826402125 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef08c98700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef04103120 0x7fef041997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef03fff700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04104250 0x7fef04199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 -- 172.21.15.78:0/3826402125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef04103120 msgr2=0x7fef041997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef04103120 0x7fef041997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 -- 172.21.15.78:0/3826402125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04104250 msgr2=0x7fef04199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04104250 0x7fef04199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 -- 172.21.15.78:0/3826402125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041a1330 con 0x7fef04104bf0 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef09499700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef041a0cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7feefc00b070 tx=0x7feefc008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef01ffb700 1 -- 172.21.15.78:0/3826402125 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc021070 con 0x7fef04104bf0 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef01ffb700 1 -- 172.21.15.78:0/3826402125 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc007c00 con 0x7fef04104bf0 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a15c0 con 0x7fef04104bf0 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.112+0000 7fef01ffb700 1 -- 172.21.15.78:0/3826402125 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc01b6b0 con 0x7fef04104bf0 2024-04-25T13:58:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.113+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a1a90 con 0x7fef04104bf0 2024-04-25T13:58:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.114+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef04051190 con 0x7fef04104bf0 2024-04-25T13:58:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.115+0000 7fef01ffb700 1 -- 172.21.15.78:0/3826402125 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7feefc024070 con 0x7fef04104bf0 2024-04-25T13:58:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.115+0000 7fef01ffb700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feef40afb80 0x7feef40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.116+0000 7fef01ffb700 1 -- 172.21.15.78:0/3826402125 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7feefc00f020 con 0x7fef04104bf0 2024-04-25T13:58:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.116+0000 7fef08c98700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feef40afb80 0x7feef40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.116+0000 7fef08c98700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feef40afb80 0x7feef40b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fef04104150 tx=0x7feef8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.120+0000 7fef01ffb700 1 -- 172.21.15.78:0/3826402125 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feefc02b020 con 0x7fef04104bf0 2024-04-25T13:58:55.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.331+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fef04063ad0 con 0x7feef40afb80 2024-04-25T13:58:55.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.333+0000 7fef01ffb700 1 -- 172.21.15.78:0/3826402125 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fef04063ad0 con 0x7feef40afb80 2024-04-25T13:58:55.332 INFO:teuthology.orchestra.run.smithi078.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T13:58:55.332 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T13:58:55.332 INFO:teuthology.orchestra.run.smithi078.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:58:55.333 INFO:teuthology.orchestra.run.smithi078.stdout:smithi129 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300HY400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feef40afb80 msgr2=0x7feef40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feef40afb80 0x7feef40b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fef04104150 tx=0x7feef8009190 comp rx=0 tx=0).stop 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 msgr2=0x7fef041a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef041a0cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7feefc00b070 tx=0x7feefc008ff0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 shutdown_connections 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feef40afb80 0x7feef40b2090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef04104bf0 0x7fef041a0cb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef04103120 0x7fef041997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 --2- 172.21.15.78:0/3826402125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef04104250 0x7fef04199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:55.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 >> 172.21.15.78:0/3826402125 conn(0x7fef040fe760 msgr2=0x7fef04107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:55.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 shutdown_connections 2024-04-25T13:58:55.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:55.335+0000 7fef0aefc700 1 -- 172.21.15.78:0/3826402125 wait complete. 2024-04-25T13:58:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:55 smithi078 ceph-mon[29163]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:58:55.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:55 smithi112 ceph-mon[32617]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:58:55.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:55 smithi129 ceph-mon[32569]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:58:56.215 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:58:56.224 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T13:58:56.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T13:58:56.464 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:56.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:56 smithi078 ceph-mon[29163]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:56 smithi078 ceph-mon[29163]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:56 smithi112 ceph-mon[32617]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:56 smithi112 ceph-mon[32617]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:56 smithi129 ceph-mon[32569]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:58:56.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:56 smithi129 ceph-mon[32569]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 -- 172.21.15.78:0/465256000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 msgr2=0x7f17e410c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 --2- 172.21.15.78:0/465256000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e410c790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00b0a0 tx=0x7f17dc00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 -- 172.21.15.78:0/465256000 shutdown_connections 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 --2- 172.21.15.78:0/465256000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e410c790 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 --2- 172.21.15.78:0/465256000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e41057d0 0x7f17e4105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 --2- 172.21.15.78:0/465256000 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e4102220 0x7f17e4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 -- 172.21.15.78:0/465256000 >> 172.21.15.78:0/465256000 conn(0x7f17e40fdef0 msgr2=0x7f17e41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 -- 172.21.15.78:0/465256000 shutdown_connections 2024-04-25T13:58:58.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 -- 172.21.15.78:0/465256000 wait complete. 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.215+0000 7f17e908e700 1 Processor -- start 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e908e700 1 -- start start 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e908e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e4102220 0x7f17e419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e908e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e908e700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e41057d0 0x7f17e41a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e908e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17e410fcb0 con 0x7f17e4102b70 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e908e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17e410fb30 con 0x7f17e4102220 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e908e700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17e410fe30 con 0x7f17e41057d0 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52176/0 (socket says 172.21.15.78:52176) 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 -- 172.21.15.78:0/535884029 learned_addr learned my addr 172.21.15.78:0/535884029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e2d9d700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e4102220 0x7f17e419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e359e700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e41057d0 0x7f17e41a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 -- 172.21.15.78:0/535884029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e41057d0 msgr2=0x7f17e41a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e41057d0 0x7f17e41a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 -- 172.21.15.78:0/535884029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e4102220 msgr2=0x7f17e419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e4102220 0x7f17e419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.216+0000 7f17e259c700 1 -- 172.21.15.78:0/535884029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e41a22c0 con 0x7f17e4102b70 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.217+0000 7f17e259c700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e419aca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17d800d980 tx=0x7f17d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.217+0000 7f17cbfff700 1 -- 172.21.15.78:0/535884029 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8016070 con 0x7f17e4102b70 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.217+0000 7f17cbfff700 1 -- 172.21.15.78:0/535884029 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d8004830 con 0x7f17e4102b70 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.217+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e41a25b0 con 0x7f17e4102b70 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.217+0000 7f17cbfff700 1 -- 172.21.15.78:0/535884029 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8005540 con 0x7f17e4102b70 2024-04-25T13:58:58.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.217+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e41a2af0 con 0x7f17e4102b70 2024-04-25T13:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.218+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e4051190 con 0x7f17e4102b70 2024-04-25T13:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.219+0000 7f17cbfff700 1 -- 172.21.15.78:0/535884029 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f17d8004220 con 0x7f17e4102b70 2024-04-25T13:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.220+0000 7f17cbfff700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17cc0afb80 0x7f17cc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:58:58.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.220+0000 7f17e2d9d700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17cc0afb80 0x7f17cc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.220+0000 7f17cbfff700 1 -- 172.21.15.78:0/535884029 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f17d80def50 con 0x7f17e4102b70 2024-04-25T13:58:58.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.221+0000 7f17e2d9d700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17cc0afb80 0x7f17cc0b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f17e406bf50 tx=0x7f17d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:58:58.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.223+0000 7f17cbfff700 1 -- 172.21.15.78:0/535884029 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17d801f020 con 0x7f17e4102b70 2024-04-25T13:58:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.467+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f17e402e510 con 0x7f17e4102b70 2024-04-25T13:58:58.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.468+0000 7f17cbfff700 1 -- 172.21.15.78:0/535884029 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3050 (secure 0 0 0) 0x7f17d80a7b20 con 0x7f17e4102b70 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:e21 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:legacy client fscid: 1 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:Filesystem 'cephfs' (1) 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:fs_name cephfs 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 21 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T13:58:58.468 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-04-25T13:57:11.484343+0000 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-04-25T13:57:16.081259+0000 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:tableserver 0 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:root 0 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:session_timeout 300 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:session_autoclose 300 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:max_file_size 1099511627776 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:max_xattr_size 65536 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:required_client_features {} 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure 0 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure_osd_epoch 0 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:max_mds 3 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:in 0,1,2 2024-04-25T13:58:58.469 INFO:teuthology.orchestra.run.smithi078.stdout:up {0=24512,1=24484,2=24454} 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:failed 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:damaged 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:stopped 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:data_pools [3] 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:metadata_pool 2 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:inline_data disabled 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:balancer 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:bal_rank_mask -1 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:standby_count_wanted 0 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:qdb_cluster leader: 24512 members: 24454,24512,24484 2024-04-25T13:58:58.470 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b{0:24512} state up:active seq 3 addr [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.h{0:24476} state up:standby-replay seq 1 addr [v2:172.21.15.112:6836/4152465978,v1:172.21.15.112:6837/4152465978] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.i{1:24484} state up:active seq 6 addr [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c{1:24466} state up:standby-replay seq 1 addr [v2:172.21.15.129:6836/750801105,v1:172.21.15.129:6837/750801105] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.f{2:24454} state up:active seq 8 addr [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.e{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.112:6834/2622963442,v1:172.21.15.112:6835/2622963442] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:Standby daemons: 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.j{-1:14571} state up:standby seq 1 addr [v2:172.21.15.78:6834/835077889,v1:172.21.15.78:6835/835077889] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d{-1:14583} state up:standby seq 1 addr [v2:172.21.15.78:6836/3031062077,v1:172.21.15.78:6837/3031062077] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.g{-1:14640} state up:standby seq 1 addr [v2:172.21.15.78:6838/2808782144,v1:172.21.15.78:6839/2808782144] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a{-1:14652} state up:standby seq 1 addr [v2:172.21.15.78:6840/1109968953,v1:172.21.15.78:6841/1109968953] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.471 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.l{-1:24412} state up:standby seq 1 addr [v2:172.21.15.129:6832/1065513259,v1:172.21.15.129:6833/1065513259] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.472 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.k{-1:24416} state up:standby seq 1 addr [v2:172.21.15.112:6832/3808802757,v1:172.21.15.112:6833/3808802757] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:58:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17cc0afb80 msgr2=0x7f17cc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17cc0afb80 0x7f17cc0b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f17e406bf50 tx=0x7f17d400b410 comp rx=0 tx=0).stop 2024-04-25T13:58:58.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 msgr2=0x7f17e419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e419aca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17d800d980 tx=0x7f17d800dc90 comp rx=0 tx=0).stop 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 shutdown_connections 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f17cc0afb80 0x7f17cc0b2090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e41057d0 0x7f17e41a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e4102220 0x7f17e419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 --2- 172.21.15.78:0/535884029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17e4102b70 0x7f17e419aca0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 >> 172.21.15.78:0/535884029 conn(0x7f17e40fdef0 msgr2=0x7f17e4108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 shutdown_connections 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:58:58.471+0000 7f17e908e700 1 -- 172.21.15.78:0/535884029 wait complete. 2024-04-25T13:58:58.473 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 21 2024-04-25T13:58:59.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T13:58:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:59 smithi078 ceph-mon[29163]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:58:59 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/535884029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:58:59.482 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:58:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:59 smithi112 ceph-mon[32617]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:59.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:58:59 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/535884029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:58:59.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:59 smithi129 ceph-mon[32569]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:58:59.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:58:59 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/535884029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:59:01.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 -- 172.21.15.78:0/3549384562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104210 msgr2=0x7efe7c104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 --2- 172.21.15.78:0/3549384562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104210 0x7efe7c104620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009990 tx=0x7efe6c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 -- 172.21.15.78:0/3549384562 shutdown_connections 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 --2- 172.21.15.78:0/3549384562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7c104bb0 0x7efe7c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 --2- 172.21.15.78:0/3549384562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe7c1030e0 0x7efe7c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 --2- 172.21.15.78:0/3549384562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104210 0x7efe7c104620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 -- 172.21.15.78:0/3549384562 >> 172.21.15.78:0/3549384562 conn(0x7efe7c0fe740 msgr2=0x7efe7c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 -- 172.21.15.78:0/3549384562 shutdown_connections 2024-04-25T13:59:01.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.329+0000 7efe81099700 1 -- 172.21.15.78:0/3549384562 wait complete. 2024-04-25T13:59:01.329 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:01 smithi078 ceph-mon[29163]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:01.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 Processor -- start 2024-04-25T13:59:01.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 -- start start 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe7c1030e0 0x7efe7c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7c104210 0x7efe7c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104bb0 0x7efe7c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c10eb50 con 0x7efe7c104bb0 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c10e9d0 con 0x7efe7c104210 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.330+0000 7efe81099700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c10ecd0 con 0x7efe7c1030e0 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104bb0 0x7efe7c1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104bb0 0x7efe7c1a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52190/0 (socket says 172.21.15.78:52190) 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 -- 172.21.15.78:0/113550881 learned_addr learned my addr 172.21.15.78:0/113550881 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7a59c700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7c104210 0x7efe7c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7ad9d700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe7c1030e0 0x7efe7c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 -- 172.21.15.78:0/113550881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe7c1030e0 msgr2=0x7efe7c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:01.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe7c1030e0 0x7efe7c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 -- 172.21.15.78:0/113550881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7c104210 msgr2=0x7efe7c19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7c104210 0x7efe7c19ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 -- 172.21.15.78:0/113550881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c1a2270 con 0x7efe7c104bb0 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe7b59e700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104bb0 0x7efe7c1a1c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe7400ada0 tx=0x7efe7400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe63fff700 1 -- 172.21.15.78:0/113550881 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74017020 con 0x7efe7c104bb0 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe63fff700 1 -- 172.21.15.78:0/113550881 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7400cd80 con 0x7efe7c104bb0 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c1a2560 con 0x7efe7c104bb0 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe63fff700 1 -- 172.21.15.78:0/113550881 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74005260 con 0x7efe7c104bb0 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.331+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c1a2aa0 con 0x7efe7c104bb0 2024-04-25T13:59:01.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.333+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7c051190 con 0x7efe7c104bb0 2024-04-25T13:59:01.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.334+0000 7efe63fff700 1 -- 172.21.15.78:0/113550881 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efe7400a090 con 0x7efe7c104bb0 2024-04-25T13:59:01.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.335+0000 7efe63fff700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efe640afc00 0x7efe640b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:01.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.335+0000 7efe63fff700 1 -- 172.21.15.78:0/113550881 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efe740de880 con 0x7efe7c104bb0 2024-04-25T13:59:01.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.335+0000 7efe7ad9d700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efe640afc00 0x7efe640b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:01.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.335+0000 7efe7ad9d700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efe640afc00 0x7efe640b2110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe7c104110 tx=0x7efe7000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:01.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.339+0000 7efe63fff700 1 -- 172.21.15.78:0/113550881 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe74021020 con 0x7efe7c104bb0 2024-04-25T13:59:01.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:01 smithi112 ceph-mon[32617]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:01.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.546+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7efe7c1a2ec0 con 0x7efe7c104bb0 2024-04-25T13:59:01.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.547+0000 7efe63fff700 1 -- 172.21.15.78:0/113550881 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5321 (secure 0 0 0) 0x7efe740a74e0 con 0x7efe7c104bb0 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 77 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-04-25T13:49:14.716038+0000 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-04-25T13:58:09.203005+0000 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:crush_version 26 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:full_ratio 0.95 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:backfillfull_ratio 0.9 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:nearfull_ratio 0.85 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:require_min_compat_client luminous 2024-04-25T13:59:01.546 INFO:teuthology.orchestra.run.smithi078.stdout:min_compat_client luminous 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:require_osd_release squid 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:stretch_mode_enabled false 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:max_osd 12 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 up in weight 1 up_from 8 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6802/2736058336,v1:172.21.15.78:6803/2736058336] [v2:172.21.15.78:6804/2736058336,v1:172.21.15.78:6805/2736058336] exists,up caa5f025-762a-4cf2-a61e-9be6a4dd8013 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 up in weight 1 up_from 13 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6800/1864123277,v1:172.21.15.112:6801/1864123277] [v2:172.21.15.112:6802/1864123277,v1:172.21.15.112:6803/1864123277] exists,up a258ec36-259b-448b-8103-de5ddfbbcfdc 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 up in weight 1 up_from 18 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/1698371012,v1:172.21.15.129:6801/1698371012] [v2:172.21.15.129:6802/1698371012,v1:172.21.15.129:6803/1698371012] exists,up 25bc85c2-81d0-479f-8b08-e5c37a158b38 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 up in weight 1 up_from 25 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6810/2856659666,v1:172.21.15.78:6811/2856659666] [v2:172.21.15.78:6812/2856659666,v1:172.21.15.78:6813/2856659666] exists,up d9b02621-7cdb-480e-863d-09bfc670da4f 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 up in weight 1 up_from 31 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6808/3169717920,v1:172.21.15.112:6809/3169717920] [v2:172.21.15.112:6810/3169717920,v1:172.21.15.112:6811/3169717920] exists,up b298aa83-5099-4f31-a030-f3b449241d55 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 up in weight 1 up_from 36 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/1391792585,v1:172.21.15.129:6809/1391792585] [v2:172.21.15.129:6810/1391792585,v1:172.21.15.129:6811/1391792585] exists,up 5283d1e6-8c10-4e92-8c1c-baf015823098 2024-04-25T13:59:01.547 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 up in weight 1 up_from 42 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6818/3216701822,v1:172.21.15.78:6819/3216701822] [v2:172.21.15.78:6820/3216701822,v1:172.21.15.78:6821/3216701822] exists,up bee711c4-03f1-463b-a61a-208445896ac9 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 up in weight 1 up_from 47 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6816/1198451048,v1:172.21.15.112:6817/1198451048] [v2:172.21.15.112:6818/1198451048,v1:172.21.15.112:6819/1198451048] exists,up 74bfe646-13d1-471c-acf9-8f84969d2313 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 up in weight 1 up_from 53 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/4162935801,v1:172.21.15.129:6817/4162935801] [v2:172.21.15.129:6818/4162935801,v1:172.21.15.129:6819/4162935801] exists,up 2220b155-081a-4bdd-96c2-e46f8e20b97d 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 up in weight 1 up_from 59 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6826/144296298,v1:172.21.15.78:6827/144296298] [v2:172.21.15.78:6828/144296298,v1:172.21.15.78:6829/144296298] exists,up 6395fc39-cdba-4a65-bfd5-1dd36eaaa385 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 up in weight 1 up_from 64 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6824/1454658808,v1:172.21.15.112:6825/1454658808] [v2:172.21.15.112:6826/1454658808,v1:172.21.15.112:6827/1454658808] exists,up a6e1b875-eeac-47df-a4e1-df92fa867c4b 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 up in weight 1 up_from 70 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6824/2167746323,v1:172.21.15.129:6825/2167746323] [v2:172.21.15.129:6826/2167746323,v1:172.21.15.129:6827/2167746323] exists,up 78830a45-c9f6-4c81-93ae-dbb8a1dc61db 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.e [2,1] 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.1b [2,4] 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.1f [7,11] 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.0 [4,6] 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.1d [4,8,5,9] 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.23 [4,7] 2024-04-25T13:59:01.548 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.25 [5,9] 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.29 [3,9] 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/360002043 expires 2024-04-26T13:50:08.037919+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/1759244565 expires 2024-04-26T13:50:08.037919+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2174092153 expires 2024-04-26T13:49:41.756206+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/3823505283 expires 2024-04-26T13:49:41.756206+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/3823505283 expires 2024-04-26T13:49:41.756206+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/3906244379 expires 2024-04-26T13:50:08.037919+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2353565210 expires 2024-04-26T13:50:08.037919+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/231757119 expires 2024-04-26T13:49:41.756206+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1444951251 expires 2024-04-26T13:49:41.756206+0000 2024-04-25T13:59:01.549 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/1759244565 expires 2024-04-26T13:50:08.037919+0000 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efe640afc00 msgr2=0x7efe640b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efe640afc00 0x7efe640b2110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe7c104110 tx=0x7efe7000a3b0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104bb0 msgr2=0x7efe7c1a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104bb0 0x7efe7c1a1c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efe7400ada0 tx=0x7efe7400c330 comp rx=0 tx=0).stop 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 shutdown_connections 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efe640afc00 0x7efe640b2110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe7c104bb0 0x7efe7c1a1c80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe7c1030e0 0x7efe7c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 --2- 172.21.15.78:0/113550881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe7c104210 0x7efe7c19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:01.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 >> 172.21.15.78:0/113550881 conn(0x7efe7c0fe740 msgr2=0x7efe7c06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:01.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 shutdown_connections 2024-04-25T13:59:01.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:01.549+0000 7efe81099700 1 -- 172.21.15.78:0/113550881 wait complete. 2024-04-25T13:59:01.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:01 smithi129 ceph-mon[32569]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:02.229 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T13:59:02.242 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:59:02.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 -- 172.21.15.78:0/2010685346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c101e60 msgr2=0x7f452c106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2010685346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c101e60 0x7f452c106b20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f451c009ca0 tx=0x7f451c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 -- 172.21.15.78:0/2010685346 shutdown_connections 2024-04-25T13:59:02.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2010685346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f452c109780 0x7f452c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2010685346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c102360 0x7f452c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2010685346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c101e60 0x7f452c106b20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 -- 172.21.15.78:0/2010685346 >> 172.21.15.78:0/2010685346 conn(0x7f452c0fd370 msgr2=0x7f452c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:02.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 -- 172.21.15.78:0/2010685346 shutdown_connections 2024-04-25T13:59:02.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.354+0000 7f4530b6c640 1 -- 172.21.15.78:0/2010685346 wait complete. 2024-04-25T13:59:02.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 Processor -- start 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 -- start start 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f452c101e60 0x7f452c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c102360 0x7f452c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c109780 0x7f452c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10f5e0 con 0x7f452c109780 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10f460 con 0x7f452c102360 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f4530b6c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10f760 con 0x7f452c101e60 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f452ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c109780 0x7f452c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f452ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c109780 0x7f452c1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52216/0 (socket says 172.21.15.78:52216) 2024-04-25T13:59:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f452ad9d640 1 -- 172.21.15.78:0/2975345736 learned_addr learned my addr 172.21.15.78:0/2975345736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.355+0000 7f452a59c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f452c101e60 0x7f452c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f4529d9b640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c102360 0x7f452c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f452ad9d640 1 -- 172.21.15.78:0/2975345736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f452c101e60 msgr2=0x7f452c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f452ad9d640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f452c101e60 0x7f452c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f452ad9d640 1 -- 172.21.15.78:0/2975345736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c102360 msgr2=0x7f452c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f452ad9d640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c102360 0x7f452c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f452ad9d640 1 -- 172.21.15.78:0/2975345736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c1a1d00 con 0x7f452c109780 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f452ad9d640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c109780 0x7f452c1a1690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f452400d750 tx=0x7f452400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f451b7fe640 1 -- 172.21.15.78:0/2975345736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524017070 con 0x7f452c109780 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f451b7fe640 1 -- 172.21.15.78:0/2975345736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45240054d0 con 0x7f452c109780 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c1a1ff0 con 0x7f452c109780 2024-04-25T13:59:02.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f451b7fe640 1 -- 172.21.15.78:0/2975345736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524004c60 con 0x7f452c109780 2024-04-25T13:59:02.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.356+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a2450 con 0x7f452c109780 2024-04-25T13:59:02.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.357+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c0fee90 con 0x7f452c109780 2024-04-25T13:59:02.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.359+0000 7f451b7fe640 1 -- 172.21.15.78:0/2975345736 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4524009bd0 con 0x7f452c109780 2024-04-25T13:59:02.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.359+0000 7f451b7fe640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f45140afaf0 0x7f45140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.359+0000 7f452a59c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f45140afaf0 0x7f45140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.359+0000 7f451b7fe640 1 -- 172.21.15.78:0/2975345736 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f45240de880 con 0x7f452c109780 2024-04-25T13:59:02.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.360+0000 7f452a59c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f45140afaf0 0x7f45140b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f451c0094e0 tx=0x7f451c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:02.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.363+0000 7f451b7fe640 1 -- 172.21.15.78:0/2975345736 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45240a6e00 con 0x7f452c109780 2024-04-25T13:59:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:02 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/113550881' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:59:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:02 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/113550881' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:59:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.541+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f452c02dda0 con 0x7f452c109780 2024-04-25T13:59:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.542+0000 7f451b7fe640 1 -- 172.21.15.78:0/2975345736 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f45240ab100 con 0x7f452c109780 2024-04-25T13:59:02.541 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:02.542 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.544+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f45140afaf0 msgr2=0x7f45140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.544+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f45140afaf0 0x7f45140b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f451c0094e0 tx=0x7f451c004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.544+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c109780 msgr2=0x7f452c1a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.544+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c109780 0x7f452c1a1690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f452400d750 tx=0x7f452400dc10 comp rx=0 tx=0).stop 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 shutdown_connections 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f45140afaf0 0x7f45140b2000 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f452c109780 0x7f452c1a1690 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f452c102360 0x7f452c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 --2- 172.21.15.78:0/2975345736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f452c101e60 0x7f452c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 >> 172.21.15.78:0/2975345736 conn(0x7f452c0fd370 msgr2=0x7f452c105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 shutdown_connections 2024-04-25T13:59:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.545+0000 7f4530b6c640 1 -- 172.21.15.78:0/2975345736 wait complete. 2024-04-25T13:59:02.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:59:02.556 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:59:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:02 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/113550881' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 -- 172.21.15.78:0/4245073881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00109a30 msgr2=0x7fda0010be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 --2- 172.21.15.78:0/4245073881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00109a30 0x7fda0010be50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009ca0 tx=0x7fd9f802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 -- 172.21.15.78:0/4245073881 shutdown_connections 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 --2- 172.21.15.78:0/4245073881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00109a30 0x7fda0010be50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 --2- 172.21.15.78:0/4245073881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda00104d30 0x7fda00105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 --2- 172.21.15.78:0/4245073881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda00103af0 0x7fda001047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 -- 172.21.15.78:0/4245073881 >> 172.21.15.78:0/4245073881 conn(0x7fda000fece0 msgr2=0x7fda00101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 -- 172.21.15.78:0/4245073881 shutdown_connections 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.667+0000 7fda06382640 1 -- 172.21.15.78:0/4245073881 wait complete. 2024-04-25T13:59:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 Processor -- start 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 -- start start 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00103af0 0x7fda0019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda00104d30 0x7fda0019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda00109a30 0x7fda001a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010f620 con 0x7fda00103af0 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010f4a0 con 0x7fda00109a30 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda06382640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010f7a0 con 0x7fda00104d30 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fd9fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00103af0 0x7fda0019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fd9fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00103af0 0x7fda0019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52242/0 (socket says 172.21.15.78:52242) 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fd9fffff640 1 -- 172.21.15.78:0/1084565123 learned_addr learned my addr 172.21.15.78:0/1084565123 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fda0491f640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda00109a30 0x7fda001a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fd9ff7fe640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda00104d30 0x7fda0019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fd9fffff640 1 -- 172.21.15.78:0/1084565123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda00104d30 msgr2=0x7fda0019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.668+0000 7fd9fffff640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda00104d30 0x7fda0019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fd9fffff640 1 -- 172.21.15.78:0/1084565123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda00109a30 msgr2=0x7fda001a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fd9fffff640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda00109a30 0x7fda001a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fd9fffff640 1 -- 172.21.15.78:0/1084565123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda001a1d40 con 0x7fda00103af0 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fd9fffff640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00103af0 0x7fda0019a200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0005d00 tx=0x7fd9f000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fd9fd7fa640 1 -- 172.21.15.78:0/1084565123 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0017070 con 0x7fda00103af0 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fd9fd7fa640 1 -- 172.21.15.78:0/1084565123 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9f0009c30 con 0x7fda00103af0 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda001a1fd0 con 0x7fda00103af0 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fd9fd7fa640 1 -- 172.21.15.78:0/1084565123 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000f620 con 0x7fda00103af0 2024-04-25T13:59:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.669+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda001a2440 con 0x7fda00103af0 2024-04-25T13:59:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.670+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda00108840 con 0x7fda00103af0 2024-04-25T13:59:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.671+0000 7fd9fd7fa640 1 -- 172.21.15.78:0/1084565123 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd9f001a070 con 0x7fda00103af0 2024-04-25T13:59:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.672+0000 7fd9fd7fa640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9e80afaf0 0x7fd9e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.672+0000 7fd9ff7fe640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9e80afaf0 0x7fd9e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.672+0000 7fd9fd7fa640 1 -- 172.21.15.78:0/1084565123 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd9f00de700 con 0x7fda00103af0 2024-04-25T13:59:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.672+0000 7fd9ff7fe640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9e80afaf0 0x7fd9e80b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005fd0 tx=0x7fd9f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.675+0000 7fd9fd7fa640 1 -- 172.21.15.78:0/1084565123 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9f0009de0 con 0x7fda00103af0 2024-04-25T13:59:02.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.851+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda0002db50 con 0x7fda00103af0 2024-04-25T13:59:02.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.851+0000 7fd9fd7fa640 1 -- 172.21.15.78:0/1084565123 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fd9f00a72d0 con 0x7fda00103af0 2024-04-25T13:59:02.850 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9e80afaf0 msgr2=0x7fd9e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9e80afaf0 0x7fd9e80b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005fd0 tx=0x7fd9f4005850 comp rx=0 tx=0).stop 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00103af0 msgr2=0x7fda0019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00103af0 0x7fda0019a200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0005d00 tx=0x7fd9f000dde0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 shutdown_connections 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda00109a30 0x7fda001a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda00104d30 0x7fda0019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd9e80afaf0 0x7fd9e80b2000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 --2- 172.21.15.78:0/1084565123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda00103af0 0x7fda0019a200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 >> 172.21.15.78:0/1084565123 conn(0x7fda000fece0 msgr2=0x7fda00105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 shutdown_connections 2024-04-25T13:59:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.854+0000 7fda06382640 1 -- 172.21.15.78:0/1084565123 wait complete. 2024-04-25T13:59:02.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:59:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/125547323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4108920 msgr2=0x7fc8f410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/125547323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4108920 0x7fc8f410ad40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ca0 tx=0x7fc8ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/125547323 shutdown_connections 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/125547323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4108920 0x7fc8f410ad40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/125547323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8f4103c20 0x7fc8f4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/125547323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8f41029e0 0x7fc8f41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/125547323 >> 172.21.15.78:0/125547323 conn(0x7fc8f40fdbd0 msgr2=0x7fc8f4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/125547323 shutdown_connections 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/125547323 wait complete. 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 Processor -- start 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.976+0000 7fc8fb4e9640 1 -- start start 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8f41029e0 0x7fc8f406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4103c20 0x7fc8f419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8f4108920 0x7fc8f41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f410e510 con 0x7fc8f4103c20 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f410e390 con 0x7fc8f41029e0 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f410e690 con 0x7fc8f4108920 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4103c20 0x7fc8f419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4103c20 0x7fc8f419f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52270/0 (socket says 172.21.15.78:52270) 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 -- 172.21.15.78:0/1911444464 learned_addr learned my addr 172.21.15.78:0/1911444464 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f9285640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8f41029e0 0x7fc8f406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 -- 172.21.15.78:0/1911444464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8f4108920 msgr2=0x7fc8f41a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8f4108920 0x7fc8f41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 -- 172.21.15.78:0/1911444464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8f41029e0 msgr2=0x7fc8f406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8f41029e0 0x7fc8f406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 -- 172.21.15.78:0/1911444464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41a2030 con 0x7fc8f4103c20 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f9285640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8f41029e0 0x7fc8f406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8f8a84640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4103c20 0x7fc8f419f2a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400e9f0 tx=0x7fc8e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8ea7fc640 1 -- 172.21.15.78:0/1911444464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400cd80 con 0x7fc8f4103c20 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8ea7fc640 1 -- 172.21.15.78:0/1911444464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e40108e0 con 0x7fc8f4103c20 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41a2320 con 0x7fc8f4103c20 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8ea7fc640 1 -- 172.21.15.78:0/1911444464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4018980 con 0x7fc8f4103c20 2024-04-25T13:59:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.977+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a2870 con 0x7fc8f4103c20 2024-04-25T13:59:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.978+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f4107730 con 0x7fc8f4103c20 2024-04-25T13:59:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.980+0000 7fc8ea7fc640 1 -- 172.21.15.78:0/1911444464 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc8e4010430 con 0x7fc8f4103c20 2024-04-25T13:59:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.981+0000 7fc8ea7fc640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8e00afaf0 0x7fc8e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.981+0000 7fc8f9285640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8e00afaf0 0x7fc8e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.981+0000 7fc8ea7fc640 1 -- 172.21.15.78:0/1911444464 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc8e40df5a0 con 0x7fc8f4103c20 2024-04-25T13:59:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.981+0000 7fc8f9285640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8e00afaf0 0x7fc8e00b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8f41035e0 tx=0x7fc8f0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:02.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:02.985+0000 7fc8ea7fc640 1 -- 172.21.15.78:0/1911444464 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8e40a8110 con 0x7fc8f4103c20 2024-04-25T13:59:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.164+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8f402dce0 con 0x7fc8f4103c20 2024-04-25T13:59:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.164+0000 7fc8ea7fc640 1 -- 172.21.15.78:0/1911444464 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fc8e40a7a10 con 0x7fc8f4103c20 2024-04-25T13:59:03.163 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T13:59:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8e00afaf0 msgr2=0x7fc8e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8e00afaf0 0x7fc8e00b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8f41035e0 tx=0x7fc8f0005e70 comp rx=0 tx=0).stop 2024-04-25T13:59:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4103c20 msgr2=0x7fc8f419f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4103c20 0x7fc8f419f2a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400e9f0 tx=0x7fc8e400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 shutdown_connections 2024-04-25T13:59:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8f4108920 0x7fc8f41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8e00afaf0 0x7fc8e00b2000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8f4103c20 0x7fc8f419f2a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 --2- 172.21.15.78:0/1911444464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8f41029e0 0x7fc8f406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 >> 172.21.15.78:0/1911444464 conn(0x7fc8f40fdbd0 msgr2=0x7fc8f4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.166+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 shutdown_connections 2024-04-25T13:59:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.167+0000 7fc8fb4e9640 1 -- 172.21.15.78:0/1911444464 wait complete. 2024-04-25T13:59:03.177 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 -- 172.21.15.78:0/3538827286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc104d30 msgr2=0x7f9efc105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 --2- 172.21.15.78:0/3538827286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc104d30 0x7f9efc105140 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009900 tx=0x7f9eec02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 -- 172.21.15.78:0/3538827286 shutdown_connections 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 --2- 172.21.15.78:0/3538827286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc109a30 0x7f9efc10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 --2- 172.21.15.78:0/3538827286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc104d30 0x7f9efc105140 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 --2- 172.21.15.78:0/3538827286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9efc103af0 0x7f9efc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 -- 172.21.15.78:0/3538827286 >> 172.21.15.78:0/3538827286 conn(0x7f9efc0fece0 msgr2=0x7f9efc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 -- 172.21.15.78:0/3538827286 shutdown_connections 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.289+0000 7f9f04706640 1 -- 172.21.15.78:0/3538827286 wait complete. 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 Processor -- start 2024-04-25T13:59:03.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 -- start start 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc103af0 0x7f9efc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc104d30 0x7f9efc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9efc109a30 0x7f9efc1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10f610 con 0x7f9efc103af0 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10f490 con 0x7f9efc104d30 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f04706640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10f790 con 0x7f9efc109a30 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f024a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc103af0 0x7f9efc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f024a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc103af0 0x7f9efc19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52292/0 (socket says 172.21.15.78:52292) 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f024a2640 1 -- 172.21.15.78:0/3531367307 learned_addr learned my addr 172.21.15.78:0/3531367307 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.290+0000 7f9f01ca1640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc104d30 0x7f9efc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f02ca3640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9efc109a30 0x7f9efc1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f024a2640 1 -- 172.21.15.78:0/3531367307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9efc109a30 msgr2=0x7f9efc1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f024a2640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9efc109a30 0x7f9efc1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f024a2640 1 -- 172.21.15.78:0/3531367307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc104d30 msgr2=0x7f9efc19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f024a2640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc104d30 0x7f9efc19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f024a2640 1 -- 172.21.15.78:0/3531367307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc1a1d70 con 0x7f9efc103af0 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f02ca3640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9efc109a30 0x7f9efc1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f024a2640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc103af0 0x7f9efc19a230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800c970 tx=0x7f9ef800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9ef37fe640 1 -- 172.21.15.78:0/3531367307 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8004220 con 0x7f9efc103af0 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9ef37fe640 1 -- 172.21.15.78:0/3531367307 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef80043c0 con 0x7f9efc103af0 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc1a2060 con 0x7f9efc103af0 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9ef37fe640 1 -- 172.21.15.78:0/3531367307 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8004a00 con 0x7f9efc103af0 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.291+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc1a25b0 con 0x7f9efc103af0 2024-04-25T13:59:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.292+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9efc108840 con 0x7f9efc103af0 2024-04-25T13:59:03.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.293+0000 7f9ef37fe640 1 -- 172.21.15.78:0/3531367307 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9ef80079b0 con 0x7f9efc103af0 2024-04-25T13:59:03.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.294+0000 7f9ef37fe640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ee80afb70 0x7f9ee80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.294+0000 7f9f01ca1640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ee80afb70 0x7f9ee80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.294+0000 7f9ef37fe640 1 -- 172.21.15.78:0/3531367307 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9ef80ded70 con 0x7f9efc103af0 2024-04-25T13:59:03.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.294+0000 7f9f01ca1640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ee80afb70 0x7f9ee80b2080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009230 tx=0x7f9eec031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:03.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.297+0000 7f9ef37fe640 1 -- 172.21.15.78:0/3531367307 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ef8014030 con 0x7f9efc103af0 2024-04-25T13:59:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:03 smithi078 ceph-mon[29163]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:59:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:03 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2975345736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:03 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1084565123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:03 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1911444464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:03.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.475+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9efc0637a0 con 0x7f9efc103af0 2024-04-25T13:59:03.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.475+0000 7f9ef37fe640 1 -- 172.21.15.78:0/3531367307 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f9ef80a79c0 con 0x7f9efc103af0 2024-04-25T13:59:03.474 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ee80afb70 msgr2=0x7f9ee80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ee80afb70 0x7f9ee80b2080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009230 tx=0x7f9eec031400 comp rx=0 tx=0).stop 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc103af0 msgr2=0x7f9efc19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc103af0 0x7f9efc19a230 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800c970 tx=0x7f9ef800ce30 comp rx=0 tx=0).stop 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 shutdown_connections 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9efc109a30 0x7f9efc1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc104d30 0x7f9efc19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ee80afb70 0x7f9ee80b2080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 --2- 172.21.15.78:0/3531367307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc103af0 0x7f9efc19a230 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 >> 172.21.15.78:0/3531367307 conn(0x7f9efc0fece0 msgr2=0x7f9efc105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:03.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 shutdown_connections 2024-04-25T13:59:03.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.477+0000 7f9f04706640 1 -- 172.21.15.78:0/3531367307 wait complete. 2024-04-25T13:59:03.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:59:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:03 smithi112 ceph-mon[32617]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:59:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:03 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2975345736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:03 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1084565123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:03.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:03 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1911444464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:03.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- 172.21.15.78:0/202378684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f68108910 msgr2=0x7f5f6810ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 --2- 172.21.15.78:0/202378684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f68108910 0x7f5f6810ad30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009900 tx=0x7f5f6402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:03.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- 172.21.15.78:0/202378684 shutdown_connections 2024-04-25T13:59:03.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 --2- 172.21.15.78:0/202378684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f68108910 0x7f5f6810ad30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 --2- 172.21.15.78:0/202378684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f68103c10 0x7f5f68104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 --2- 172.21.15.78:0/202378684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f681029d0 0x7f5f681036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- 172.21.15.78:0/202378684 >> 172.21.15.78:0/202378684 conn(0x7f5f680fdba0 msgr2=0x7f5f68100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:03.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- 172.21.15.78:0/202378684 shutdown_connections 2024-04-25T13:59:03.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- 172.21.15.78:0/202378684 wait complete. 2024-04-25T13:59:03.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 Processor -- start 2024-04-25T13:59:03.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- start start 2024-04-25T13:59:03.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f681029d0 0x7f5f6819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f68103c10 0x7f5f6819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f68108910 0x7f5f681a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6810f600 con 0x7f5f681029d0 2024-04-25T13:59:03.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6810f480 con 0x7f5f68108910 2024-04-25T13:59:03.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.596+0000 7f5f70184640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6810f780 con 0x7f5f68103c10 2024-04-25T13:59:03.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f681029d0 0x7f5f6819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f681029d0 0x7f5f6819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52304/0 (socket says 172.21.15.78:52304) 2024-04-25T13:59:03.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 -- 172.21.15.78:0/687744313 learned_addr learned my addr 172.21.15.78:0/687744313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:03.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6d71f640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f68103c10 0x7f5f6819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 -- 172.21.15.78:0/687744313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f68103c10 msgr2=0x7f5f6819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f68103c10 0x7f5f6819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 -- 172.21.15.78:0/687744313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f68108910 msgr2=0x7f5f681a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f68108910 0x7f5f681a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 -- 172.21.15.78:0/687744313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681a1cd0 con 0x7f5f681029d0 2024-04-25T13:59:03.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6d71f640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f68103c10 0x7f5f6819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:03.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f6df20640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f681029d0 0x7f5f6819a190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000ea50 tx=0x7f5f6000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:03.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f5effd640 1 -- 172.21.15.78:0/687744313 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cdc0 con 0x7f5f681029d0 2024-04-25T13:59:03.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f681a1fc0 con 0x7f5f681029d0 2024-04-25T13:59:03.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f5effd640 1 -- 172.21.15.78:0/687744313 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f60011920 con 0x7f5f681029d0 2024-04-25T13:59:03.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f5effd640 1 -- 172.21.15.78:0/687744313 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6001aae0 con 0x7f5f681029d0 2024-04-25T13:59:03.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.597+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a2420 con 0x7f5f681029d0 2024-04-25T13:59:03.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.598+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f68069340 con 0x7f5f681029d0 2024-04-25T13:59:03.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.600+0000 7f5f5effd640 1 -- 172.21.15.78:0/687744313 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5f6001ac80 con 0x7f5f681029d0 2024-04-25T13:59:03.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.600+0000 7f5f5effd640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f540afaf0 0x7f5f540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.600+0000 7f5f6d71f640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f540afaf0 0x7f5f540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.600+0000 7f5f5effd640 1 -- 172.21.15.78:0/687744313 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5f600df1d0 con 0x7f5f681029d0 2024-04-25T13:59:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.601+0000 7f5f6d71f640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f540afaf0 0x7f5f540b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f6819b230 tx=0x7f5f58005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:03.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.605+0000 7f5f5effd640 1 -- 172.21.15.78:0/687744313 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f68069340 con 0x7f5f681029d0 2024-04-25T13:59:03.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:03 smithi129 ceph-mon[32569]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:59:03.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:03 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2975345736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:03.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:03 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1084565123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:03.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:03 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1911444464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:03.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.817+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f68063870 con 0x7f5f681029d0 2024-04-25T13:59:03.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.817+0000 7f5f5effd640 1 -- 172.21.15.78:0/687744313 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f5f60016030 con 0x7f5f681029d0 2024-04-25T13:59:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:03.816 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:57:16.081259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24512,24484]},"id":1}]} 2024-04-25T13:59:03.817 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 21 2024-04-25T13:59:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f540afaf0 msgr2=0x7f5f540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f540afaf0 0x7f5f540b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f6819b230 tx=0x7f5f58005e70 comp rx=0 tx=0).stop 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f681029d0 msgr2=0x7f5f6819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f681029d0 0x7f5f6819a190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000ea50 tx=0x7f5f6000ef10 comp rx=0 tx=0).stop 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 shutdown_connections 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f68108910 0x7f5f681a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f68103c10 0x7f5f6819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f540afaf0 0x7f5f540b2000 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 --2- 172.21.15.78:0/687744313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f681029d0 0x7f5f6819a190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 >> 172.21.15.78:0/687744313 conn(0x7f5f680fdba0 msgr2=0x7f5f681048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 shutdown_connections 2024-04-25T13:59:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.819+0000 7f5f70184640 1 -- 172.21.15.78:0/687744313 wait complete. 2024-04-25T13:59:03.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:59:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 -- 172.21.15.78:0/3257568666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8103c00 msgr2=0x7f7ef8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 --2- 172.21.15.78:0/3257568666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8103c00 0x7f7ef8104010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009900 tx=0x7f7ee802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 -- 172.21.15.78:0/3257568666 shutdown_connections 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 --2- 172.21.15.78:0/3257568666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8108900 0x7f7ef810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 --2- 172.21.15.78:0/3257568666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8103c00 0x7f7ef8104010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 --2- 172.21.15.78:0/3257568666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef81029c0 0x7f7ef81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 -- 172.21.15.78:0/3257568666 >> 172.21.15.78:0/3257568666 conn(0x7f7ef80fdbd0 msgr2=0x7f7ef8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 -- 172.21.15.78:0/3257568666 shutdown_connections 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.938+0000 7f7f00029640 1 -- 172.21.15.78:0/3257568666 wait complete. 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 Processor -- start 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 -- start start 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef81029c0 0x7f7ef806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8103c00 0x7f7ef819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8108900 0x7f7ef81a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810e4e0 con 0x7f7ef8108900 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810e360 con 0x7f7ef8103c00 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7f00029640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810e660 con 0x7f7ef81029c0 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8108900 0x7f7ef81a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8108900 0x7f7ef81a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52328/0 (socket says 172.21.15.78:52328) 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 -- 172.21.15.78:0/2530935359 learned_addr learned my addr 172.21.15.78:0/2530935359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efddc5640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef81029c0 0x7f7ef806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efd5c4640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8103c00 0x7f7ef819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 -- 172.21.15.78:0/2530935359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef81029c0 msgr2=0x7f7ef806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef81029c0 0x7f7ef806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 -- 172.21.15.78:0/2530935359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8103c00 msgr2=0x7f7ef819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8103c00 0x7f7ef819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 -- 172.21.15.78:0/2530935359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef81a1d40 con 0x7f7ef8108900 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efd5c4640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8103c00 0x7f7ef819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7efe5c6640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8108900 0x7f7ef81a1520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400d750 tx=0x7f7ef400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.939+0000 7f7eeeffd640 1 -- 172.21.15.78:0/2530935359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4017070 con 0x7f7ef8108900 2024-04-25T13:59:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.940+0000 7f7eeeffd640 1 -- 172.21.15.78:0/2530935359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef40054d0 con 0x7f7ef8108900 2024-04-25T13:59:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.940+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef81a1fd0 con 0x7f7ef8108900 2024-04-25T13:59:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.940+0000 7f7eeeffd640 1 -- 172.21.15.78:0/2530935359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4004c60 con 0x7f7ef8108900 2024-04-25T13:59:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.940+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a2440 con 0x7f7ef8108900 2024-04-25T13:59:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.941+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef8107710 con 0x7f7ef8108900 2024-04-25T13:59:03.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.942+0000 7f7eeeffd640 1 -- 172.21.15.78:0/2530935359 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7ef4009bd0 con 0x7f7ef8108900 2024-04-25T13:59:03.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.943+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee40afaf0 0x7f7ee40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:03.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.943+0000 7f7efddc5640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee40afaf0 0x7f7ee40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:03.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.943+0000 7f7eeeffd640 1 -- 172.21.15.78:0/2530935359 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7ef40dea70 con 0x7f7ef8108900 2024-04-25T13:59:03.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.943+0000 7f7efddc5640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee40afaf0 0x7f7ee40b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ef81035c0 tx=0x7f7ef0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:03.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:03.947+0000 7f7eeeffd640 1 -- 172.21.15.78:0/2530935359 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ef40a7640 con 0x7f7ef8108900 2024-04-25T13:59:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.125+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ef802dd20 con 0x7f7ef8108900 2024-04-25T13:59:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.127+0000 7f7eeeffd640 1 -- 172.21.15.78:0/2530935359 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f7ef40a6f40 con 0x7f7ef8108900 2024-04-25T13:59:04.126 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:04.126 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee40afaf0 msgr2=0x7f7ee40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee40afaf0 0x7f7ee40b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ef81035c0 tx=0x7f7ef0009500 comp rx=0 tx=0).stop 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8108900 msgr2=0x7f7ef81a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8108900 0x7f7ef81a1520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400d750 tx=0x7f7ef400dc10 comp rx=0 tx=0).stop 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 shutdown_connections 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee40afaf0 0x7f7ee40b2000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8108900 0x7f7ef81a1520 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8103c00 0x7f7ef819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 --2- 172.21.15.78:0/2530935359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef81029c0 0x7f7ef806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 >> 172.21.15.78:0/2530935359 conn(0x7f7ef80fdbd0 msgr2=0x7f7ef81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 shutdown_connections 2024-04-25T13:59:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.129+0000 7f7f00029640 1 -- 172.21.15.78:0/2530935359 wait complete. 2024-04-25T13:59:04.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:59:04.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/1726058698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381033d0 msgr2=0x7f8c381037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/1726058698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381033d0 0x7f8c381037e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009900 tx=0x7f8c2c02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/1726058698 shutdown_connections 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/1726058698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c381080d0 0x7f8c3810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/1726058698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381033d0 0x7f8c381037e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/1726058698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c38102190 0x7f8c38102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/1726058698 >> 172.21.15.78:0/1726058698 conn(0x7f8c380fd380 msgr2=0x7f8c380ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/1726058698 shutdown_connections 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/1726058698 wait complete. 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.254+0000 7f8c3dcf2640 1 Processor -- start 2024-04-25T13:59:04.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c3dcf2640 1 -- start start 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c3dcf2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c38102190 0x7f8c3810b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c3dcf2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c381033d0 0x7f8c3819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c3dcf2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381080d0 0x7f8c381a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c3dcf2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3810e520 con 0x7f8c381080d0 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c3dcf2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3810e3a0 con 0x7f8c38102190 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c3dcf2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3810e6a0 con 0x7f8c381033d0 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381080d0 0x7f8c381a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381080d0 0x7f8c381a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52358/0 (socket says 172.21.15.78:52358) 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 -- 172.21.15.78:0/779565749 learned_addr learned my addr 172.21.15.78:0/779565749 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c36ffd640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c381033d0 0x7f8c3819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c377fe640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c38102190 0x7f8c3810b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 -- 172.21.15.78:0/779565749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c381033d0 msgr2=0x7f8c3819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c381033d0 0x7f8c3819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 -- 172.21.15.78:0/779565749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c38102190 msgr2=0x7f8c3810b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c38102190 0x7f8c3810b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 -- 172.21.15.78:0/779565749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c381a1d40 con 0x7f8c381080d0 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c37fff640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381080d0 0x7f8c381a1520 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000cad0 tx=0x7f8c3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.255+0000 7f8c34ff9640 1 -- 172.21.15.78:0/779565749 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000ae20 con 0x7f8c381080d0 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.256+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c381a1fd0 con 0x7f8c381080d0 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.256+0000 7f8c34ff9640 1 -- 172.21.15.78:0/779565749 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c30005020 con 0x7f8c381080d0 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.256+0000 7f8c34ff9640 1 -- 172.21.15.78:0/779565749 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000f6b0 con 0x7f8c381080d0 2024-04-25T13:59:04.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.256+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381a2440 con 0x7f8c381080d0 2024-04-25T13:59:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.256+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c38106ee0 con 0x7f8c381080d0 2024-04-25T13:59:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.258+0000 7f8c34ff9640 1 -- 172.21.15.78:0/779565749 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8c300051d0 con 0x7f8c381080d0 2024-04-25T13:59:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.258+0000 7f8c34ff9640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c200afaf0 0x7f8c200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.258+0000 7f8c377fe640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c200afaf0 0x7f8c200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.259+0000 7f8c34ff9640 1 -- 172.21.15.78:0/779565749 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8c300df0f0 con 0x7f8c381080d0 2024-04-25T13:59:04.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.259+0000 7f8c377fe640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c200afaf0 0x7f8c200b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c28005fd0 tx=0x7f8c28005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:04.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.262+0000 7f8c34ff9640 1 -- 172.21.15.78:0/779565749 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c30014030 con 0x7f8c381080d0 2024-04-25T13:59:04.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.443+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c380637a0 con 0x7f8c381080d0 2024-04-25T13:59:04.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.443+0000 7f8c34ff9640 1 -- 172.21.15.78:0/779565749 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f8c300a7cc0 con 0x7f8c381080d0 2024-04-25T13:59:04.442 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:59:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c200afaf0 msgr2=0x7f8c200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c200afaf0 0x7f8c200b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c28005fd0 tx=0x7f8c28005850 comp rx=0 tx=0).stop 2024-04-25T13:59:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381080d0 msgr2=0x7f8c381a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381080d0 0x7f8c381a1520 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000cad0 tx=0x7f8c3000cf90 comp rx=0 tx=0).stop 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 shutdown_connections 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c200afaf0 0x7f8c200b2000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c381080d0 0x7f8c381a1520 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c381033d0 0x7f8c3819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 --2- 172.21.15.78:0/779565749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c38102190 0x7f8c3810b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 >> 172.21.15.78:0/779565749 conn(0x7f8c380fd380 msgr2=0x7f8c381040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 shutdown_connections 2024-04-25T13:59:04.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.445+0000 7f8c3dcf2640 1 -- 172.21.15.78:0/779565749 wait complete. 2024-04-25T13:59:04.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:04 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3531367307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:04.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:04 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/687744313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:04.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:04 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2530935359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:04.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:04 smithi078 ceph-mon[29163]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:04.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:59:04.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:04 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3531367307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:04.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:04 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/687744313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:04.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:04 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2530935359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:04.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:04 smithi112 ceph-mon[32617]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:04.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.565+0000 7f0294a4c640 1 -- 172.21.15.78:0/495343218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0290103c20 msgr2=0x7f0290104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 --2- 172.21.15.78:0/495343218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0290103c20 0x7f0290104030 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f028402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 -- 172.21.15.78:0/495343218 shutdown_connections 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 --2- 172.21.15.78:0/495343218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0290108920 0x7f029010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 --2- 172.21.15.78:0/495343218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0290103c20 0x7f0290104030 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 --2- 172.21.15.78:0/495343218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f02901029e0 0x7f02901036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 -- 172.21.15.78:0/495343218 >> 172.21.15.78:0/495343218 conn(0x7f02900fdbd0 msgr2=0x7f0290100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 -- 172.21.15.78:0/495343218 shutdown_connections 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 -- 172.21.15.78:0/495343218 wait complete. 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 Processor -- start 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.566+0000 7f0294a4c640 1 -- start start 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02901029e0 0x7f0290075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0290103c20 0x7f029019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0290108920 0x7f02901a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f029010e550 con 0x7f02901029e0 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f029010e3d0 con 0x7f0290108920 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f029010e6d0 con 0x7f0290103c20 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02901029e0 0x7f0290075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02901029e0 0x7f0290075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52366/0 (socket says 172.21.15.78:52366) 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 -- 172.21.15.78:0/2744401125 learned_addr learned my addr 172.21.15.78:0/2744401125 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 -- 172.21.15.78:0/2744401125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0290103c20 msgr2=0x7f029019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028dd9b640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0290103c20 0x7f029019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0290103c20 0x7f029019f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 -- 172.21.15.78:0/2744401125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0290108920 msgr2=0x7f02901a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0290108920 0x7f02901a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 -- 172.21.15.78:0/2744401125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02901a1dd0 con 0x7f02901029e0 2024-04-25T13:59:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f028e59c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02901029e0 0x7f0290075860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f028000c970 tx=0x7f028000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:04.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f027f7fe640 1 -- 172.21.15.78:0/2744401125 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280004220 con 0x7f02901029e0 2024-04-25T13:59:04.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f027f7fe640 1 -- 172.21.15.78:0/2744401125 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02800043c0 con 0x7f02901029e0 2024-04-25T13:59:04.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901a20c0 con 0x7f02901029e0 2024-04-25T13:59:04.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f027f7fe640 1 -- 172.21.15.78:0/2744401125 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280004a00 con 0x7f02901029e0 2024-04-25T13:59:04.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.567+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a2610 con 0x7f02901029e0 2024-04-25T13:59:04.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.568+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290107730 con 0x7f02901029e0 2024-04-25T13:59:04.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.570+0000 7f027f7fe640 1 -- 172.21.15.78:0/2744401125 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f02800079b0 con 0x7f02901029e0 2024-04-25T13:59:04.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.570+0000 7f027f7fe640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02780afb70 0x7f02780b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.570+0000 7f028dd9b640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02780afb70 0x7f02780b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.571+0000 7f027f7fe640 1 -- 172.21.15.78:0/2744401125 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f02800dee20 con 0x7f02901029e0 2024-04-25T13:59:04.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.571+0000 7f028dd9b640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02780afb70 0x7f02780b2080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f028402a930 tx=0x7f0284005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:04.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.574+0000 7f027f7fe640 1 -- 172.21.15.78:0/2744401125 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0280014030 con 0x7f02901029e0 2024-04-25T13:59:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:04 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3531367307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:04 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/687744313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:04.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:04 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2530935359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:04.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:04 smithi129 ceph-mon[32569]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:04.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.751+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f029002de00 con 0x7f02901029e0 2024-04-25T13:59:04.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.752+0000 7f027f7fe640 1 -- 172.21.15.78:0/2744401125 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f02800a7a70 con 0x7f02901029e0 2024-04-25T13:59:04.752 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T13:59:04.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02780afb70 msgr2=0x7f02780b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02780afb70 0x7f02780b2080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f028402a930 tx=0x7f0284005fd0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02901029e0 msgr2=0x7f0290075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02901029e0 0x7f0290075860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f028000c970 tx=0x7f028000ce30 comp rx=0 tx=0).stop 2024-04-25T13:59:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 shutdown_connections 2024-04-25T13:59:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0290108920 0x7f02901a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0290103c20 0x7f029019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f02780afb70 0x7f02780b2080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 --2- 172.21.15.78:0/2744401125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02901029e0 0x7f0290075860 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 >> 172.21.15.78:0/2744401125 conn(0x7f02900fdbd0 msgr2=0x7f0290104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 shutdown_connections 2024-04-25T13:59:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.754+0000 7f0294a4c640 1 -- 172.21.15.78:0/2744401125 wait complete. 2024-04-25T13:59:04.765 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:59:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 -- 172.21.15.78:0/1828265772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 msgr2=0x7f6b001036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 --2- 172.21.15.78:0/1828265772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b001036f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009900 tx=0x7f6afc02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 -- 172.21.15.78:0/1828265772 shutdown_connections 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 --2- 172.21.15.78:0/1828265772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b00108930 0x7f6b0010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 --2- 172.21.15.78:0/1828265772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b00103c30 0x7f6b00104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 --2- 172.21.15.78:0/1828265772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b001036f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 -- 172.21.15.78:0/1828265772 >> 172.21.15.78:0/1828265772 conn(0x7f6b000fdbc0 msgr2=0x7f6b00100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 -- 172.21.15.78:0/1828265772 shutdown_connections 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 -- 172.21.15.78:0/1828265772 wait complete. 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 Processor -- start 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.877+0000 7f6b087b2640 1 -- start start 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b0019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b00103c30 0x7f6b0019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b00108930 0x7f6b001a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0010f5e0 con 0x7f6b001029f0 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0010f460 con 0x7f6b00108930 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0010f760 con 0x7f6b00103c30 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b0019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b0019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52378/0 (socket says 172.21.15.78:52378) 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 -- 172.21.15.78:0/807735870 learned_addr learned my addr 172.21.15.78:0/807735870 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 -- 172.21.15.78:0/807735870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b00103c30 msgr2=0x7f6b0019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b00103c30 0x7f6b0019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 -- 172.21.15.78:0/807735870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b00108930 msgr2=0x7f6b001a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b00108930 0x7f6b001a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 -- 172.21.15.78:0/807735870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b001a1cd0 con 0x7f6b001029f0 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b0654e640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b0019a240 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6afc032fd0 tx=0x7f6afc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6af77fe640 1 -- 172.21.15.78:0/807735870 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc03f070 con 0x7f6b001029f0 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6af77fe640 1 -- 172.21.15.78:0/807735870 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6afc0044d0 con 0x7f6b001029f0 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b001a1f60 con 0x7f6b001029f0 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6af77fe640 1 -- 172.21.15.78:0/807735870 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc035070 con 0x7f6b001029f0 2024-04-25T13:59:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.878+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b001a23c0 con 0x7f6b001029f0 2024-04-25T13:59:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.879+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b00107740 con 0x7f6b001029f0 2024-04-25T13:59:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.881+0000 7f6af77fe640 1 -- 172.21.15.78:0/807735870 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6afc03b5f0 con 0x7f6b001029f0 2024-04-25T13:59:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.881+0000 7f6af77fe640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6aec0afaf0 0x7f6aec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.881+0000 7f6b05d4d640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6aec0afaf0 0x7f6aec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.881+0000 7f6af77fe640 1 -- 172.21.15.78:0/807735870 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6afc0ff2a0 con 0x7f6b001029f0 2024-04-25T13:59:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.882+0000 7f6b05d4d640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6aec0afaf0 0x7f6aec0b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b0019b2e0 tx=0x7f6af0005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:04.886+0000 7f6af77fe640 1 -- 172.21.15.78:0/807735870 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6afc0c7e70 con 0x7f6b001029f0 2024-04-25T13:59:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.062+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6b000638e0 con 0x7f6b001029f0 2024-04-25T13:59:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.063+0000 7f6af77fe640 1 -- 172.21.15.78:0/807735870 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f6afc0c7770 con 0x7f6b001029f0 2024-04-25T13:59:05.063 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6aec0afaf0 msgr2=0x7f6aec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6aec0afaf0 0x7f6aec0b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b0019b2e0 tx=0x7f6af0005c70 comp rx=0 tx=0).stop 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 msgr2=0x7f6b0019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b0019a240 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6afc032fd0 tx=0x7f6afc0042e0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 shutdown_connections 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b00108930 0x7f6b001a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b00103c30 0x7f6b0019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6aec0afaf0 0x7f6aec0b2000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 --2- 172.21.15.78:0/807735870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b001029f0 0x7f6b0019a240 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 >> 172.21.15.78:0/807735870 conn(0x7f6b000fdbc0 msgr2=0x7f6b00104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 shutdown_connections 2024-04-25T13:59:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.065+0000 7f6b087b2640 1 -- 172.21.15.78:0/807735870 wait complete. 2024-04-25T13:59:05.076 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:59:05.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 -- 172.21.15.78:0/3742159656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf81029e0 msgr2=0x7f6bf81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/3742159656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf81029e0 0x7f6bf81036e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009900 tx=0x7f6bf402a3f0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 -- 172.21.15.78:0/3742159656 shutdown_connections 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/3742159656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8108920 0x7f6bf810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/3742159656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf8103c20 0x7f6bf8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/3742159656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf81029e0 0x7f6bf81036e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 -- 172.21.15.78:0/3742159656 >> 172.21.15.78:0/3742159656 conn(0x7f6bf80fdbd0 msgr2=0x7f6bf8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 -- 172.21.15.78:0/3742159656 shutdown_connections 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.188+0000 7f6bfff9d640 1 -- 172.21.15.78:0/3742159656 wait complete. 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 Processor -- start 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 -- start start 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf81029e0 0x7f6bf810d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8103c20 0x7f6bf810b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf8108920 0x7f6bf810be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810f650 con 0x7f6bf8108920 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810f4d0 con 0x7f6bf8103c20 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfff9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810f7d0 con 0x7f6bf81029e0 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf8108920 0x7f6bf810be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf8108920 0x7f6bf810be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52388/0 (socket says 172.21.15.78:52388) 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 -- 172.21.15.78:0/2813888077 learned_addr learned my addr 172.21.15.78:0/2813888077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfdd39640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf81029e0 0x7f6bf810d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfd538640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8103c20 0x7f6bf810b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 -- 172.21.15.78:0/2813888077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf81029e0 msgr2=0x7f6bf810d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf81029e0 0x7f6bf810d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 -- 172.21.15.78:0/2813888077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8103c20 msgr2=0x7f6bf810b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8103c20 0x7f6bf810b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 -- 172.21.15.78:0/2813888077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf810c530 con 0x7f6bf8108920 2024-04-25T13:59:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfdd39640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf81029e0 0x7f6bf810d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfd538640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8103c20 0x7f6bf810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.189+0000 7f6bfe53a640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf8108920 0x7f6bf810be90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000cce0 tx=0x7f6bf000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.190+0000 7f6beaffd640 1 -- 172.21.15.78:0/2813888077 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf000f450 con 0x7f6bf8108920 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.190+0000 7f6beaffd640 1 -- 172.21.15.78:0/2813888077 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bf000faa0 con 0x7f6bf8108920 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.190+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf81a4400 con 0x7f6bf8108920 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.190+0000 7f6beaffd640 1 -- 172.21.15.78:0/2813888077 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0018b70 con 0x7f6bf8108920 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.190+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf81a4890 con 0x7f6bf8108920 2024-04-25T13:59:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.191+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf8107730 con 0x7f6bf8108920 2024-04-25T13:59:05.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.192+0000 7f6beaffd640 1 -- 172.21.15.78:0/2813888077 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6bf0018d10 con 0x7f6bf8108920 2024-04-25T13:59:05.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.192+0000 7f6beaffd640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be40afaf0 0x7f6be40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.193+0000 7f6bfdd39640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be40afaf0 0x7f6be40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.193+0000 7f6beaffd640 1 -- 172.21.15.78:0/2813888077 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6bf00df200 con 0x7f6bf8108920 2024-04-25T13:59:05.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.193+0000 7f6bfdd39640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be40afaf0 0x7f6be40b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009230 tx=0x7f6bf4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.196+0000 7f6beaffd640 1 -- 172.21.15.78:0/2813888077 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bf0014030 con 0x7f6bf8108920 2024-04-25T13:59:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.415+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bf802db50 con 0x7f6bf8108920 2024-04-25T13:59:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.416+0000 7f6beaffd640 1 -- 172.21.15.78:0/2813888077 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f6bf00a7e60 con 0x7f6bf8108920 2024-04-25T13:59:05.416 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:05.416 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:57:16.081259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24512,24484]},"id":1}]} 2024-04-25T13:59:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 21 2024-04-25T13:59:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be40afaf0 msgr2=0x7f6be40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be40afaf0 0x7f6be40b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009230 tx=0x7f6bf4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf8108920 msgr2=0x7f6bf810be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf8108920 0x7f6bf810be90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000cce0 tx=0x7f6bf000a620 comp rx=0 tx=0).stop 2024-04-25T13:59:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 shutdown_connections 2024-04-25T13:59:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be40afaf0 0x7f6be40b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf8108920 0x7f6bf810be90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8103c20 0x7f6bf810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 --2- 172.21.15.78:0/2813888077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf81029e0 0x7f6bf810d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 >> 172.21.15.78:0/2813888077 conn(0x7f6bf80fdbd0 msgr2=0x7f6bf8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 shutdown_connections 2024-04-25T13:59:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.418+0000 7f6bfff9d640 1 -- 172.21.15.78:0/2813888077 wait complete. 2024-04-25T13:59:05.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:05 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/779565749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:05 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2744401125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:05 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/807735870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.430 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:59:05.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:05 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/779565749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:05 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2744401125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:05 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/807735870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 -- 172.21.15.78:0/3527189263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d601050a0 msgr2=0x7f0d601054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 --2- 172.21.15.78:0/3527189263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d601050a0 0x7f0d601054b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009900 tx=0x7f0d5c02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:05.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 -- 172.21.15.78:0/3527189263 shutdown_connections 2024-04-25T13:59:05.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 --2- 172.21.15.78:0/3527189263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d60102250 0x7f0d6010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 --2- 172.21.15.78:0/3527189263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d60101750 0x7f0d60101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 --2- 172.21.15.78:0/3527189263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d601050a0 0x7f0d601054b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 -- 172.21.15.78:0/3527189263 >> 172.21.15.78:0/3527189263 conn(0x7f0d600fd3b0 msgr2=0x7f0d600ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 -- 172.21.15.78:0/3527189263 shutdown_connections 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.538+0000 7f0d679b7640 1 -- 172.21.15.78:0/3527189263 wait complete. 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 Processor -- start 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 -- start start 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d60101750 0x7f0d6019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d60102250 0x7f0d6019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d601050a0 0x7f0d601a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6010f610 con 0x7f0d60101750 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6010f490 con 0x7f0d601050a0 2024-04-25T13:59:05.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d679b7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6010f790 con 0x7f0d60102250 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d60101750 0x7f0d6019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d60101750 0x7f0d6019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52394/0 (socket says 172.21.15.78:52394) 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 -- 172.21.15.78:0/2100979585 learned_addr learned my addr 172.21.15.78:0/2100979585 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65f54640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d601050a0 0x7f0d601a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 -- 172.21.15.78:0/2100979585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d60102250 msgr2=0x7f0d6019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d60102250 0x7f0d6019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 -- 172.21.15.78:0/2100979585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d601050a0 msgr2=0x7f0d601a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d601050a0 0x7f0d601a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.539+0000 7f0d65753640 1 -- 172.21.15.78:0/2100979585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d601a1cd0 con 0x7f0d60101750 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.540+0000 7f0d65753640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d60101750 0x7f0d6019a240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c032fd0 tx=0x7f0d5c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.540+0000 7f0d567fc640 1 -- 172.21.15.78:0/2100979585 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c03f070 con 0x7f0d60101750 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.540+0000 7f0d567fc640 1 -- 172.21.15.78:0/2100979585 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d5c0044d0 con 0x7f0d60101750 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.540+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d601a1f60 con 0x7f0d60101750 2024-04-25T13:59:05.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.540+0000 7f0d567fc640 1 -- 172.21.15.78:0/2100979585 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c035070 con 0x7f0d60101750 2024-04-25T13:59:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.540+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d601a2420 con 0x7f0d60101750 2024-04-25T13:59:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.541+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d60108a30 con 0x7f0d60101750 2024-04-25T13:59:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.542+0000 7f0d567fc640 1 -- 172.21.15.78:0/2100979585 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0d5c03b5f0 con 0x7f0d60101750 2024-04-25T13:59:05.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.543+0000 7f0d567fc640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0d4c0afb70 0x7f0d4c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.543+0000 7f0d64f52640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0d4c0afb70 0x7f0d4c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.543+0000 7f0d567fc640 1 -- 172.21.15.78:0/2100979585 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0d5c0ff8a0 con 0x7f0d60101750 2024-04-25T13:59:05.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.543+0000 7f0d64f52640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0d4c0afb70 0x7f0d4c0b2080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d6019b2e0 tx=0x7f0d50005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.546+0000 7f0d567fc640 1 -- 172.21.15.78:0/2100979585 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d5c0c8470 con 0x7f0d60101750 2024-04-25T13:59:05.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:05 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/779565749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:05 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2744401125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:05 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/807735870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.725+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d6002de00 con 0x7f0d60101750 2024-04-25T13:59:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.726+0000 7f0d567fc640 1 -- 172.21.15.78:0/2100979585 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f0d5c0c7d70 con 0x7f0d60101750 2024-04-25T13:59:05.726 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:05.727 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:59:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.728+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0d4c0afb70 msgr2=0x7f0d4c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.728+0000 7f0d679b7640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0d4c0afb70 0x7f0d4c0b2080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d6019b2e0 tx=0x7f0d50005c70 comp rx=0 tx=0).stop 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.728+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d60101750 msgr2=0x7f0d6019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.728+0000 7f0d679b7640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d60101750 0x7f0d6019a240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c032fd0 tx=0x7f0d5c0042e0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 shutdown_connections 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d601050a0 0x7f0d601a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d60102250 0x7f0d6019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0d4c0afb70 0x7f0d4c0b2080 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 --2- 172.21.15.78:0/2100979585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d60101750 0x7f0d6019a240 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 >> 172.21.15.78:0/2100979585 conn(0x7f0d600fd3b0 msgr2=0x7f0d60105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 shutdown_connections 2024-04-25T13:59:05.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.729+0000 7f0d679b7640 1 -- 172.21.15.78:0/2100979585 wait complete. 2024-04-25T13:59:05.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:57:11.484343+0000', 'modified': '2024-04-25T13:57:16.081259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24484, 'mds_2': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6835/2622963442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 2622963442}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 2622963442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.129:6835/2365161980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 2365161980}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 2365161980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6837/750801105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 750801105}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 750801105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6837/4152465978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6836', 'nonce': 4152465978}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 4152465978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.129:6839/1046146051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 1046146051}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 1046146051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6839/67760277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 67760277}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 67760277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24454, 24512, 24484]} 2024-04-25T13:59:05.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T13:59:05.741 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 -- 172.21.15.78:0/881442369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d81029e0 msgr2=0x7f65d81036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 --2- 172.21.15.78:0/881442369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d81029e0 0x7f65d81036e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009900 tx=0x7f65c802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 -- 172.21.15.78:0/881442369 shutdown_connections 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 --2- 172.21.15.78:0/881442369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d8108920 0x7f65d810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 --2- 172.21.15.78:0/881442369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65d8103c20 0x7f65d8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 --2- 172.21.15.78:0/881442369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d81029e0 0x7f65d81036e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 -- 172.21.15.78:0/881442369 >> 172.21.15.78:0/881442369 conn(0x7f65d80fdbd0 msgr2=0x7f65d8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 -- 172.21.15.78:0/881442369 shutdown_connections 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.849+0000 7f65dda42640 1 -- 172.21.15.78:0/881442369 wait complete. 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 Processor -- start 2024-04-25T13:59:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 -- start start 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d81029e0 0x7f65d806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d8103c20 0x7f65d819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65d8108920 0x7f65d81a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65d810e4e0 con 0x7f65d8103c20 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65d810e360 con 0x7f65d8108920 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65dda42640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65d810e660 con 0x7f65d81029e0 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d8103c20 0x7f65d819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d8103c20 0x7f65d819f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52408/0 (socket says 172.21.15.78:52408) 2024-04-25T13:59:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 -- 172.21.15.78:0/3583173418 learned_addr learned my addr 172.21.15.78:0/3583173418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d81029e0 0x7f65d806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65d77fe640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65d8108920 0x7f65d81a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 -- 172.21.15.78:0/3583173418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d81029e0 msgr2=0x7f65d806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d81029e0 0x7f65d806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 -- 172.21.15.78:0/3583173418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65d8108920 msgr2=0x7f65d81a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65d8108920 0x7f65d81a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65cffff640 1 -- 172.21.15.78:0/3583173418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d81a2060 con 0x7f65d8103c20 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.850+0000 7f65d77fe640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65d8108920 0x7f65d81a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.851+0000 7f65cffff640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d8103c20 0x7f65d819f2d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d950 tx=0x7f65c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.851+0000 7f65dca40640 1 -- 172.21.15.78:0/3583173418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0017070 con 0x7f65d8103c20 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.851+0000 7f65dca40640 1 -- 172.21.15.78:0/3583173418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c0009c30 con 0x7f65d8103c20 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.851+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d81a2350 con 0x7f65d8103c20 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.851+0000 7f65dca40640 1 -- 172.21.15.78:0/3583173418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c000f620 con 0x7f65d8103c20 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.851+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a28f0 con 0x7f65d8103c20 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.853+0000 7f65dca40640 1 -- 172.21.15.78:0/3583173418 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f65c0010460 con 0x7f65d8103c20 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.853+0000 7f65e49cb740 1 Processor -- start 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.853+0000 7f65e49cb740 1 -- start start 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.853+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd2069370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.853+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c9f3b0 0x557bd20698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.853+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65e49cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557bd1fce740 con 0x557bd1c9f3b0 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65e49cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bd1f6a3a0 con 0x557bd1ca7590 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65e49cb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557bd1fca700 con 0x557bd1cc8270 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c9f3b0 0x557bd20698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c9f3b0 0x557bd20698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52414/0 (socket says 172.21.15.78:52414) 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 -- 172.21.15.78:0/992560224 learned_addr learned my addr 172.21.15.78:0/992560224 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65cffff640 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d77fe640 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd2069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 -- 172.21.15.78:0/992560224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 msgr2=0x557bd2069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd2069370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 -- 172.21.15.78:0/992560224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 msgr2=0x557bd20707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65dca40640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65c40afb70 0x7f65c40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 -- 172.21.15.78:0/992560224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd1f86750 con 0x557bd1c9f3b0 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65cffff640 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d77fe640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65c40afb70 0x7f65c40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c9f3b0 0x557bd20698f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000c00 tx=0x7f65c8032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866b696bd200d0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65dca40640 1 -- 172.21.15.78:0/3583173418 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f65c00de930 con 0x7f65d8103c20 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65cdffb640 1 -- 172.21.15.78:0/992560224 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c803f070 con 0x557bd1c9f3b0 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65cdffb640 1 -- 172.21.15.78:0/992560224 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c8004040 con 0x557bd1c9f3b0 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.854+0000 7f65cdffb640 1 -- 172.21.15.78:0/992560224 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8035070 con 0x557bd1c9f3b0 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 -- 172.21.15.78:0/992560224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c9f3b0 msgr2=0x557bd20698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c9f3b0 0x557bd20698f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000c00 tx=0x7f65c8032ba0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65d77fe640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65c40afb70 0x7f65c40b2080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x557bd206a540 tx=0x7f65d0009420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 -- 172.21.15.78:0/992560224 shutdown_connections 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c9f3b0 0x557bd20698f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 --2- 172.21.15.78:0/992560224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd2069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 -- 172.21.15.78:0/992560224 >> 172.21.15.78:0/992560224 conn(0x557bd1cbc3c0 msgr2=0x557bd2064d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 -- 172.21.15.78:0/992560224 shutdown_connections 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.855+0000 7f65e49cb740 1 -- 172.21.15.78:0/992560224 wait complete. 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 -- start start 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 Processor -- start 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd20731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1ca7590 0x557bd2073730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd207a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557bd20711e0 con 0x557bd1ca7590 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bd1f55fe0 con 0x557bd1c9f3b0 2024-04-25T13:59:05.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557bd2071360 con 0x557bd1cc8270 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65e49cb740 10 client.0 _ll_register_callbacks cb 0x557bd1f6c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1ca7590 0x557bd2073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1ca7590 0x557bd2073730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52416/0 (socket says 172.21.15.78:52416) 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 -- 172.21.15.78:0/2213250686 learned_addr learned my addr 172.21.15.78:0/2213250686 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65d77fe640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd207a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd20731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 -- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 msgr2=0x557bd207a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd207a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 -- 172.21.15.78:0/2213250686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 msgr2=0x557bd20731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd20731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65cffff640 1 -- 172.21.15.78:0/2213250686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd1ce02d0 con 0x557bd1ca7590 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.856+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd20731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65cffff640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1ca7590 0x557bd2073730 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f65c00a4010 tx=0x7f65c00a6820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d099be96f9a817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65b6ffd640 1 -- 172.21.15.78:0/2213250686 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00aa020 con 0x557bd1ca7590 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65b6ffd640 1 -- 172.21.15.78:0/2213250686 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f65c00a6000 con 0x557bd1ca7590 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c8004040 con 0x557bd1ca7590 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65b6ffd640 1 -- 172.21.15.78:0/2213250686 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f65c0020070 con 0x557bd1ca7590 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 10 client.15237 fetch_fsmap learned FSMap version 21 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557bd1daa6b0 con 0x557bd1ca7590 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65b6ffd640 1 -- 172.21.15.78:0/2213250686 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c8004040 con 0x557bd1ca7590 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65b6ffd640 1 -- 172.21.15.78:0/2213250686 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f65c0010750 con 0x557bd1ca7590 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 10 client.15237 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 10 client.15237 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 20 client.15237 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1c86e40 0x557bd2080fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 4 client.15237 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.857+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bd1fdc350 con 0x557bd1c86e40 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.858+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1c86e40 0x557bd2080fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.858+0000 7f65b6ffd640 10 client.15237 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.858+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1c86e40 0x557bd2080fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fcd653e0c2f0450 server_cookie=71a6838a8f73a25b in_seq=0 out_seq=0 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65b6ffd640 1 -- 172.21.15.78:0/2213250686 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bd1fdc350 con 0x557bd1c86e40 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65b6ffd640 10 client.15237 handle_command_reply: tid=0 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 client.15237 shutdown 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1ca7590 msgr2=0x557bd2073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1ca7590 0x557bd2073730 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f65c00a4010 tx=0x7f65c00a6820 comp rx=0 tx=0).stop 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 shutdown_connections 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1c86e40 0x557bd2080fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1cc8270 0x557bd207a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65b6ffd640 0 client.15237 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1ca7590 0x557bd2073730 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2213250686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd20731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 >> 172.21.15.78:0/2213250686 conn(0x557bd1cbc3c0 msgr2=0x557bd2063d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 shutdown_connections 2024-04-25T13:59:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 1 -- 172.21.15.78:0/2213250686 wait complete. 2024-04-25T13:59:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.859+0000 7f65e49cb740 20 client.15237 trim_cache size 0 max 16384 2024-04-25T13:59:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 Processor -- start 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 -- start start 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd207a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1ca7590 0x557bd206c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557bd207f320 con 0x557bd1c86e40 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bd207f9f0 con 0x557bd1c9f3b0 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65e49cb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557bd20800c0 con 0x557bd1ca7590 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52426/0 (socket says 172.21.15.78:52426) 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 -- 172.21.15.78:0/2538750170 learned_addr learned my addr 172.21.15.78:0/2538750170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 -- 172.21.15.78:0/2538750170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1ca7590 msgr2=0x557bd206c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/2538750170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1ca7590 0x557bd206c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 -- 172.21.15.78:0/2538750170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 msgr2=0x557bd207a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/2538750170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd207a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 -- 172.21.15.78:0/2538750170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd206e0c0 con 0x557bd1c86e40 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.878+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/2538750170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2071e40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009900 tx=0x7f65c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d27890648cd83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65cd7fa640 1 -- 172.21.15.78:0/2538750170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8053070 con 0x557bd1c86e40 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65cd7fa640 1 -- 172.21.15.78:0/2538750170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c8031d10 con 0x557bd1c86e40 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65cd7fa640 1 -- 172.21.15.78:0/2538750170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c803f070 con 0x557bd1c86e40 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 -- 172.21.15.78:0/2538750170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 msgr2=0x557bd2071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2538750170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2071e40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009900 tx=0x7f65c802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 -- 172.21.15.78:0/2538750170 shutdown_connections 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2538750170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1ca7590 0x557bd206c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2538750170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1c9f3b0 0x557bd207a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 --2- 172.21.15.78:0/2538750170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2071e40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 -- 172.21.15.78:0/2538750170 >> 172.21.15.78:0/2538750170 conn(0x557bd1cbc3c0 msgr2=0x557bd2071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 -- 172.21.15.78:0/2538750170 shutdown_connections 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.879+0000 7f65e49cb740 1 -- 172.21.15.78:0/2538750170 wait complete. 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 -- start start 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 Processor -- start 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2075d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1c9f3b0 0x557bd2076270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557bd2080e60 con 0x557bd1c86e40 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bd2080790 con 0x557bd1ca7590 2024-04-25T13:59:05.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557bd2081530 con 0x557bd1c9f3b0 2024-04-25T13:59:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65e49cb740 10 client.0 _ll_register_callbacks cb 0x7f65c00a6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2075d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2075d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52440/0 (socket says 172.21.15.78:52440) 2024-04-25T13:59:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 -- 172.21.15.78:0/1951481553 learned_addr learned my addr 172.21.15.78:0/1951481553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:05.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65cffff640 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1c9f3b0 0x557bd2076270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 -- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1c9f3b0 msgr2=0x557bd2076270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1c9f3b0 0x557bd2076270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 -- 172.21.15.78:0/1951481553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 msgr2=0x557bd20791b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65d6ffd640 1 -- 172.21.15.78:0/1951481553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd2079770 con 0x557bd1c86e40 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.880+0000 7f65cffff640 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1c9f3b0 0x557bd2076270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65d6ffd640 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2075d30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f65c8051040 tx=0x7f65c8031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac1170e2a9f9063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65b6ffd640 1 -- 172.21.15.78:0/1951481553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c803f030 con 0x557bd1c86e40 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65b6ffd640 1 -- 172.21.15.78:0/1951481553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f65c8031b10 con 0x557bd1c86e40 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c8031d10 con 0x557bd1c86e40 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65b6ffd640 1 -- 172.21.15.78:0/1951481553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f65c8053070 con 0x557bd1c86e40 2024-04-25T13:59:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 10 client.15243 fetch_fsmap learned FSMap version 21 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557bd20e9e60 con 0x557bd1c86e40 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65b6ffd640 1 -- 172.21.15.78:0/1951481553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c8031d10 con 0x557bd1c86e40 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65b6ffd640 1 -- 172.21.15.78:0/1951481553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f65c8044960 con 0x557bd1c86e40 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 10 client.15243 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 10 client.15243 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 20 client.15243 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1cc8270 0x557bd20ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 4 client.15243 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.881+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "status"}) -- 0x557bd1fd61c0 con 0x557bd1cc8270 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65cffff640 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1cc8270 0x557bd20ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65b6ffd640 10 client.15243 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65cffff640 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1cc8270 0x557bd20ed030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf6e98deee351f03 server_cookie=763336b45fcdef2a in_seq=0 out_seq=0 2024-04-25T13:59:05.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65b6ffd640 1 -- 172.21.15.78:0/1951481553 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557bd1fd61c0 con 0x557bd1cc8270 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65b6ffd640 10 client.15243 handle_command_reply: tid=0 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65e49cb740 1 client.15243 shutdown 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 msgr2=0x557bd2075d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65e49cb740 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2075d30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f65c8051040 tx=0x7f65c8031850 comp rx=0 tx=0).stop 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 shutdown_connections 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.882+0000 7f65e49cb740 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x557bd1cc8270 0x557bd20ed030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65e49cb740 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bd1ca7590 0x557bd20791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65e49cb740 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557bd1c9f3b0 0x557bd2076270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65b6ffd640 0 client.15243 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65e49cb740 1 --2- 172.21.15.78:0/1951481553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557bd1c86e40 0x557bd2075d30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 >> 172.21.15.78:0/1951481553 conn(0x557bd1cbc3c0 msgr2=0x557bd2071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 shutdown_connections 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65e49cb740 1 -- 172.21.15.78:0/1951481553 wait complete. 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65e49cb740 20 client.15243 trim_cache size 0 max 16384 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-04-25T13:59:05.894 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24454, 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 110.807643126, 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 21, 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T13:59:05.895 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 137.140364022 2024-04-25T13:59:05.896 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65c40afb70 msgr2=0x7f65c40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65c40afb70 0x7f65c40b2080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x557bd206a540 tx=0x7f65d0009420 comp rx=0 tx=0).stop 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d8103c20 msgr2=0x7f65d819f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d8103c20 0x7f65d819f2d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d950 tx=0x7f65c000de10 comp rx=0 tx=0).stop 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 shutdown_connections 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65d8108920 0x7f65d81a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65c40afb70 0x7f65c40b2080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65d8103c20 0x7f65d819f2d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 --2- 172.21.15.78:0/3583173418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d81029e0 0x7f65d806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 >> 172.21.15.78:0/3583173418 conn(0x7f65d80fdbd0 msgr2=0x7f65d8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 shutdown_connections 2024-04-25T13:59:05.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:05.883+0000 7f65dda42640 1 -- 172.21.15.78:0/3583173418 wait complete. 2024-04-25T13:59:05.901 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:59:06.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 -- 172.21.15.78:0/789111671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc108e90 msgr2=0x7fdefc10b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 --2- 172.21.15.78:0/789111671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc108e90 0x7fdefc10b330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009900 tx=0x7fdef802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:06.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 -- 172.21.15.78:0/789111671 shutdown_connections 2024-04-25T13:59:06.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 --2- 172.21.15.78:0/789111671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdefc10b9c0 0x7fdefc10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 --2- 172.21.15.78:0/789111671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc108e90 0x7fdefc10b330 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 --2- 172.21.15.78:0/789111671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdefc103fe0 0x7fdefc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 -- 172.21.15.78:0/789111671 >> 172.21.15.78:0/789111671 conn(0x7fdefc0fd380 msgr2=0x7fdefc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 -- 172.21.15.78:0/789111671 shutdown_connections 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.009+0000 7fdf04053640 1 -- 172.21.15.78:0/789111671 wait complete. 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 Processor -- start 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 -- start start 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc103fe0 0x7fdefc19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdefc108e90 0x7fdefc19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdefc10b9c0 0x7fdefc1a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc111860 con 0x7fdefc103fe0 2024-04-25T13:59:06.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1116e0 con 0x7fdefc108e90 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf04053640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1119e0 con 0x7fdefc10b9c0 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc103fe0 0x7fdefc19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc103fe0 0x7fdefc19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52460/0 (socket says 172.21.15.78:52460) 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 -- 172.21.15.78:0/3416532393 learned_addr learned my addr 172.21.15.78:0/3416532393 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf025f0640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdefc10b9c0 0x7fdefc1a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf015ee640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdefc108e90 0x7fdefc19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 -- 172.21.15.78:0/3416532393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdefc10b9c0 msgr2=0x7fdefc1a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdefc10b9c0 0x7fdefc1a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 -- 172.21.15.78:0/3416532393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdefc108e90 msgr2=0x7fdefc19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdefc108e90 0x7fdefc19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf01def640 1 -- 172.21.15.78:0/3416532393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc1a3fa0 con 0x7fdefc103fe0 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf015ee640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdefc108e90 0x7fdefc19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:06.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.010+0000 7fdf025f0640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdefc10b9c0 0x7fdefc1a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:06.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.011+0000 7fdf01def640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc103fe0 0x7fdefc19c460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00d740 tx=0x7fdeec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.011+0000 7fdef2ffd640 1 -- 172.21.15.78:0/3416532393 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec017070 con 0x7fdefc103fe0 2024-04-25T13:59:06.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.011+0000 7fdef2ffd640 1 -- 172.21.15.78:0/3416532393 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeec00de50 con 0x7fdefc103fe0 2024-04-25T13:59:06.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.011+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc1a4290 con 0x7fdefc103fe0 2024-04-25T13:59:06.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.011+0000 7fdef2ffd640 1 -- 172.21.15.78:0/3416532393 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00f620 con 0x7fdefc103fe0 2024-04-25T13:59:06.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.011+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1a47e0 con 0x7fdefc103fe0 2024-04-25T13:59:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.013+0000 7fdef2ffd640 1 -- 172.21.15.78:0/3416532393 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdeec01a070 con 0x7fdefc103fe0 2024-04-25T13:59:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.013+0000 7fdf0afe0740 1 Processor -- start 2024-04-25T13:59:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.013+0000 7fdf0afe0740 1 -- start start 2024-04-25T13:59:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35ccae270 0x55b35d04f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc853b0 0x55b35d04f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d0567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b35cfb4740 con 0x55b35cc8d590 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b35cf503a0 con 0x55b35ccae270 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b35cfb0700 con 0x55b35cc853b0 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d0567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d0567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52468/0 (socket says 172.21.15.78:52468) 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 -- 172.21.15.78:0/1060356871 learned_addr learned my addr 172.21.15.78:0/1060356871 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf025f0640 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35ccae270 0x55b35d04f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf01def640 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc853b0 0x55b35d04f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdef2ffd640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdee80afaf0 0x7fdee80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 -- 172.21.15.78:0/1060356871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc853b0 msgr2=0x55b35d04f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc853b0 0x55b35d04f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 -- 172.21.15.78:0/1060356871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35ccae270 msgr2=0x55b35d04f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35ccae270 0x55b35d04f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 -- 172.21.15.78:0/1060356871 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35cf6c750 con 0x55b35cc8d590 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf025f0640 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35ccae270 0x55b35d04f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf025f0640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdee80afaf0 0x7fdee80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdf015ee640 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d0567f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdef8006010 tx=0x7fdef8032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc97528ab2fe6ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.014+0000 7fdef2ffd640 1 -- 172.21.15.78:0/3416532393 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdeec0de6d0 con 0x7fdefc103fe0 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdee7fff640 1 -- 172.21.15.78:0/1060356871 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef803f030 con 0x55b35cc8d590 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdee7fff640 1 -- 172.21.15.78:0/1060356871 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef8032e40 con 0x55b35cc8d590 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdee7fff640 1 -- 172.21.15.78:0/1060356871 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8035070 con 0x55b35cc8d590 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdf0afe0740 1 -- 172.21.15.78:0/1060356871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 msgr2=0x55b35d0567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d0567f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdef8006010 tx=0x7fdef8032c50 comp rx=0 tx=0).stop 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdf025f0640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdee80afaf0 0x7fdee80b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55b35d050540 tx=0x7fdef4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdf0afe0740 1 -- 172.21.15.78:0/1060356871 shutdown_connections 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d0567f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc853b0 0x55b35d04f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.015+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/1060356871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35ccae270 0x55b35d04f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 -- 172.21.15.78:0/1060356871 >> 172.21.15.78:0/1060356871 conn(0x55b35cca23c0 msgr2=0x55b35d04ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 -- 172.21.15.78:0/1060356871 shutdown_connections 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 -- 172.21.15.78:0/1060356871 wait complete. 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 -- start start 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 Processor -- start 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d0591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d059700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 0x55b35d060660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b35d0571e0 con 0x55b35cc8d590 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b35cf3bfe0 con 0x55b35cc853b0 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b35d057360 con 0x55b35ccae270 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.016+0000 7fdf0afe0740 10 client.0 _ll_register_callbacks cb 0x55b35cf52150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d059700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d059700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52472/0 (socket says 172.21.15.78:52472) 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 -- 172.21.15.78:0/710394252 learned_addr learned my addr 172.21.15.78:0/710394252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf025f0640 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 0x55b35d060660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf01def640 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d0591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 -- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 msgr2=0x55b35d060660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 0x55b35d060660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 -- 172.21.15.78:0/710394252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 msgr2=0x55b35d0591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d0591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 -- 172.21.15.78:0/710394252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35ccc62d0 con 0x55b35cc8d590 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf015ee640 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d059700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdef8033000 tx=0x7fdef8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409b5e49ae2a269d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdee57fa640 1 -- 172.21.15.78:0/710394252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8035050 con 0x55b35cc8d590 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdee57fa640 1 -- 172.21.15.78:0/710394252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdef8004030 con 0x55b35cc8d590 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b35cfc2350 con 0x55b35cc8d590 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdee57fa640 1 -- 172.21.15.78:0/710394252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdef803f040 con 0x55b35cc8d590 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf0afe0740 10 client.15255 fetch_fsmap learned FSMap version 21 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b35cd906b0 con 0x55b35cc8d590 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.017+0000 7fdee57fa640 1 -- 172.21.15.78:0/710394252 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b35cfc2350 con 0x55b35cc8d590 2024-04-25T13:59:06.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdee57fa640 1 -- 172.21.15.78:0/710394252 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdef80035f0 con 0x55b35cc8d590 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf0afe0740 10 client.15255 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf0afe0740 10 client.15255 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf0afe0740 20 client.15255 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc6ce40 0x55b35d066a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf0afe0740 4 client.15255 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b35cfbc1c0 con 0x55b35cc6ce40 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf01def640 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc6ce40 0x55b35d066a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdee57fa640 10 client.15255 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.018+0000 7fdf01def640 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc6ce40 0x55b35d066a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=260873aad1a25918 server_cookie=10086ebaf31a0527 in_seq=0 out_seq=0 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.019+0000 7fdee57fa640 1 -- 172.21.15.78:0/710394252 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b35cfbc1c0 con 0x55b35cc6ce40 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.019+0000 7fdee57fa640 10 client.15255 handle_command_reply: tid=0 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.019+0000 7fdf0afe0740 1 client.15255 shutdown 2024-04-25T13:59:06.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.019+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 msgr2=0x55b35d059700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.019+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d059700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdef8033000 tx=0x7fdef8005640 comp rx=0 tx=0).stop 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 shutdown_connections 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc6ce40 0x55b35d066a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 0x55b35d060660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc8d590 0x55b35d059700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdee57fa640 0 client.15255 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/710394252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d0591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 >> 172.21.15.78:0/710394252 conn(0x55b35cca23c0 msgr2=0x55b35d049d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 shutdown_connections 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 1 -- 172.21.15.78:0/710394252 wait complete. 2024-04-25T13:59:06.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.020+0000 7fdf0afe0740 20 client.15255 trim_cache size 0 max 16384 2024-04-25T13:59:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 Processor -- start 2024-04-25T13:59:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 -- start start 2024-04-25T13:59:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc853b0 0x55b35d053cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc8d590 0x55b35d057e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 0x55b35d045a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b35d064d70 con 0x55b35cc853b0 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b35d065440 con 0x55b35cc8d590 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b35d065b10 con 0x55b35ccae270 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc853b0 0x55b35d053cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc853b0 0x55b35d053cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52488/0 (socket says 172.21.15.78:52488) 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 -- 172.21.15.78:0/3962301560 learned_addr learned my addr 172.21.15.78:0/3962301560 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 -- 172.21.15.78:0/3962301560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 msgr2=0x55b35d045a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 --2- 172.21.15.78:0/3962301560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 0x55b35d045a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 -- 172.21.15.78:0/3962301560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc8d590 msgr2=0x55b35d057e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 --2- 172.21.15.78:0/3962301560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc8d590 0x55b35d057e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 -- 172.21.15.78:0/3962301560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35d04ef20 con 0x55b35cc853b0 2024-04-25T13:59:06.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.038+0000 7fdf01def640 1 --2- 172.21.15.78:0/3962301560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc853b0 0x55b35d053cb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0a6ae0 tx=0x7fdeec0c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9734bab7b0f5d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdee77fe640 1 -- 172.21.15.78:0/3962301560 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec021040 con 0x55b35cc853b0 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdee77fe640 1 -- 172.21.15.78:0/3962301560 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeec0c7e50 con 0x55b35cc853b0 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdee77fe640 1 -- 172.21.15.78:0/3962301560 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0a2e30 con 0x55b35cc853b0 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 -- 172.21.15.78:0/3962301560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc853b0 msgr2=0x55b35d053cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/3962301560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc853b0 0x55b35d053cb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0a6ae0 tx=0x7fdeec0c9040 comp rx=0 tx=0).stop 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 -- 172.21.15.78:0/3962301560 shutdown_connections 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/3962301560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35ccae270 0x55b35d045a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/3962301560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc8d590 0x55b35d057e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/3962301560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35cc853b0 0x55b35d053cb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 -- 172.21.15.78:0/3962301560 >> 172.21.15.78:0/3962301560 conn(0x55b35cca23c0 msgr2=0x55b35d057760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 -- 172.21.15.78:0/3962301560 shutdown_connections 2024-04-25T13:59:06.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.039+0000 7fdf0afe0740 1 -- 172.21.15.78:0/3962301560 wait complete. 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 -- start start 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 Processor -- start 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d05b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc8d590 0x55b35d05b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35ccae270 0x55b35d05e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b35d0668b0 con 0x55b35ccae270 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b35d0661e0 con 0x55b35cc853b0 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b35d066f80 con 0x55b35cc8d590 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf0afe0740 10 client.0 _ll_register_callbacks cb 0x7fdef8032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf025f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35ccae270 0x55b35d05e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf025f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35ccae270 0x55b35d05e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52504/0 (socket says 172.21.15.78:52504) 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.040+0000 7fdf025f0640 1 -- 172.21.15.78:0/4130942292 learned_addr learned my addr 172.21.15.78:0/4130942292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf01def640 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d05b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf015ee640 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc8d590 0x55b35d05b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf025f0640 1 -- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc8d590 msgr2=0x55b35d05b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf025f0640 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc8d590 0x55b35d05b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf025f0640 1 -- 172.21.15.78:0/4130942292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 msgr2=0x55b35d05b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf025f0640 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d05b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf025f0640 1 -- 172.21.15.78:0/4130942292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35d05ecb0 con 0x55b35ccae270 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf025f0640 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35ccae270 0x55b35d05e6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdef40138d0 tx=0x7fdef40094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ed3e1ed35edd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdee57fa640 1 -- 172.21.15.78:0/4130942292 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef401da90 con 0x55b35ccae270 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeec0c7e50 con 0x55b35ccae270 2024-04-25T13:59:06.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdee57fa640 1 -- 172.21.15.78:0/4130942292 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdef401dc30 con 0x55b35ccae270 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdee57fa640 1 -- 172.21.15.78:0/4130942292 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdef401de30 con 0x55b35ccae270 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf0afe0740 10 client.15261 fetch_fsmap learned FSMap version 21 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b35d0cf8b0 con 0x55b35ccae270 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdee57fa640 1 -- 172.21.15.78:0/4130942292 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeec0c7e50 con 0x55b35ccae270 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.041+0000 7fdee57fa640 1 -- 172.21.15.78:0/4130942292 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdef4030050 con 0x55b35ccae270 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf0afe0740 10 client.15261 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf0afe0740 10 client.15261 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf0afe0740 20 client.15261 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc42280 0x55b35d0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf0afe0740 4 client.15261 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "status"}) -- 0x55b35cf379a0 con 0x55b35cc42280 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf01def640 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc42280 0x55b35d0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdee57fa640 10 client.15261 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.042+0000 7fdf01def640 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc42280 0x55b35d0d0df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c8574e5a965ebce server_cookie=5b1abb090373a03 in_seq=0 out_seq=0 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdee57fa640 1 -- 172.21.15.78:0/4130942292 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b35cf379a0 con 0x55b35cc42280 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdee57fa640 10 client.15261 handle_command_reply: tid=0 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 client.15261 shutdown 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35ccae270 msgr2=0x55b35d05e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35ccae270 0x55b35d05e6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdef40138d0 tx=0x7fdef40094a0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 shutdown_connections 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b35ccae270 0x55b35d05e6f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55b35cc42280 0x55b35d0d0df0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b35cc8d590 0x55b35d05b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 --2- 172.21.15.78:0/4130942292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b35cc853b0 0x55b35d05b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdee57fa640 0 client.15261 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:06.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 >> 172.21.15.78:0/4130942292 conn(0x55b35cca23c0 msgr2=0x55b35d052580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 shutdown_connections 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 1 -- 172.21.15.78:0/4130942292 wait complete. 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf0afe0740 20 client.15261 trim_cache size 0 max 16384 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24484, 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T13:59:06.051 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T13:59:06.052 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 111.885399406, 2024-04-25T13:59:06.052 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 21, 2024-04-25T13:59:06.052 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-04-25T13:59:06.052 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T13:59:06.052 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 128.95588420999999 2024-04-25T13:59:06.052 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdee80afaf0 msgr2=0x7fdee80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdee80afaf0 0x7fdee80b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55b35d050540 tx=0x7fdef4004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc103fe0 msgr2=0x7fdefc19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc103fe0 0x7fdefc19c460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00d740 tx=0x7fdeec00dc00 comp rx=0 tx=0).stop 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 shutdown_connections 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdefc10b9c0 0x7fdefc1a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdefc108e90 0x7fdefc19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdee80afaf0 0x7fdee80b2000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 --2- 172.21.15.78:0/3416532393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdefc103fe0 0x7fdefc19c460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.043+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 >> 172.21.15.78:0/3416532393 conn(0x7fdefc0fd380 msgr2=0x7fdefc109ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.044+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 shutdown_connections 2024-04-25T13:59:06.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.044+0000 7fdf04053640 1 -- 172.21.15.78:0/3416532393 wait complete. 2024-04-25T13:59:06.055 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:59:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 -- 172.21.15.78:0/166702437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95201050a0 msgr2=0x7f95201054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 --2- 172.21.15.78:0/166702437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95201050a0 0x7f95201054b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f951c009900 tx=0x7f951c02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 -- 172.21.15.78:0/166702437 shutdown_connections 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 --2- 172.21.15.78:0/166702437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102250 0x7f952010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 --2- 172.21.15.78:0/166702437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9520101750 0x7f9520101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 --2- 172.21.15.78:0/166702437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95201050a0 0x7f95201054b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 -- 172.21.15.78:0/166702437 >> 172.21.15.78:0/166702437 conn(0x7f95200fd350 msgr2=0x7f95200ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 -- 172.21.15.78:0/166702437 shutdown_connections 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.163+0000 7f952811f640 1 -- 172.21.15.78:0/166702437 wait complete. 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 Processor -- start 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 -- start start 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9520101750 0x7f952019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9520102250 0x7f952019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95201050a0 0x7f95201a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f952010f610 con 0x7f9520101750 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f952010f490 con 0x7f95201050a0 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f952811f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f952010f790 con 0x7f9520102250 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f9525ebb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9520101750 0x7f952019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f9525ebb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9520101750 0x7f952019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52526/0 (socket says 172.21.15.78:52526) 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.164+0000 7f9525ebb640 1 -- 172.21.15.78:0/233799948 learned_addr learned my addr 172.21.15.78:0/233799948 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f95266bc640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95201050a0 0x7f95201a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9525ebb640 1 -- 172.21.15.78:0/233799948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9520102250 msgr2=0x7f952019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f95256ba640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9520102250 0x7f952019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9525ebb640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9520102250 0x7f952019a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9525ebb640 1 -- 172.21.15.78:0/233799948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95201050a0 msgr2=0x7f95201a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9525ebb640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95201050a0 0x7f95201a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9525ebb640 1 -- 172.21.15.78:0/233799948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95201a1dd0 con 0x7f9520101750 2024-04-25T13:59:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f95266bc640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95201050a0 0x7f95201a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9525ebb640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9520101750 0x7f952019a300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f951c032fd0 tx=0x7f951c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9516ffd640 1 -- 172.21.15.78:0/233799948 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c03f070 con 0x7f9520101750 2024-04-25T13:59:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9516ffd640 1 -- 172.21.15.78:0/233799948 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f951c0044d0 con 0x7f9520101750 2024-04-25T13:59:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95201a2060 con 0x7f9520101750 2024-04-25T13:59:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f9516ffd640 1 -- 172.21.15.78:0/233799948 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c035070 con 0x7f9520101750 2024-04-25T13:59:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.165+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95201a2550 con 0x7f9520101750 2024-04-25T13:59:06.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.167+0000 7f9516ffd640 1 -- 172.21.15.78:0/233799948 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f951c03b5f0 con 0x7f9520101750 2024-04-25T13:59:06.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.167+0000 7f952efcf740 1 Processor -- start 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f952efcf740 1 -- start start 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f952efcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ee1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f952efcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f952efcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 0x55d7a0ee87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f952efcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0e46740 con 0x55d7a0b173b0 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f952efcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0de23a0 con 0x55d7a0b40270 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f952efcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0e42700 con 0x55d7a0b1f590 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52536/0 (socket says 172.21.15.78:52536) 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 -- 172.21.15.78:0/2763320472 learned_addr learned my addr 172.21.15.78:0/2763320472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f95256ba640 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 0x55d7a0ee87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f95266bc640 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ee1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9516ffd640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f950c0afaf0 0x7f950c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 -- 172.21.15.78:0/2763320472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 msgr2=0x55d7a0ee87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 0x55d7a0ee87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 -- 172.21.15.78:0/2763320472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 msgr2=0x55d7a0ee1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ee1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f9525ebb640 1 -- 172.21.15.78:0/2763320472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0dfe750 con 0x55d7a0b173b0 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f95266bc640 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ee1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.168+0000 7f95266bc640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f950c0afaf0 0x7f950c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.169+0000 7f9516ffd640 1 -- 172.21.15.78:0/233799948 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f951c0ff2a0 con 0x7f9520101750 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.169+0000 7f9525ebb640 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee18f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f951c0ab150 tx=0x7f951c08dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83a7ff55066b9b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.169+0000 7f950bfff640 1 -- 172.21.15.78:0/2763320472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0c70c0 con 0x55d7a0b173b0 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.169+0000 7f95266bc640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f950c0afaf0 0x7f950c0b2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55d7a0ee2540 tx=0x7f9518004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.169+0000 7f950bfff640 1 -- 172.21.15.78:0/2763320472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f951c0c7280 con 0x55d7a0b173b0 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.169+0000 7f950bfff640 1 -- 172.21.15.78:0/2763320472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0e04e0 con 0x55d7a0b173b0 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 -- 172.21.15.78:0/2763320472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 msgr2=0x55d7a0ee18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee18f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f951c0ab150 tx=0x7f951c08dbe0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 -- 172.21.15.78:0/2763320472 shutdown_connections 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 0x55d7a0ee87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee18f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 --2- 172.21.15.78:0/2763320472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ee1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 -- 172.21.15.78:0/2763320472 >> 172.21.15.78:0/2763320472 conn(0x55d7a0b343c0 msgr2=0x55d7a0edcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 -- 172.21.15.78:0/2763320472 shutdown_connections 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 -- 172.21.15.78:0/2763320472 wait complete. 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 -- start start 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.170+0000 7f952efcf740 1 Processor -- start 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eeae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b1f590 0x55d7a0eeb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ef2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ee9000 con 0x55d7a0b1f590 2024-04-25T13:59:06.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0dcdfe0 con 0x55d7a0b40270 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ee9180 con 0x55d7a0b173b0 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 10 client.0 _ll_register_callbacks cb 0x55d7a0de4150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b1f590 0x55d7a0eeb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b1f590 0x55d7a0eeb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52540/0 (socket says 172.21.15.78:52540) 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 -- 172.21.15.78:0/3186878921 learned_addr learned my addr 172.21.15.78:0/3186878921 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f9525ebb640 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eeae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95266bc640 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ef2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 -- 172.21.15.78:0/3186878921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 msgr2=0x55d7a0eeae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eeae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 -- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 msgr2=0x55d7a0ef2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ef2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 -- 172.21.15.78:0/3186878921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a0b582d0 con 0x55d7a0b1f590 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95256ba640 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b1f590 0x55d7a0eeb3d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f95100106e0 tx=0x7f9510010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e664a9721cde0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95097fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95097fa640 1 -- 172.21.15.78:0/3186878921 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951001d060 con 0x55d7a0b1f590 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f95097fa640 1 -- 172.21.15.78:0/3186878921 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9510018430 con 0x55d7a0b1f590 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.171+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f951c0c7280 con 0x55d7a0b1f590 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f95097fa640 1 -- 172.21.15.78:0/3186878921 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f951000ee60 con 0x55d7a0b1f590 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 10 client.15273 fetch_fsmap learned FSMap version 21 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d7a0c226b0 con 0x55d7a0b1f590 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f95097fa640 1 -- 172.21.15.78:0/3186878921 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f951c0c7280 con 0x55d7a0b1f590 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f95097fa640 1 -- 172.21.15.78:0/3186878921 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f95100224a0 con 0x55d7a0b1f590 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 10 client.15273 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 20 client.15273 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0afee40 0x55d7a0ef8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 4 client.15273 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:06.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a0e54350 con 0x55d7a0afee40 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.172+0000 7f9525ebb640 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0afee40 0x55d7a0ef8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.173+0000 7f95097fa640 10 client.15273 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.173+0000 7f9525ebb640 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0afee40 0x55d7a0ef8c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b78c62c1e69703 server_cookie=beabb3cb7aa5ec18 in_seq=0 out_seq=0 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.173+0000 7f95097fa640 1 -- 172.21.15.78:0/3186878921 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7a0e54350 con 0x55d7a0afee40 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.173+0000 7f95097fa640 10 client.15273 handle_command_reply: tid=0 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.173+0000 7f952efcf740 1 client.15273 shutdown 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b1f590 msgr2=0x55d7a0eeb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b1f590 0x55d7a0eeb3d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f95100106e0 tx=0x7f9510010ba0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 shutdown_connections 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ef2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b1f590 0x55d7a0eeb3d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0afee40 0x55d7a0ef8c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 --2- 172.21.15.78:0/3186878921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eeae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f95097fa640 0 client.15273 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 >> 172.21.15.78:0/3186878921 conn(0x55d7a0b343c0 msgr2=0x55d7a0edbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 shutdown_connections 2024-04-25T13:59:06.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 1 -- 172.21.15.78:0/3186878921 wait complete. 2024-04-25T13:59:06.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.174+0000 7f952efcf740 20 client.15273 trim_cache size 0 max 16384 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 Processor -- start 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 -- start start 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 0x55d7a0ee4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ef2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ef6fc0 con 0x55d7a0b173b0 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ef7690 con 0x55d7a0b40270 2024-04-25T13:59:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.192+0000 7f952efcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ef7d60 con 0x55d7a0b1f590 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52554/0 (socket says 172.21.15.78:52554) 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 -- 172.21.15.78:0/4249599597 learned_addr learned my addr 172.21.15.78:0/4249599597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 -- 172.21.15.78:0/4249599597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 msgr2=0x55d7a0ee4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 --2- 172.21.15.78:0/4249599597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 0x55d7a0ee4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 -- 172.21.15.78:0/4249599597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 msgr2=0x55d7a0ef2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 --2- 172.21.15.78:0/4249599597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ef2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 -- 172.21.15.78:0/4249599597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0ee6070 con 0x55d7a0b173b0 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f9525ebb640 1 --2- 172.21.15.78:0/4249599597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee9b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f951c0c3070 tx=0x7f951c0ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fe2a2d010e8560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f950b7fe640 1 -- 172.21.15.78:0/4249599597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c048040 con 0x55d7a0b173b0 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f950b7fe640 1 -- 172.21.15.78:0/4249599597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f951c0ea0b0 con 0x55d7a0b173b0 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.193+0000 7f950b7fe640 1 -- 172.21.15.78:0/4249599597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c0c6ee0 con 0x55d7a0b173b0 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 -- 172.21.15.78:0/4249599597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 msgr2=0x55d7a0ee9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 --2- 172.21.15.78:0/4249599597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee9b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f951c0c3070 tx=0x7f951c0ea040 comp rx=0 tx=0).stop 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 -- 172.21.15.78:0/4249599597 shutdown_connections 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 --2- 172.21.15.78:0/4249599597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b40270 0x55d7a0ef2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 --2- 172.21.15.78:0/4249599597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b1f590 0x55d7a0ee4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 --2- 172.21.15.78:0/4249599597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b173b0 0x55d7a0ee9b80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 -- 172.21.15.78:0/4249599597 >> 172.21.15.78:0/4249599597 conn(0x55d7a0b343c0 msgr2=0x55d7a0ee9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 -- 172.21.15.78:0/4249599597 shutdown_connections 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 -- 172.21.15.78:0/4249599597 wait complete. 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 -- start start 2024-04-25T13:59:06.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.194+0000 7f952efcf740 1 Processor -- start 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b1f590 0x55d7a0eed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b40270 0x55d7a0ef0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ef8b00 con 0x55d7a0b40270 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ef8430 con 0x55d7a0b1f590 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0ef91d0 con 0x55d7a0b173b0 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 10 client.0 _ll_register_callbacks cb 0x7f951000b5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b40270 0x55d7a0ef0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b40270 0x55d7a0ef0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52558/0 (socket says 172.21.15.78:52558) 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 -- 172.21.15.78:0/3535242945 learned_addr learned my addr 172.21.15.78:0/3535242945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95256ba640 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b1f590 0x55d7a0eed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f9525ebb640 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 -- 172.21.15.78:0/3535242945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 msgr2=0x55d7a0eed460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eed460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 -- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b1f590 msgr2=0x55d7a0eed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b1f590 0x55d7a0eed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 -- 172.21.15.78:0/3535242945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a0ef0e50 con 0x55d7a0b40270 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95266bc640 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b40270 0x55d7a0ef0890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9518012e60 tx=0x7f951800a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df07d9d87cc23758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95097fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95097fa640 1 -- 172.21.15.78:0/3535242945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951801de00 con 0x55d7a0b40270 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f95097fa640 1 -- 172.21.15.78:0/3535242945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9518012020 con 0x55d7a0b40270 2024-04-25T13:59:06.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.195+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7a0e4e1c0 con 0x55d7a0b40270 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f95097fa640 1 -- 172.21.15.78:0/3535242945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9518012220 con 0x55d7a0b40270 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 10 client.15279 fetch_fsmap learned FSMap version 21 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d7a0ef21a0 con 0x55d7a0b40270 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f95097fa640 1 -- 172.21.15.78:0/3535242945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7a0e4e1c0 con 0x55d7a0b40270 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f95097fa640 1 -- 172.21.15.78:0/3535242945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f951802e050 con 0x55d7a0b40270 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 10 client.15279 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 20 client.15279 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0ad4280 0x55d7a0f62b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 4 client.15279 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "status"}) -- 0x55d7a0dc99a0 con 0x55d7a0ad4280 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.196+0000 7f9525ebb640 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0ad4280 0x55d7a0f62b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f95097fa640 10 client.15279 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:06.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f9525ebb640 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0ad4280 0x55d7a0f62b30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b83f603bb09e981 server_cookie=25c1b6b40a7d960f in_seq=0 out_seq=0 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f95097fa640 1 -- 172.21.15.78:0/3535242945 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d7a0dc99a0 con 0x55d7a0ad4280 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f95097fa640 10 client.15279 handle_command_reply: tid=0 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 client.15279 shutdown 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b40270 msgr2=0x55d7a0ef0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b40270 0x55d7a0ef0890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9518012e60 tx=0x7f951800a5a0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 shutdown_connections 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7a0b40270 0x55d7a0ef0890 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7a0b1f590 0x55d7a0eed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7a0ad4280 0x55d7a0f62b30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 --2- 172.21.15.78:0/3535242945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7a0b173b0 0x55d7a0eed460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f95097fa640 0 client.15279 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 >> 172.21.15.78:0/3535242945 conn(0x55d7a0b343c0 msgr2=0x55d7a0eea120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 shutdown_connections 2024-04-25T13:59:06.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 1 -- 172.21.15.78:0/3535242945 wait complete. 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.197+0000 7f952efcf740 20 client.15279 trim_cache size 0 max 16384 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "5cff6f4a-030a-11ef-bc93-c7b262605968", 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24512, 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 114.073278148, 2024-04-25T13:59:06.204 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 21, 2024-04-25T13:59:06.205 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 77, 2024-04-25T13:59:06.205 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T13:59:06.205 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 120.29658145099999 2024-04-25T13:59:06.205 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:06.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f950c0afaf0 msgr2=0x7f950c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f950c0afaf0 0x7f950c0b2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55d7a0ee2540 tx=0x7f9518004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9520101750 msgr2=0x7f952019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9520101750 0x7f952019a300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f951c032fd0 tx=0x7f951c0042e0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 shutdown_connections 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95201050a0 0x7f95201a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9520102250 0x7f952019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f950c0afaf0 0x7f950c0b2000 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 --2- 172.21.15.78:0/233799948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9520101750 0x7f952019a300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 >> 172.21.15.78:0/233799948 conn(0x7f95200fd350 msgr2=0x7f9520105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 shutdown_connections 2024-04-25T13:59:06.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:06.198+0000 7f952811f640 1 -- 172.21.15.78:0/233799948 wait complete. 2024-04-25T13:59:06.209 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:59:06.253 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:59:06.253 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:59:06.253 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:59:06.253 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T13:59:06.311 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T13:59:06.311 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T13:59:06.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:06.312 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-25T13:59:06.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:06.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-04-25T13:59:06.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:06 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2813888077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:06.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:06 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2100979585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:06.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:06 smithi078 ceph-mon[29163]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:59:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2024-04-25T13:59:06.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:59:06.393 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T13:59:06.393 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:59:06.393 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:06.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T13:59:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:06 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2813888077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:06 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2100979585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:06 smithi112 ceph-mon[32617]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:59:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:06 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2813888077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:06 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2100979585' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:06 smithi129 ceph-mon[32569]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T13:59:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:59:08.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 -- 172.21.15.78:0/667490486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984103390 msgr2=0x7ff9841037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:08.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 --2- 172.21.15.78:0/667490486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984103390 0x7ff9841037a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00b0d0 tx=0x7ff97c00b3e0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 -- 172.21.15.78:0/667490486 shutdown_connections 2024-04-25T13:59:08.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 --2- 172.21.15.78:0/667490486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff984108610 0x7ff98410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 --2- 172.21.15.78:0/667490486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984103390 0x7ff9841037a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 --2- 172.21.15.78:0/667490486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff984107cc0 0x7ff9841080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 -- 172.21.15.78:0/667490486 >> 172.21.15.78:0/667490486 conn(0x7ff9840fec90 msgr2=0x7ff9841010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:08.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 -- 172.21.15.78:0/667490486 shutdown_connections 2024-04-25T13:59:08.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.441+0000 7ff989de1700 1 -- 172.21.15.78:0/667490486 wait complete. 2024-04-25T13:59:08.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.442+0000 7ff989de1700 1 Processor -- start 2024-04-25T13:59:08.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.442+0000 7ff989de1700 1 -- start start 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff984103390 0x7ff98419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984107cc0 0x7ff98419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff984108610 0x7ff98419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff984113240 con 0x7ff984107cc0 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841130c0 con 0x7ff984103390 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841133c0 con 0x7ff984108610 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984107cc0 0x7ff98419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984107cc0 0x7ff98419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42936/0 (socket says 172.21.15.78:42936) 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 -- 172.21.15.78:0/1927267462 learned_addr learned my addr 172.21.15.78:0/1927267462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff9837fe700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff984103390 0x7ff98419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 -- 172.21.15.78:0/1927267462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff984108610 msgr2=0x7ff98419c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff984108610 0x7ff98419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 -- 172.21.15.78:0/1927267462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff984103390 msgr2=0x7ff98419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff984103390 0x7ff98419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 -- 172.21.15.78:0/1927267462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9841a6e90 con 0x7ff984107cc0 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff982ffd700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984107cc0 0x7ff98419c080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c8f0 tx=0x7ff97800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff980ff9700 1 -- 172.21.15.78:0/1927267462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978007b00 con 0x7ff984107cc0 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff980ff9700 1 -- 172.21.15.78:0/1927267462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff978007ca0 con 0x7ff984107cc0 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff984199990 con 0x7ff984107cc0 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff980ff9700 1 -- 172.21.15.78:0/1927267462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978018660 con 0x7ff984107cc0 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.443+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff984199f20 con 0x7ff984107cc0 2024-04-25T13:59:08.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.444+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff984051190 con 0x7ff984107cc0 2024-04-25T13:59:08.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.447+0000 7ff980ff9700 1 -- 172.21.15.78:0/1927267462 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff97800f450 con 0x7ff984107cc0 2024-04-25T13:59:08.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.447+0000 7ff980ff9700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff96c0afc00 0x7ff96c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:08.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.447+0000 7ff9837fe700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff96c0afc00 0x7ff96c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:08.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.447+0000 7ff980ff9700 1 -- 172.21.15.78:0/1927267462 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff9780dec10 con 0x7ff984107cc0 2024-04-25T13:59:08.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.448+0000 7ff9837fe700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff96c0afc00 0x7ff96c0b2110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009fd0 tx=0x7ff97c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:08.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.451+0000 7ff980ff9700 1 -- 172.21.15.78:0/1927267462 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff978014030 con 0x7ff984107cc0 2024-04-25T13:59:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.691+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff98402e510 con 0x7ff984107cc0 2024-04-25T13:59:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.691+0000 7ff980ff9700 1 -- 172.21.15.78:0/1927267462 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7ff9780a78f0 con 0x7ff984107cc0 2024-04-25T13:59:08.691 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:59:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff96c0afc00 msgr2=0x7ff96c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff96c0afc00 0x7ff96c0b2110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009fd0 tx=0x7ff97c00b040 comp rx=0 tx=0).stop 2024-04-25T13:59:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984107cc0 msgr2=0x7ff98419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984107cc0 0x7ff98419c080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c8f0 tx=0x7ff97800ccb0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 shutdown_connections 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff96c0afc00 0x7ff96c0b2110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff984108610 0x7ff98419c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff984103390 0x7ff98419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 --2- 172.21.15.78:0/1927267462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff984107cc0 0x7ff98419c080 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 >> 172.21.15.78:0/1927267462 conn(0x7ff9840fec90 msgr2=0x7ff984109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 shutdown_connections 2024-04-25T13:59:08.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:08.693+0000 7ff989de1700 1 -- 172.21.15.78:0/1927267462 wait complete. 2024-04-25T13:59:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:09 smithi078 ceph-mon[29163]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:59:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:09 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1927267462' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:59:09.508 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:59:09.508 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:59:09.508 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-04-25T13:59:09.509 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:59:09.509 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:09.509 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:59:09.509 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:59:09.509 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:59:09.509 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:59:09.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:09.509 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-04-25T13:59:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:09 smithi112 ceph-mon[32617]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:59:09.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:09 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1927267462' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:59:09.526 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:59:09.527 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:59:09.527 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:59:09.527 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:59:09.527 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-04-25T13:59:09.527 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:59:09.527 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:59:09.527 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42069sec preferred_lft 42069sec 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::93cc:cc59:36ef:6ab2/64 scope link noprefixroute 2024-04-25T13:59:09.528 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:59:09.529 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:59:09.529 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:59:09.529 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T13:59:09.529 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:59:09.529 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-04-25T13:59:09.529 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-04-25T13:59:09.529 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:59:09.529 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T13:59:09.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:59:09.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:09 smithi129 ceph-mon[32569]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T13:59:09.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:09 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1927267462' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:59:09.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:09.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:09.714 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:59:09.738 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-04-25T13:59:09.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:09.740 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-04-25T13:59:09.808 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-04-25T13:59:09.808 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:59:09.808 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:59:09.808 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:59:09.808 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:59:09.809 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:59:09.809 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T13:59:09.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:59:09.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:59:09.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:59:09.809 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T13:59:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:59:10.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:10.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:10.083 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-25T13:59:10.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:10.098 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-04-25T13:59:10.153 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:59:10.154 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T13:59:10.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:59:10.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:59:10.154 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T13:59:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:59:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:10.271 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:59:10.271 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:59:10.271 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T13:59:10.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:59:10.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:59:10.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:59:10.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:59:10.272 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:59:10.272 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T13:59:10.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:59:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:10.403 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:59:10.403 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T13:59:10.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-04-25T13:59:10.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:59:10.404 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T13:59:10.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:59:10.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:10.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:59:10.479 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:10.493 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:59:10.494 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:59:10.494 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:10 smithi112 ceph-mon[32617]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:10.548 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T13:59:10.548 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T13:59:10.548 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/ac7dbde1-25c1-4f0d-b6ec-2d78eebd8cd1 2024-04-25T13:59:10.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:10.548 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/ac7dbde1-25c1-4f0d-b6ec-2d78eebd8cd1 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T13:59:10.609 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:10 smithi078 ceph-mon[29163]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:10.644 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T13:59:10.644 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-25T13:59:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:10 smithi129 ceph-mon[32569]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T13:59:10.752 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T13:59:10.752 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-25T13:59:10.752 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.112:3300,172.21.15.129:3300" 2024-04-25T13:59:10.752 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@5cff6f4a-030a-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/ac7dbde1-25c1-4f0d-b6ec-2d78eebd8cd1 2024-04-25T13:59:10.752 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.78:3300/172.21.15.112:3300/172.21.15.129:3300" will pass to kernel 2024-04-25T13:59:10.753 INFO:teuthology.orchestra.run.smithi078.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T13:59:10.753 INFO:teuthology.orchestra.run.smithi078.stdout: You just mounted a file system that supports labels which does not 2024-04-25T13:59:10.753 INFO:teuthology.orchestra.run.smithi078.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T13:59:10.753 INFO:teuthology.orchestra.run.smithi078.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T13:59:10.753 INFO:teuthology.orchestra.run.smithi078.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T13:59:10.755 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T13:59:10.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:10.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:10.825 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:59:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:59:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:10.921 INFO:teuthology.orchestra.run.smithi078.stdout:client24847 2024-04-25T13:59:10.922 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T13:59:10.922 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:59:10.938 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:59:11.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:59:11.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:11.034 INFO:teuthology.orchestra.run.smithi078.stdout:client24847 2024-04-25T13:59:11.035 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:59:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:59:11.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:11.089 INFO:teuthology.orchestra.run.smithi078.stdout:client24847 2024-04-25T13:59:11.090 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968.client24847/status') 2024-04-25T13:59:11.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:59:11.147 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.24847 (3)172.21.15.78:0/2639871742 2024-04-25T13:59:11.147 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-04-25T13:59:11.148 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-04-25T13:59:11.148 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-04-25T13:59:11.148 INFO:teuthology.orchestra.run.smithi078.stderr:71 bytes copied, 4.3773e-05 s, 1.6 MB/s 2024-04-25T13:59:11.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:59:11.155 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:59:11.217 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:59:11.217 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T13:59:11.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T13:59:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/319452621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4109a70 msgr2=0x7f0ac410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/319452621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4109a70 0x7f0ac410be90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009900 tx=0x7f0abc02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/319452621 shutdown_connections 2024-04-25T13:59:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/319452621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4109a70 0x7f0ac410be90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/319452621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac4104d70 0x7f0ac4105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/319452621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ac4103b30 0x7f0ac4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/319452621 >> 172.21.15.78:0/319452621 conn(0x7f0ac40fed40 msgr2=0x7f0ac41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:11.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/319452621 shutdown_connections 2024-04-25T13:59:11.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.327+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/319452621 wait complete. 2024-04-25T13:59:11.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 Processor -- start 2024-04-25T13:59:11.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 -- start start 2024-04-25T13:59:11.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4103b30 0x7f0ac419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ac4104d70 0x7f0ac419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac4109a70 0x7f0ac41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410f6b0 con 0x7f0ac4103b30 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410f530 con 0x7f0ac4109a70 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac8b4e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410f830 con 0x7f0ac4104d70 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4103b30 0x7f0ac419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4103b30 0x7f0ac419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42962/0 (socket says 172.21.15.78:42962) 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 -- 172.21.15.78:0/2374133796 learned_addr learned my addr 172.21.15.78:0/2374133796 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac1d9b640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ac4104d70 0x7f0ac419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 -- 172.21.15.78:0/2374133796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ac4104d70 msgr2=0x7f0ac419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ac4104d70 0x7f0ac419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 -- 172.21.15.78:0/2374133796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac4109a70 msgr2=0x7f0ac41a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac4109a70 0x7f0ac41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.328+0000 7f0ac259c640 1 -- 172.21.15.78:0/2374133796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac41a1cb0 con 0x7f0ac4103b30 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.329+0000 7f0ac259c640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4103b30 0x7f0ac419a220 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b990 tx=0x7f0ab400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.329+0000 7f0ab37fe640 1 -- 172.21.15.78:0/2374133796 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4019070 con 0x7f0ac4103b30 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.329+0000 7f0ab37fe640 1 -- 172.21.15.78:0/2374133796 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab400cba0 con 0x7f0ac4103b30 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.329+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41a1fa0 con 0x7f0ac4103b30 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.329+0000 7f0ab37fe640 1 -- 172.21.15.78:0/2374133796 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40125a0 con 0x7f0ac4103b30 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.329+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41a2480 con 0x7f0ac4103b30 2024-04-25T13:59:11.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.330+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac4108880 con 0x7f0ac4103b30 2024-04-25T13:59:11.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.331+0000 7f0ab37fe640 1 -- 172.21.15.78:0/2374133796 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0ab401c070 con 0x7f0ac4103b30 2024-04-25T13:59:11.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.332+0000 7f0ab37fe640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0aac0afae0 0x7f0aac0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.332+0000 7f0ac1d9b640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0aac0afae0 0x7f0aac0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:11.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.332+0000 7f0ab37fe640 1 -- 172.21.15.78:0/2374133796 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0ab40de6c0 con 0x7f0ac4103b30 2024-04-25T13:59:11.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.332+0000 7f0ac1d9b640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0aac0afae0 0x7f0aac0b1ff0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005e10 tx=0x7f0ab8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:11.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.335+0000 7f0ab37fe640 1 -- 172.21.15.78:0/2374133796 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ab40a7290 con 0x7f0ac4103b30 2024-04-25T13:59:11.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.552+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f0ac402de00 con 0x7f0ac4103b30 2024-04-25T13:59:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.604+0000 7f0ab37fe640 1 -- 172.21.15.78:0/2374133796 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v22) ==== 117+0+0 (secure 0 0 0) 0x7f0ab40af310 con 0x7f0ac4103b30 2024-04-25T13:59:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0aac0afae0 msgr2=0x7f0aac0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0aac0afae0 0x7f0aac0b1ff0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005e10 tx=0x7f0ab8005da0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4103b30 msgr2=0x7f0ac419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4103b30 0x7f0ac419a220 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b990 tx=0x7f0ab400be50 comp rx=0 tx=0).stop 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 shutdown_connections 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ac4109a70 0x7f0ac41a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ac4104d70 0x7f0ac419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0aac0afae0 0x7f0aac0b1ff0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 --2- 172.21.15.78:0/2374133796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4103b30 0x7f0ac419a220 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 >> 172.21.15.78:0/2374133796 conn(0x7f0ac40fed40 msgr2=0x7f0ac4105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 shutdown_connections 2024-04-25T13:59:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.606+0000 7f0ac8b4e640 1 -- 172.21.15.78:0/2374133796 wait complete. 2024-04-25T13:59:11.618 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T13:59:11.629 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-25T13:59:11.666 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:59:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 -- 172.21.15.78:0/3197778412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688101730 msgr2=0x7f0688101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 --2- 172.21.15.78:0/3197778412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688101730 0x7f0688101ba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f067c009900 tx=0x7f067c02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 -- 172.21.15.78:0/3197778412 shutdown_connections 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 --2- 172.21.15.78:0/3197778412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0688102230 0x7f068810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 --2- 172.21.15.78:0/3197778412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688101730 0x7f0688101ba0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 --2- 172.21.15.78:0/3197778412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0688105080 0x7f0688105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 -- 172.21.15.78:0/3197778412 >> 172.21.15.78:0/3197778412 conn(0x7f06880fd330 msgr2=0x7f06880ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 -- 172.21.15.78:0/3197778412 shutdown_connections 2024-04-25T13:59:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 -- 172.21.15.78:0/3197778412 wait complete. 2024-04-25T13:59:11.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 Processor -- start 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.776+0000 7f068fc9f640 1 -- start start 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0688101730 0x7f068819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0688102230 0x7f068819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688105080 0x7f06881a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f068810f610 con 0x7f0688105080 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f068810f490 con 0x7f0688101730 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f068810f790 con 0x7f0688102230 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688105080 0x7f06881a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688105080 0x7f06881a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42988/0 (socket says 172.21.15.78:42988) 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 -- 172.21.15.78:0/1710193849 learned_addr learned my addr 172.21.15.78:0/1710193849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068da3b640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0688101730 0x7f068819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068d23a640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0688102230 0x7f068819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:11.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 -- 172.21.15.78:0/1710193849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0688102230 msgr2=0x7f068819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0688102230 0x7f068819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 -- 172.21.15.78:0/1710193849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0688101730 msgr2=0x7f068819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0688101730 0x7f068819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 -- 172.21.15.78:0/1710193849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06881a1db0 con 0x7f0688105080 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068d23a640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0688102230 0x7f068819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068e23c640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688105080 0x7f06881a1740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f067800cad0 tx=0x7f067800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f0686ffd640 1 -- 172.21.15.78:0/1710193849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067800ae20 con 0x7f0688105080 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f0686ffd640 1 -- 172.21.15.78:0/1710193849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0678005020 con 0x7f0688105080 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f0686ffd640 1 -- 172.21.15.78:0/1710193849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067800f6b0 con 0x7f0688105080 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06881a20a0 con 0x7f0688105080 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.777+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06881a25f0 con 0x7f0688105080 2024-04-25T13:59:11.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.778+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0688108a10 con 0x7f0688105080 2024-04-25T13:59:11.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.780+0000 7f0686ffd640 1 -- 172.21.15.78:0/1710193849 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f06780051d0 con 0x7f0688105080 2024-04-25T13:59:11.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.780+0000 7f0686ffd640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06700afb70 0x7f06700b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:11.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.781+0000 7f068da3b640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06700afb70 0x7f06700b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:11.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.781+0000 7f0686ffd640 1 -- 172.21.15.78:0/1710193849 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f06780df000 con 0x7f0688105080 2024-04-25T13:59:11.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.781+0000 7f068da3b640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06700afb70 0x7f06700b2080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0680005fd0 tx=0x7f0680005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:11.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.784+0000 7f0686ffd640 1 -- 172.21.15.78:0/1710193849 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0678014030 con 0x7f0688105080 2024-04-25T13:59:11.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:11 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2374133796' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:59:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.961+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06880637a0 con 0x7f0688105080 2024-04-25T13:59:11.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.963+0000 7f0686ffd640 1 -- 172.21.15.78:0/1710193849 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f06780a7cf0 con 0x7f0688105080 2024-04-25T13:59:11.963 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:11.963 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06700afb70 msgr2=0x7f06700b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06700afb70 0x7f06700b2080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0680005fd0 tx=0x7f0680005850 comp rx=0 tx=0).stop 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688105080 msgr2=0x7f06881a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688105080 0x7f06881a1740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f067800cad0 tx=0x7f067800cf90 comp rx=0 tx=0).stop 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 shutdown_connections 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f06700afb70 0x7f06700b2080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0688105080 0x7f06881a1740 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0688102230 0x7f068819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 --2- 172.21.15.78:0/1710193849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0688101730 0x7f068819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 >> 172.21.15.78:0/1710193849 conn(0x7f06880fd330 msgr2=0x7f0688105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:11.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 shutdown_connections 2024-04-25T13:59:11.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:11.965+0000 7f068fc9f640 1 -- 172.21.15.78:0/1710193849 wait complete. 2024-04-25T13:59:11.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:11 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2374133796' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:59:11.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:59:11.977 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:59:12.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:11 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2374133796' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 -- 172.21.15.78:0/1210689750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 msgr2=0x7ff184102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 --2- 172.21.15.78:0/1210689750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff184102e90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff178009900 tx=0x7ff17802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 -- 172.21.15.78:0/1210689750 shutdown_connections 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 --2- 172.21.15.78:0/1210689750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1841080d0 0x7ff18410a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 --2- 172.21.15.78:0/1210689750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1841033d0 0x7ff1841037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 --2- 172.21.15.78:0/1210689750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff184102e90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 -- 172.21.15.78:0/1210689750 >> 172.21.15.78:0/1210689750 conn(0x7ff1840fd380 msgr2=0x7ff1840ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 -- 172.21.15.78:0/1210689750 shutdown_connections 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.088+0000 7ff18b042640 1 -- 172.21.15.78:0/1210689750 wait complete. 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 Processor -- start 2024-04-25T13:59:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 -- start start 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff18419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1841033d0 0x7ff18419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1841080d0 0x7ff1841a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410d470 con 0x7ff184102190 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410d2f0 con 0x7ff1841033d0 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff18b042640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410d5f0 con 0x7ff1841080d0 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff188dde640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff18419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff188dde640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff18419a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43002/0 (socket says 172.21.15.78:43002) 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff188dde640 1 -- 172.21.15.78:0/2920917641 learned_addr learned my addr 172.21.15.78:0/2920917641 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.089+0000 7ff183fff640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1841033d0 0x7ff18419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff1895df640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1841080d0 0x7ff1841a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff188dde640 1 -- 172.21.15.78:0/2920917641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1841080d0 msgr2=0x7ff1841a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff188dde640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1841080d0 0x7ff1841a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff188dde640 1 -- 172.21.15.78:0/2920917641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1841033d0 msgr2=0x7ff18419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff188dde640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1841033d0 0x7ff18419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff188dde640 1 -- 172.21.15.78:0/2920917641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1841a1db0 con 0x7ff184102190 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff188dde640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff18419a270 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff178031440 tx=0x7ff178003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff181ffb640 1 -- 172.21.15.78:0/2920917641 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17803f070 con 0x7ff184102190 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff181ffb640 1 -- 172.21.15.78:0/2920917641 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff178031c50 con 0x7ff184102190 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1841a2040 con 0x7ff184102190 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff181ffb640 1 -- 172.21.15.78:0/2920917641 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178035030 con 0x7ff184102190 2024-04-25T13:59:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.090+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1841a2530 con 0x7ff184102190 2024-04-25T13:59:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.091+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff184106ee0 con 0x7ff184102190 2024-04-25T13:59:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.093+0000 7ff181ffb640 1 -- 172.21.15.78:0/2920917641 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff178042070 con 0x7ff184102190 2024-04-25T13:59:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.093+0000 7ff181ffb640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff1740afaf0 0x7ff1740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.093+0000 7ff183fff640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff1740afaf0 0x7ff1740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.093+0000 7ff181ffb640 1 -- 172.21.15.78:0/2920917641 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff1780ff4c0 con 0x7ff184102190 2024-04-25T13:59:12.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.094+0000 7ff183fff640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff1740afaf0 0x7ff1740b2000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff18419b310 tx=0x7ff170010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.098+0000 7ff181ffb640 1 -- 172.21.15.78:0/2920917641 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1780c8090 con 0x7ff184102190 2024-04-25T13:59:12.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.284+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1840637a0 con 0x7ff184102190 2024-04-25T13:59:12.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.285+0000 7ff181ffb640 1 -- 172.21.15.78:0/2920917641 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7ff1780c7990 con 0x7ff184102190 2024-04-25T13:59:12.285 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:59:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff1740afaf0 msgr2=0x7ff1740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff1740afaf0 0x7ff1740b2000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff18419b310 tx=0x7ff170010040 comp rx=0 tx=0).stop 2024-04-25T13:59:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 msgr2=0x7ff18419a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff18419a270 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff178031440 tx=0x7ff178003c10 comp rx=0 tx=0).stop 2024-04-25T13:59:12.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 shutdown_connections 2024-04-25T13:59:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1841080d0 0x7ff1841a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1841033d0 0x7ff18419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff1740afaf0 0x7ff1740b2000 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 --2- 172.21.15.78:0/2920917641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff184102190 0x7ff18419a270 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 >> 172.21.15.78:0/2920917641 conn(0x7ff1840fd380 msgr2=0x7ff1841040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 shutdown_connections 2024-04-25T13:59:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.287+0000 7ff18b042640 1 -- 172.21.15.78:0/2920917641 wait complete. 2024-04-25T13:59:12.299 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:59:12.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 -- 172.21.15.78:0/974601844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98109a30 msgr2=0x7fbd9810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 --2- 172.21.15.78:0/974601844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98109a30 0x7fbd9810be50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009ca0 tx=0x7fbd9002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 -- 172.21.15.78:0/974601844 shutdown_connections 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 --2- 172.21.15.78:0/974601844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98109a30 0x7fbd9810be50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 --2- 172.21.15.78:0/974601844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd98104d30 0x7fbd98105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 --2- 172.21.15.78:0/974601844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd98103af0 0x7fbd981047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 -- 172.21.15.78:0/974601844 >> 172.21.15.78:0/974601844 conn(0x7fbd980fece0 msgr2=0x7fbd98101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 -- 172.21.15.78:0/974601844 shutdown_connections 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.409+0000 7fbd9e813640 1 -- 172.21.15.78:0/974601844 wait complete. 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 Processor -- start 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 -- start start 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98103af0 0x7fbd9819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd98104d30 0x7fbd9819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd98109a30 0x7fbd981a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810f620 con 0x7fbd98103af0 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810f4a0 con 0x7fbd98109a30 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd9e813640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810f7a0 con 0x7fbd98104d30 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98103af0 0x7fbd9819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98103af0 0x7fbd9819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43018/0 (socket says 172.21.15.78:43018) 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.410+0000 7fbd97fff640 1 -- 172.21.15.78:0/3895238091 learned_addr learned my addr 172.21.15.78:0/3895238091 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd9cdb0640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd98109a30 0x7fbd981a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd97fff640 1 -- 172.21.15.78:0/3895238091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd98104d30 msgr2=0x7fbd9819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd977fe640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd98104d30 0x7fbd9819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd97fff640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd98104d30 0x7fbd9819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd97fff640 1 -- 172.21.15.78:0/3895238091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd98109a30 msgr2=0x7fbd981a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd97fff640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd98109a30 0x7fbd981a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd97fff640 1 -- 172.21.15.78:0/3895238091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd981a1d10 con 0x7fbd98103af0 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd9cdb0640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd98109a30 0x7fbd981a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd977fe640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd98104d30 0x7fbd9819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd97fff640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98103af0 0x7fbd9819a1d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800d740 tx=0x7fbd8800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd957fa640 1 -- 172.21.15.78:0/3895238091 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88017070 con 0x7fbd98103af0 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd957fa640 1 -- 172.21.15.78:0/3895238091 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd8800de50 con 0x7fbd98103af0 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd957fa640 1 -- 172.21.15.78:0/3895238091 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88003c20 con 0x7fbd98103af0 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981a1fa0 con 0x7fbd98103af0 2024-04-25T13:59:12.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.411+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981a24b0 con 0x7fbd98103af0 2024-04-25T13:59:12.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.412+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd98108840 con 0x7fbd98103af0 2024-04-25T13:59:12.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.414+0000 7fbd957fa640 1 -- 172.21.15.78:0/3895238091 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbd88003dc0 con 0x7fbd98103af0 2024-04-25T13:59:12.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.414+0000 7fbd957fa640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd800afb70 0x7fbd800b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.414+0000 7fbd977fe640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd800afb70 0x7fbd800b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.415+0000 7fbd957fa640 1 -- 172.21.15.78:0/3895238091 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbd880dea60 con 0x7fbd98103af0 2024-04-25T13:59:12.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.415+0000 7fbd977fe640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd800afb70 0x7fbd800b2080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbd9819b270 tx=0x7fbd8c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.418+0000 7fbd957fa640 1 -- 172.21.15.78:0/3895238091 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd880a76b0 con 0x7fbd98103af0 2024-04-25T13:59:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.594+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd9802dd00 con 0x7fbd98103af0 2024-04-25T13:59:12.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.595+0000 7fbd957fa640 1 -- 172.21.15.78:0/3895238091 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fbd880a6fb0 con 0x7fbd98103af0 2024-04-25T13:59:12.595 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T13:59:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd800afb70 msgr2=0x7fbd800b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd800afb70 0x7fbd800b2080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbd9819b270 tx=0x7fbd8c005e70 comp rx=0 tx=0).stop 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98103af0 msgr2=0x7fbd9819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98103af0 0x7fbd9819a1d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800d740 tx=0x7fbd8800dc00 comp rx=0 tx=0).stop 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 shutdown_connections 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd98109a30 0x7fbd981a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd98104d30 0x7fbd9819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbd800afb70 0x7fbd800b2080 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 --2- 172.21.15.78:0/3895238091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd98103af0 0x7fbd9819a1d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 >> 172.21.15.78:0/3895238091 conn(0x7fbd980fece0 msgr2=0x7fbd98105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 shutdown_connections 2024-04-25T13:59:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.597+0000 7fbd9e813640 1 -- 172.21.15.78:0/3895238091 wait complete. 2024-04-25T13:59:12.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:59:12.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 -- 172.21.15.78:0/1902415781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 msgr2=0x7f294c10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 --2- 172.21.15.78:0/1902415781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c10bcc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2944009ca0 tx=0x7f294402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 -- 172.21.15.78:0/1902415781 shutdown_connections 2024-04-25T13:59:12.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 --2- 172.21.15.78:0/1902415781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c10bcc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 --2- 172.21.15.78:0/1902415781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f294c101760 0x7f294c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 --2- 172.21.15.78:0/1902415781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f294c1050b0 0x7f294c1054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 -- 172.21.15.78:0/1902415781 >> 172.21.15.78:0/1902415781 conn(0x7f294c0fd380 msgr2=0x7f294c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 -- 172.21.15.78:0/1902415781 shutdown_connections 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.719+0000 7f2954534640 1 -- 172.21.15.78:0/1902415781 wait complete. 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 Processor -- start 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 -- start start 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f294c101760 0x7f294c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f294c1050b0 0x7f294c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f294c10f620 con 0x7f294c102260 2024-04-25T13:59:12.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f294c10f4a0 con 0x7f294c1050b0 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2954534640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f294c10f7a0 con 0x7f294c101760 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2951acf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2951acf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43038/0 (socket says 172.21.15.78:43038) 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f2951acf640 1 -- 172.21.15.78:0/1851525015 learned_addr learned my addr 172.21.15.78:0/1851525015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.720+0000 7f29522d0640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f294c101760 0x7f294c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2952ad1640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f294c1050b0 0x7f294c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2951acf640 1 -- 172.21.15.78:0/1851525015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f294c101760 msgr2=0x7f294c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2951acf640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f294c101760 0x7f294c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2951acf640 1 -- 172.21.15.78:0/1851525015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f294c1050b0 msgr2=0x7f294c1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2951acf640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f294c1050b0 0x7f294c1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2951acf640 1 -- 172.21.15.78:0/1851525015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c1a1d10 con 0x7f294c102260 2024-04-25T13:59:12.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2952ad1640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f294c1050b0 0x7f294c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:12.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2951acf640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c19a710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f29437fe640 1 -- 172.21.15.78:0/1851525015 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c00cd80 con 0x7f294c102260 2024-04-25T13:59:12.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f29437fe640 1 -- 172.21.15.78:0/1851525015 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c0108e0 con 0x7f294c102260 2024-04-25T13:59:12.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c1a1fa0 con 0x7f294c102260 2024-04-25T13:59:12.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f29437fe640 1 -- 172.21.15.78:0/1851525015 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c018980 con 0x7f294c102260 2024-04-25T13:59:12.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.721+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c1a2490 con 0x7f294c102260 2024-04-25T13:59:12.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.722+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294c10aa50 con 0x7f294c102260 2024-04-25T13:59:12.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.723+0000 7f29437fe640 1 -- 172.21.15.78:0/1851525015 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f293c010430 con 0x7f294c102260 2024-04-25T13:59:12.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.724+0000 7f29437fe640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29380afaf0 0x7f29380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:12.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.725+0000 7f29522d0640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29380afaf0 0x7f29380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:12.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.725+0000 7f29437fe640 1 -- 172.21.15.78:0/1851525015 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f293c0df4f0 con 0x7f294c102260 2024-04-25T13:59:12.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.725+0000 7f29522d0640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29380afaf0 0x7f29380b2000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f294c06b2d0 tx=0x7f2948005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.728+0000 7f29437fe640 1 -- 172.21.15.78:0/1851525015 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f293c0a8060 con 0x7f294c102260 2024-04-25T13:59:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.905+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f294c0637a0 con 0x7f294c102260 2024-04-25T13:59:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.906+0000 7f29437fe640 1 -- 172.21.15.78:0/1851525015 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f293c0b00e0 con 0x7f294c102260 2024-04-25T13:59:12.905 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T13:59:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.907+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29380afaf0 msgr2=0x7f29380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.907+0000 7f2954534640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29380afaf0 0x7f29380b2000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f294c06b2d0 tx=0x7f2948005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.907+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 msgr2=0x7f294c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.907+0000 7f2954534640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c19a710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 shutdown_connections 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f294c1050b0 0x7f294c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29380afaf0 0x7f29380b2000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f294c102260 0x7f294c19a710 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 --2- 172.21.15.78:0/1851525015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f294c101760 0x7f294c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 >> 172.21.15.78:0/1851525015 conn(0x7f294c0fd380 msgr2=0x7f294c107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 shutdown_connections 2024-04-25T13:59:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:12.908+0000 7f2954534640 1 -- 172.21.15.78:0/1851525015 wait complete. 2024-04-25T13:59:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2374133796' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:59:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:12 smithi129 ceph-mon[32569]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:59:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1710193849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:12 smithi129 ceph-mon[32569]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T13:59:12.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2920917641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:12.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3895238091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:12.918 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2374133796' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:59:12.918 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:12 smithi078 ceph-mon[29163]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:59:12.918 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1710193849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:12.918 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:12 smithi078 ceph-mon[29163]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T13:59:12.918 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2920917641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:12.918 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3895238091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:12.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:59:13.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2374133796' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T13:59:13.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:12 smithi112 ceph-mon[32617]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T13:59:13.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1710193849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:13.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:12 smithi112 ceph-mon[32617]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T13:59:13.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2920917641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:13.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3895238091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 -- 172.21.15.78:0/660386259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 msgr2=0x7f95fc10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 --2- 172.21.15.78:0/660386259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc10ad20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009ca0 tx=0x7f95f402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 -- 172.21.15.78:0/660386259 shutdown_connections 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 --2- 172.21.15.78:0/660386259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc10ad20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 --2- 172.21.15.78:0/660386259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95fc103c00 0x7f95fc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 --2- 172.21.15.78:0/660386259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc1029c0 0x7f95fc1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 -- 172.21.15.78:0/660386259 >> 172.21.15.78:0/660386259 conn(0x7f95fc0fdbd0 msgr2=0x7f95fc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 -- 172.21.15.78:0/660386259 shutdown_connections 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 -- 172.21.15.78:0/660386259 wait complete. 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.027+0000 7f9601a82640 1 Processor -- start 2024-04-25T13:59:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f9601a82640 1 -- start start 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f9601a82640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95fc1029c0 0x7f95fc06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f9601a82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc103c00 0x7f95fc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f9601a82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f9601a82640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc10e4f0 con 0x7f95fc108900 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f9601a82640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc10e370 con 0x7f95fc103c00 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f9601a82640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc10e670 con 0x7f95fc1029c0 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc1a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43058/0 (socket says 172.21.15.78:43058) 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 -- 172.21.15.78:0/390838750 learned_addr learned my addr 172.21.15.78:0/390838750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fb7fe640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95fc1029c0 0x7f95fc06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95faffd640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc103c00 0x7f95fc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 -- 172.21.15.78:0/390838750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95fc1029c0 msgr2=0x7f95fc06a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95fc1029c0 0x7f95fc06a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 -- 172.21.15.78:0/390838750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc103c00 msgr2=0x7f95fc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc103c00 0x7f95fc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 -- 172.21.15.78:0/390838750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc1a1cf0 con 0x7f95fc108900 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.028+0000 7f95fbfff640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc1a14d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f95f4004af0 tx=0x7f95f4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.029+0000 7f95f8ff9640 1 -- 172.21.15.78:0/390838750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4003b70 con 0x7f95fc108900 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.029+0000 7f95f8ff9640 1 -- 172.21.15.78:0/390838750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f4003d10 con 0x7f95fc108900 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.029+0000 7f95f8ff9640 1 -- 172.21.15.78:0/390838750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f402fb20 con 0x7f95fc108900 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.029+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc1a1f80 con 0x7f95fc108900 2024-04-25T13:59:13.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.029+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc1a23e0 con 0x7f95fc108900 2024-04-25T13:59:13.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.030+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc06afa0 con 0x7f95fc108900 2024-04-25T13:59:13.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.032+0000 7f95f8ff9640 1 -- 172.21.15.78:0/390838750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f95f4004580 con 0x7f95fc108900 2024-04-25T13:59:13.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.032+0000 7f95f8ff9640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95e40afaf0 0x7f95e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.033+0000 7f95fb7fe640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95e40afaf0 0x7f95e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.033+0000 7f95f8ff9640 1 -- 172.21.15.78:0/390838750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f95f4100350 con 0x7f95fc108900 2024-04-25T13:59:13.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.033+0000 7f95fb7fe640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95e40afaf0 0x7f95e40b2000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f95fc1035c0 tx=0x7f95ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.036+0000 7f95f8ff9640 1 -- 172.21.15.78:0/390838750 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95f40c8ec0 con 0x7f95fc108900 2024-04-25T13:59:13.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.215+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95fc02dc10 con 0x7f95fc108900 2024-04-25T13:59:13.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.217+0000 7f95f8ff9640 1 -- 172.21.15.78:0/390838750 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f95f40c87c0 con 0x7f95fc108900 2024-04-25T13:59:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:13.217 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95e40afaf0 msgr2=0x7f95e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95e40afaf0 0x7f95e40b2000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f95fc1035c0 tx=0x7f95ec005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 msgr2=0x7f95fc1a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc1a14d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f95f4004af0 tx=0x7f95f4031700 comp rx=0 tx=0).stop 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 shutdown_connections 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f95e40afaf0 0x7f95e40b2000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95fc108900 0x7f95fc1a14d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc103c00 0x7f95fc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 --2- 172.21.15.78:0/390838750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95fc1029c0 0x7f95fc06a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 >> 172.21.15.78:0/390838750 conn(0x7f95fc0fdbd0 msgr2=0x7f95fc1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 shutdown_connections 2024-04-25T13:59:13.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.219+0000 7f9601a82640 1 -- 172.21.15.78:0/390838750 wait complete. 2024-04-25T13:59:13.231 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:59:13.232 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:59:13.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 -- 172.21.15.78:0/4184445226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 msgr2=0x7f23d0102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 --2- 172.21.15.78:0/4184445226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d0102240 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009900 tx=0x7f23c002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:13.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 -- 172.21.15.78:0/4184445226 shutdown_connections 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 --2- 172.21.15.78:0/4184445226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d0109780 0x7f23d010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 --2- 172.21.15.78:0/4184445226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d0106c10 0x7f23d01090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 --2- 172.21.15.78:0/4184445226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d0102240 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 -- 172.21.15.78:0/4184445226 >> 172.21.15.78:0/4184445226 conn(0x7f23d00fd340 msgr2=0x7f23d00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 -- 172.21.15.78:0/4184445226 shutdown_connections 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 -- 172.21.15.78:0/4184445226 wait complete. 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 Processor -- start 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.343+0000 7f23d6114640 1 -- start start 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23d6114640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23d6114640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d0106c10 0x7f23d019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23d6114640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d0109780 0x7f23d01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23d6114640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23d010f610 con 0x7f23d0101e30 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23d6114640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23d010f490 con 0x7f23d0109780 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23d6114640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23d010f790 con 0x7f23d0106c10 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43076/0 (socket says 172.21.15.78:43076) 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 -- 172.21.15.78:0/4149760174 learned_addr learned my addr 172.21.15.78:0/4149760174 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23c7fff640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d0106c10 0x7f23d019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 -- 172.21.15.78:0/4149760174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d0106c10 msgr2=0x7f23d019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d0106c10 0x7f23d019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 -- 172.21.15.78:0/4149760174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d0109780 msgr2=0x7f23d01a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cffff640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d0109780 0x7f23d01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d0109780 0x7f23d01a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 -- 172.21.15.78:0/4149760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d01a1d80 con 0x7f23d0101e30 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cf7fe640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d019a240 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23c0031440 tx=0x7f23c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cd7fa640 1 -- 172.21.15.78:0/4149760174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c003f070 con 0x7f23d0101e30 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d01a2010 con 0x7f23d0101e30 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.344+0000 7f23cd7fa640 1 -- 172.21.15.78:0/4149760174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c0031c50 con 0x7f23d0101e30 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.345+0000 7f23cd7fa640 1 -- 172.21.15.78:0/4149760174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0035030 con 0x7f23d0101e30 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.345+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d01a2550 con 0x7f23d0101e30 2024-04-25T13:59:13.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.345+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d00fee60 con 0x7f23d0101e30 2024-04-25T13:59:13.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.348+0000 7f23cd7fa640 1 -- 172.21.15.78:0/4149760174 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f23c0042070 con 0x7f23d0101e30 2024-04-25T13:59:13.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.348+0000 7f23cd7fa640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23bc0afaf0 0x7f23bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.348+0000 7f23c7fff640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23bc0afaf0 0x7f23bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.348+0000 7f23cd7fa640 1 -- 172.21.15.78:0/4149760174 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f23c00ffca0 con 0x7f23d0101e30 2024-04-25T13:59:13.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.349+0000 7f23c7fff640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23bc0afaf0 0x7f23bc0b2000 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f23d019b2e0 tx=0x7f23b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.352+0000 7f23cd7fa640 1 -- 172.21.15.78:0/4149760174 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23c00c8870 con 0x7f23d0101e30 2024-04-25T13:59:13.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.529+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23d00637a0 con 0x7f23d0101e30 2024-04-25T13:59:13.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.529+0000 7f23cd7fa640 1 -- 172.21.15.78:0/4149760174 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f23c00c8170 con 0x7f23d0101e30 2024-04-25T13:59:13.529 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23bc0afaf0 msgr2=0x7f23bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23bc0afaf0 0x7f23bc0b2000 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f23d019b2e0 tx=0x7f23b8005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 msgr2=0x7f23d019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d019a240 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23c0031440 tx=0x7f23c0003c10 comp rx=0 tx=0).stop 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 shutdown_connections 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23d0109780 0x7f23d01a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d0106c10 0x7f23d019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23bc0afaf0 0x7f23bc0b2000 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 --2- 172.21.15.78:0/4149760174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23d0101e30 0x7f23d019a240 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 >> 172.21.15.78:0/4149760174 conn(0x7f23d00fd340 msgr2=0x7f23d010a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:13.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 shutdown_connections 2024-04-25T13:59:13.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.531+0000 7f23d6114640 1 -- 172.21.15.78:0/4149760174 wait complete. 2024-04-25T13:59:13.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:59:13.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 -- 172.21.15.78:0/4228765355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 msgr2=0x7f47a81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 --2- 172.21.15.78:0/4228765355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a81047f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f479c009900 tx=0x7f479c02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 -- 172.21.15.78:0/4228765355 shutdown_connections 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 --2- 172.21.15.78:0/4228765355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a8109a30 0x7f47a810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 --2- 172.21.15.78:0/4228765355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47a8104d30 0x7f47a8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 --2- 172.21.15.78:0/4228765355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a81047f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 -- 172.21.15.78:0/4228765355 >> 172.21.15.78:0/4228765355 conn(0x7f47a80fece0 msgr2=0x7f47a8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 -- 172.21.15.78:0/4228765355 shutdown_connections 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.654+0000 7f47aecf2640 1 -- 172.21.15.78:0/4228765355 wait complete. 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 Processor -- start 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 -- start start 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a8104d30 0x7f47a819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47a8109a30 0x7f47a81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810f5f0 con 0x7f47a8103af0 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810f470 con 0x7f47a8104d30 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aecf2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810f770 con 0x7f47a8109a30 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43088/0 (socket says 172.21.15.78:43088) 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 -- 172.21.15.78:0/2615397218 learned_addr learned my addr 172.21.15.78:0/2615397218 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47a7fff640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a8104d30 0x7f47a819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47ad28f640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47a8109a30 0x7f47a81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 -- 172.21.15.78:0/2615397218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47a8109a30 msgr2=0x7f47a81a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47a8109a30 0x7f47a81a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 -- 172.21.15.78:0/2615397218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a8104d30 msgr2=0x7f47a819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a8104d30 0x7f47a819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.655+0000 7f47aca8e640 1 -- 172.21.15.78:0/2615397218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a81a1d90 con 0x7f47a8103af0 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.656+0000 7f47aca8e640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a819a250 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f479c006010 tx=0x7f479c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.656+0000 7f47a5ffb640 1 -- 172.21.15.78:0/2615397218 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c03f030 con 0x7f47a8103af0 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.656+0000 7f47a5ffb640 1 -- 172.21.15.78:0/2615397218 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479c003aa0 con 0x7f47a8103af0 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.656+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a81a2020 con 0x7f47a8103af0 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.656+0000 7f47a5ffb640 1 -- 172.21.15.78:0/2615397218 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c035070 con 0x7f47a8103af0 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.656+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a2510 con 0x7f47a8103af0 2024-04-25T13:59:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.657+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a806a160 con 0x7f47a8103af0 2024-04-25T13:59:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.658+0000 7f47a5ffb640 1 -- 172.21.15.78:0/2615397218 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f479c042030 con 0x7f47a8103af0 2024-04-25T13:59:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.659+0000 7f47a5ffb640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47980afaf0 0x7f47980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.659+0000 7f47a7fff640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47980afaf0 0x7f47980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.659+0000 7f47a5ffb640 1 -- 172.21.15.78:0/2615397218 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f479c0ff7b0 con 0x7f47a8103af0 2024-04-25T13:59:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.660+0000 7f47a7fff640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47980afaf0 0x7f47980b2000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f47a819b2f0 tx=0x7f4794005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.663+0000 7f47a5ffb640 1 -- 172.21.15.78:0/2615397218 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f479c0c8360 con 0x7f47a8103af0 2024-04-25T13:59:13.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.840+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47a802db50 con 0x7f47a8103af0 2024-04-25T13:59:13.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.841+0000 7f47a5ffb640 1 -- 172.21.15.78:0/2615397218 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f479c0d03e0 con 0x7f47a8103af0 2024-04-25T13:59:13.840 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T13:59:13.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.842+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47980afaf0 msgr2=0x7f47980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.842+0000 7f47aecf2640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47980afaf0 0x7f47980b2000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f47a819b2f0 tx=0x7f4794005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.842+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 msgr2=0x7f47a819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.842+0000 7f47aecf2640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a819a250 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f479c006010 tx=0x7f479c004270 comp rx=0 tx=0).stop 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 shutdown_connections 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47a8109a30 0x7f47a81a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47a8104d30 0x7f47a819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47980afaf0 0x7f47980b2000 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 --2- 172.21.15.78:0/2615397218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a8103af0 0x7f47a819a250 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 >> 172.21.15.78:0/2615397218 conn(0x7f47a80fece0 msgr2=0x7f47a8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 shutdown_connections 2024-04-25T13:59:13.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.843+0000 7f47aecf2640 1 -- 172.21.15.78:0/2615397218 wait complete. 2024-04-25T13:59:13.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1851525015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:13.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/390838750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:13.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4149760174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:13.854 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:59:13.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1851525015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:13.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/390838750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:13.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4149760174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 -- 172.21.15.78:0/235967528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 msgr2=0x7f66d4103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 --2- 172.21.15.78:0/235967528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d4103700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009900 tx=0x7f66c402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 -- 172.21.15.78:0/235967528 shutdown_connections 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 --2- 172.21.15.78:0/235967528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d4108940 0x7f66d410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 --2- 172.21.15.78:0/235967528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4103c40 0x7f66d4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 --2- 172.21.15.78:0/235967528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d4103700 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 -- 172.21.15.78:0/235967528 >> 172.21.15.78:0/235967528 conn(0x7f66d40fdbf0 msgr2=0x7f66d41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 -- 172.21.15.78:0/235967528 shutdown_connections 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.965+0000 7f66d9d06640 1 -- 172.21.15.78:0/235967528 wait complete. 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 Processor -- start 2024-04-25T13:59:13.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 -- start start 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d4103c40 0x7f66d419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4108940 0x7f66d41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410e500 con 0x7f66d4102a00 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410e380 con 0x7f66d4108940 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.966+0000 7f66d9d06640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410e680 con 0x7f66d4103c40 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43098/0 (socket says 172.21.15.78:43098) 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 -- 172.21.15.78:0/475791378 learned_addr learned my addr 172.21.15.78:0/475791378 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d3fff640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4108940 0x7f66d41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d2ffd640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d4103c40 0x7f66d419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 -- 172.21.15.78:0/475791378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d4103c40 msgr2=0x7f66d419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d4103c40 0x7f66d419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 -- 172.21.15.78:0/475791378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4108940 msgr2=0x7f66d41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4108940 0x7f66d41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 -- 172.21.15.78:0/475791378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d41a1cf0 con 0x7f66d4102a00 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d3fff640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4108940 0x7f66d41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d2ffd640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d4103c40 0x7f66d419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d37fe640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d419a260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006010 tx=0x7f66c4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d0ff9640 1 -- 172.21.15.78:0/475791378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c403f030 con 0x7f66d4102a00 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d41a1f80 con 0x7f66d4102a00 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d0ff9640 1 -- 172.21.15.78:0/475791378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c4003aa0 con 0x7f66d4102a00 2024-04-25T13:59:13.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d0ff9640 1 -- 172.21.15.78:0/475791378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4035070 con 0x7f66d4102a00 2024-04-25T13:59:13.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.967+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d41a23e0 con 0x7f66d4102a00 2024-04-25T13:59:13.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.968+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d41a26d0 con 0x7f66d4102a00 2024-04-25T13:59:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.970+0000 7f66d0ff9640 1 -- 172.21.15.78:0/475791378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f66c4042030 con 0x7f66d4102a00 2024-04-25T13:59:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.970+0000 7f66d0ff9640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66bc0afaf0 0x7f66bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.970+0000 7f66d2ffd640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66bc0afaf0 0x7f66bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.970+0000 7f66d0ff9640 1 -- 172.21.15.78:0/475791378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f66c40ff670 con 0x7f66d4102a00 2024-04-25T13:59:13.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.971+0000 7f66d2ffd640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66bc0afaf0 0x7f66bc0b2000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66d419b300 tx=0x7f66c8005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:13.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:13.974+0000 7f66d0ff9640 1 -- 172.21.15.78:0/475791378 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66c40c8240 con 0x7f66d4102a00 2024-04-25T13:59:14.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1851525015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:14.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/390838750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:14.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4149760174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:14.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.153+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66d41a29c0 con 0x7f66d4102a00 2024-04-25T13:59:14.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.153+0000 7f66d0ff9640 1 -- 172.21.15.78:0/475791378 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f66c40c7b40 con 0x7f66d4102a00 2024-04-25T13:59:14.154 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T13:59:14.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66bc0afaf0 msgr2=0x7f66bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66bc0afaf0 0x7f66bc0b2000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66d419b300 tx=0x7f66c8005db0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 msgr2=0x7f66d419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d419a260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006010 tx=0x7f66c4031a50 comp rx=0 tx=0).stop 2024-04-25T13:59:14.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 shutdown_connections 2024-04-25T13:59:14.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4108940 0x7f66d41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66d4103c40 0x7f66d419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f66bc0afaf0 0x7f66bc0b2000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 --2- 172.21.15.78:0/475791378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102a00 0x7f66d419a260 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.155+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 >> 172.21.15.78:0/475791378 conn(0x7f66d40fdbf0 msgr2=0x7f66d4109ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:14.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.156+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 shutdown_connections 2024-04-25T13:59:14.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.156+0000 7f66d9d06640 1 -- 172.21.15.78:0/475791378 wait complete. 2024-04-25T13:59:14.167 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-25T13:59:14.168 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:59:14.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 -- 172.21.15.78:0/1965992431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c1029c0 msgr2=0x7fac9c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 --2- 172.21.15.78:0/1965992431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c1029c0 0x7fac9c1036c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fac90009900 tx=0x7fac9002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 -- 172.21.15.78:0/1965992431 shutdown_connections 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 --2- 172.21.15.78:0/1965992431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c108900 0x7fac9c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 --2- 172.21.15.78:0/1965992431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac9c103c00 0x7fac9c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 --2- 172.21.15.78:0/1965992431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c1029c0 0x7fac9c1036c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 -- 172.21.15.78:0/1965992431 >> 172.21.15.78:0/1965992431 conn(0x7fac9c0fdbd0 msgr2=0x7fac9c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 -- 172.21.15.78:0/1965992431 shutdown_connections 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 -- 172.21.15.78:0/1965992431 wait complete. 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.277+0000 7faca2a42640 1 Processor -- start 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca2a42640 1 -- start start 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca2a42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c1029c0 0x7fac9c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca2a42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c103c00 0x7fac9c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca2a42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac9c108900 0x7fac9c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca2a42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10f5f0 con 0x7fac9c103c00 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca2a42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10f470 con 0x7fac9c1029c0 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca2a42640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10f770 con 0x7fac9c108900 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c103c00 0x7fac9c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c103c00 0x7fac9c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43108/0 (socket says 172.21.15.78:43108) 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 -- 172.21.15.78:0/2082016280 learned_addr learned my addr 172.21.15.78:0/2082016280 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7faca0fdf640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac9c108900 0x7fac9c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9bfff640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c1029c0 0x7fac9c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 -- 172.21.15.78:0/2082016280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac9c108900 msgr2=0x7fac9c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac9c108900 0x7fac9c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 -- 172.21.15.78:0/2082016280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c1029c0 msgr2=0x7fac9c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c1029c0 0x7fac9c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 -- 172.21.15.78:0/2082016280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c1a1d80 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9bfff640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c1029c0 0x7fac9c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.278+0000 7fac9b7fe640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c103c00 0x7fac9c19a780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c970 tx=0x7fac8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.279+0000 7fac997fa640 1 -- 172.21.15.78:0/2082016280 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c007bb0 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.279+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c1a2070 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.279+0000 7fac997fa640 1 -- 172.21.15.78:0/2082016280 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac8c007d50 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.279+0000 7fac997fa640 1 -- 172.21.15.78:0/2082016280 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00f450 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.279+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c1a2570 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.280+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c107710 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.281+0000 7fac997fa640 1 -- 172.21.15.78:0/2082016280 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fac8c004f00 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.282+0000 7fac997fa640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac840afaf0 0x7fac840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.282+0000 7fac9bfff640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac840afaf0 0x7fac840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.282+0000 7fac997fa640 1 -- 172.21.15.78:0/2082016280 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fac8c0dee20 con 0x7fac9c103c00 2024-04-25T13:59:14.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.282+0000 7fac9bfff640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac840afaf0 0x7fac840b2000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fac90005f70 tx=0x7fac90000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.286+0000 7fac997fa640 1 -- 172.21.15.78:0/2082016280 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac8c014030 con 0x7fac9c103c00 2024-04-25T13:59:14.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.503+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac9c02db50 con 0x7fac9c103c00 2024-04-25T13:59:14.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.504+0000 7fac997fa640 1 -- 172.21.15.78:0/2082016280 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fac8c0a79f0 con 0x7fac9c103c00 2024-04-25T13:59:14.503 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:14.504 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T13:59:14.504 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T13:59:14.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac840afaf0 msgr2=0x7fac840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac840afaf0 0x7fac840b2000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fac90005f70 tx=0x7fac90000bc0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c103c00 msgr2=0x7fac9c19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c103c00 0x7fac9c19a780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c970 tx=0x7fac8c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 shutdown_connections 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac9c108900 0x7fac9c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac840afaf0 0x7fac840b2000 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac9c103c00 0x7fac9c19a780 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 --2- 172.21.15.78:0/2082016280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac9c1029c0 0x7fac9c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 >> 172.21.15.78:0/2082016280 conn(0x7fac9c0fdbd0 msgr2=0x7fac9c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:14.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 shutdown_connections 2024-04-25T13:59:14.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.506+0000 7faca2a42640 1 -- 172.21.15.78:0/2082016280 wait complete. 2024-04-25T13:59:14.517 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-25T13:59:14.517 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-25T13:59:14.518 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 -- 172.21.15.78:0/1598899805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec106f30 msgr2=0x7f0cec109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/1598899805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec106f30 0x7f0cec109350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009900 tx=0x7f0ce002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 -- 172.21.15.78:0/1598899805 shutdown_connections 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/1598899805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec109890 0x7f0cec10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/1598899805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec106f30 0x7f0cec109350 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/1598899805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cec1045d0 0x7f0cec1069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 -- 172.21.15.78:0/1598899805 >> 172.21.15.78:0/1598899805 conn(0x7f0cec0fdbd0 msgr2=0x7f0cec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:14.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 -- 172.21.15.78:0/1598899805 shutdown_connections 2024-04-25T13:59:14.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 -- 172.21.15.78:0/1598899805 wait complete. 2024-04-25T13:59:14.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.629+0000 7f0cf21f3640 1 Processor -- start 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cf21f3640 1 -- start start 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cf21f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec1045d0 0x7f0cec19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cf21f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cec106f30 0x7f0cec19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cf21f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec109890 0x7f0cec1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cf21f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec10f640 con 0x7f0cec1045d0 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cf21f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec10f4c0 con 0x7f0cec106f30 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cf21f3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec10f7c0 con 0x7f0cec109890 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec1045d0 0x7f0cec19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec1045d0 0x7f0cec19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43134/0 (socket says 172.21.15.78:43134) 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4253136402 learned_addr learned my addr 172.21.15.78:0/4253136402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0cebfff640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec109890 0x7f0cec1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceaffd640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cec106f30 0x7f0cec19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4253136402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec109890 msgr2=0x7f0cec1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec109890 0x7f0cec1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4253136402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cec106f30 msgr2=0x7f0cec19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cec106f30 0x7f0cec19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.630+0000 7f0ceb7fe640 1 -- 172.21.15.78:0/4253136402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec1a1df0 con 0x7f0cec1045d0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.631+0000 7f0ceb7fe640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec1045d0 0x7f0cec19a2b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00e9c0 tx=0x7f0cdc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.631+0000 7f0ce8ff9640 1 -- 172.21.15.78:0/4253136402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00cd80 con 0x7f0cec1045d0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.631+0000 7f0ce8ff9640 1 -- 172.21.15.78:0/4253136402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cdc0054d0 con 0x7f0cec1045d0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.631+0000 7f0ce8ff9640 1 -- 172.21.15.78:0/4253136402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc010430 con 0x7f0cec1045d0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.631+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec1a20e0 con 0x7f0cec1045d0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.631+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec1a2600 con 0x7f0cec1045d0 2024-04-25T13:59:14.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.632+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec0ff6f0 con 0x7f0cec1045d0 2024-04-25T13:59:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.633+0000 7f0ce8ff9640 1 -- 172.21.15.78:0/4253136402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0cdc004fb0 con 0x7f0cec1045d0 2024-04-25T13:59:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.634+0000 7f0ce8ff9640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.634+0000 7f0ceaffd640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.634+0000 7f0ce8ff9640 1 -- 172.21.15.78:0/4253136402 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0cdc0df1f0 con 0x7f0cec1045d0 2024-04-25T13:59:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.634+0000 7f0ceaffd640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0004770 tx=0x7f0ce00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:14.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.637+0000 7f0ce8ff9640 1 -- 172.21.15.78:0/4253136402 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cdc053ff0 con 0x7f0cec1045d0 2024-04-25T13:59:14.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.812+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cec062fe0 con 0x7f0cd40afaf0 2024-04-25T13:59:14.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.820+0000 7f0ce8ff9640 1 -- 172.21.15.78:0/4253136402 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259096 (secure 0 0 0) 0x7f0cec062fe0 con 0x7f0cd40afaf0 2024-04-25T13:59:14.823 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T13:59:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 msgr2=0x7f0cd40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0004770 tx=0x7f0ce00314a0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec1045d0 msgr2=0x7f0cec19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec1045d0 0x7f0cec19a2b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00e9c0 tx=0x7f0cdc00ee80 comp rx=0 tx=0).stop 2024-04-25T13:59:14.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 shutdown_connections 2024-04-25T13:59:14.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec109890 0x7f0cec1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cec106f30 0x7f0cec19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 --2- 172.21.15.78:0/4253136402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cec1045d0 0x7f0cec19a2b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 >> 172.21.15.78:0/4253136402 conn(0x7f0cec0fdbd0 msgr2=0x7f0cec1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:14.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 shutdown_connections 2024-04-25T13:59:14.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.823+0000 7f0cf21f3640 1 -- 172.21.15.78:0/4253136402 wait complete. 2024-04-25T13:59:14.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:14 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2615397218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:14.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:14 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/475791378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:14.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:14 smithi078 ceph-mon[29163]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T13:59:14.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:14 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2082016280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:14.834 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-04-25T13:59:14.159924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656879,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":804,"num_read_kb":499,"num_write":295,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953457,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371852,"kb_used_data":10220,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569972,"statfs":{"total":1151940427776,"available":1151559651328,"internally_reserved":0,"allocated":10465280,"data_stored":6790869,"data_compressed":182799,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":36,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003429"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.169510+0000","last_change":"2024-04-25T13:57:12.147384+0000","last_active":"2024-04-25T13:58:38.169510+0000","last_peered":"2024-04-25T13:58:38.169510+0000","last_clean":"2024-04-25T13:58:38.169510+0000","last_became_active":"2024-04-25T13:57:12.146349+0000","last_became_peered":"2024-04-25T13:57:12.146349+0000","last_unstale":"2024-04-25T13:58:38.169510+0000","last_undegraded":"2024-04-25T13:58:38.169510+0000","last_fullsized":"2024-04-25T13:58:38.169510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.151563+0000","last_change":"2024-04-25T13:57:12.156698+0000","last_active":"2024-04-25T13:58:38.151563+0000","last_peered":"2024-04-25T13:58:38.151563+0000","last_clean":"2024-04-25T13:58:38.151563+0000","last_became_active":"2024-04-25T13:57:12.155790+0000","last_became_peered":"2024-04-25T13:57:12.155790+0000","last_unstale":"2024-04-25T13:58:38.151563+0000","last_undegraded":"2024-04-25T13:58:38.151563+0000","last_fullsized":"2024-04-25T13:58:38.151563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:25.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178850+0000","last_change":"2024-04-25T13:57:12.149458+0000","last_active":"2024-04-25T13:58:38.178850+0000","last_peered":"2024-04-25T13:58:38.178850+0000","last_clean":"2024-04-25T13:58:38.178850+0000","last_became_active":"2024-04-25T13:57:12.148179+0000","last_became_peered":"2024-04-25T13:57:12.148179+0000","last_unstale":"2024-04-25T13:58:38.178850+0000","last_undegraded":"2024-04-25T13:58:38.178850+0000","last_fullsized":"2024-04-25T13:58:38.178850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:00.502798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.167668+0000","last_change":"2024-04-25T13:57:12.145776+0000","last_active":"2024-04-25T13:58:38.167668+0000","last_peered":"2024-04-25T13:58:38.167668+0000","last_clean":"2024-04-25T13:58:38.167668+0000","last_became_active":"2024-04-25T13:57:12.145038+0000","last_became_peered":"2024-04-25T13:57:12.145038+0000","last_unstale":"2024-04-25T13:58:38.167668+0000","last_undegraded":"2024-04-25T13:58:38.167668+0000","last_fullsized":"2024-04-25T13:58:38.167668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:52.742746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.148319+0000","last_change":"2024-04-25T13:57:12.160273+0000","last_active":"2024-04-25T13:58:38.148319+0000","last_peered":"2024-04-25T13:58:38.148319+0000","last_clean":"2024-04-25T13:58:38.148319+0000","last_became_active":"2024-04-25T13:57:12.159805+0000","last_became_peered":"2024-04-25T13:57:12.159805+0000","last_unstale":"2024-04-25T13:58:38.148319+0000","last_undegraded":"2024-04-25T13:58:38.148319+0000","last_fullsized":"2024-04-25T13:58:38.148319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:15.044610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158942+0000","last_change":"2024-04-25T13:57:12.151597+0000","last_active":"2024-04-25T13:58:38.158942+0000","last_peered":"2024-04-25T13:58:38.158942+0000","last_clean":"2024-04-25T13:58:38.158942+0000","last_became_active":"2024-04-25T13:57:12.151203+0000","last_became_peered":"2024-04-25T13:57:12.151203+0000","last_unstale":"2024-04-25T13:58:38.158942+0000","last_undegraded":"2024-04-25T13:58:38.158942+0000","last_fullsized":"2024-04-25T13:58:38.158942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.645264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.176553+0000","last_change":"2024-04-25T13:57:12.144959+0000","last_active":"2024-04-25T13:58:38.176553+0000","last_peered":"2024-04-25T13:58:38.176553+0000","last_clean":"2024-04-25T13:58:38.176553+0000","last_became_active":"2024-04-25T13:57:12.144587+0000","last_became_peered":"2024-04-25T13:57:12.144587+0000","last_unstale":"2024-04-25T13:58:38.176553+0000","last_undegraded":"2024-04-25T13:58:38.176553+0000","last_fullsized":"2024-04-25T13:58:38.176553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:45.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.154354+0000","last_change":"2024-04-25T13:57:12.157016+0000","last_active":"2024-04-25T13:58:38.154354+0000","last_peered":"2024-04-25T13:58:38.154354+0000","last_clean":"2024-04-25T13:58:38.154354+0000","last_became_active":"2024-04-25T13:57:12.156209+0000","last_became_peered":"2024-04-25T13:57:12.156209+0000","last_unstale":"2024-04-25T13:58:38.154354+0000","last_undegraded":"2024-04-25T13:58:38.154354+0000","last_fullsized":"2024-04-25T13:58:38.154354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.172499+0000","last_change":"2024-04-25T13:57:12.146696+0000","last_active":"2024-04-25T13:58:38.172499+0000","last_peered":"2024-04-25T13:58:38.172499+0000","last_clean":"2024-04-25T13:58:38.172499+0000","last_became_active":"2024-04-25T13:57:12.146317+0000","last_became_peered":"2024-04-25T13:57:12.146317+0000","last_unstale":"2024-04-25T13:58:38.172499+0000","last_undegraded":"2024-04-25T13:58:38.172499+0000","last_fullsized":"2024-04-25T13:58:38.172499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.078941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164792+0000","last_change":"2024-04-25T13:57:12.155100+0000","last_active":"2024-04-25T13:58:38.164792+0000","last_peered":"2024-04-25T13:58:38.164792+0000","last_clean":"2024-04-25T13:58:38.164792+0000","last_became_active":"2024-04-25T13:57:12.154602+0000","last_became_peered":"2024-04-25T13:57:12.154602+0000","last_unstale":"2024-04-25T13:58:38.164792+0000","last_undegraded":"2024-04-25T13:58:38.164792+0000","last_fullsized":"2024-04-25T13:58:38.164792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.837117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180815+0000","last_change":"2024-04-25T13:57:12.149386+0000","last_active":"2024-04-25T13:58:38.180815+0000","last_peered":"2024-04-25T13:58:38.180815+0000","last_clean":"2024-04-25T13:58:38.180815+0000","last_became_active":"2024-04-25T13:57:12.148710+0000","last_became_peered":"2024-04-25T13:57:12.148710+0000","last_unstale":"2024-04-25T13:58:38.180815+0000","last_undegraded":"2024-04-25T13:58:38.180815+0000","last_fullsized":"2024-04-25T13:58:38.180815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:17.655260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.167445+0000","last_change":"2024-04-25T13:57:12.156103+0000","last_active":"2024-04-25T13:58:38.167445+0000","last_peered":"2024-04-25T13:58:38.167445+0000","last_clean":"2024-04-25T13:58:38.167445+0000","last_became_active":"2024-04-25T13:57:12.155738+0000","last_became_peered":"2024-04-25T13:57:12.155738+0000","last_unstale":"2024-04-25T13:58:38.167445+0000","last_undegraded":"2024-04-25T13:58:38.167445+0000","last_fullsized":"2024-04-25T13:58:38.167445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:38.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160381+0000","last_change":"2024-04-25T13:57:12.154873+0000","last_active":"2024-04-25T13:58:38.160381+0000","last_peered":"2024-04-25T13:58:38.160381+0000","last_clean":"2024-04-25T13:58:38.160381+0000","last_became_active":"2024-04-25T13:57:12.154523+0000","last_became_peered":"2024-04-25T13:57:12.154523+0000","last_unstale":"2024-04-25T13:58:38.160381+0000","last_undegraded":"2024-04-25T13:58:38.160381+0000","last_fullsized":"2024-04-25T13:58:38.160381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:28.376005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164599+0000","last_change":"2024-04-25T13:57:12.156536+0000","last_active":"2024-04-25T13:58:38.164599+0000","last_peered":"2024-04-25T13:58:38.164599+0000","last_clean":"2024-04-25T13:58:38.164599+0000","last_became_active":"2024-04-25T13:57:12.156143+0000","last_became_peered":"2024-04-25T13:57:12.156143+0000","last_unstale":"2024-04-25T13:58:38.164599+0000","last_undegraded":"2024-04-25T13:58:38.164599+0000","last_fullsized":"2024-04-25T13:58:38.164599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:35.210059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163271+0000","last_change":"2024-04-25T13:57:12.144477+0000","last_active":"2024-04-25T13:58:38.163271+0000","last_peered":"2024-04-25T13:58:38.163271+0000","last_clean":"2024-04-25T13:58:38.163271+0000","last_became_active":"2024-04-25T13:57:12.144115+0000","last_became_peered":"2024-04-25T13:57:12.144115+0000","last_unstale":"2024-04-25T13:58:38.163271+0000","last_undegraded":"2024-04-25T13:58:38.163271+0000","last_fullsized":"2024-04-25T13:58:38.163271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:30.733713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179821+0000","last_change":"2024-04-25T13:57:12.148837+0000","last_active":"2024-04-25T13:58:38.179821+0000","last_peered":"2024-04-25T13:58:38.179821+0000","last_clean":"2024-04-25T13:58:38.179821+0000","last_became_active":"2024-04-25T13:57:12.148494+0000","last_became_peered":"2024-04-25T13:57:12.148494+0000","last_unstale":"2024-04-25T13:58:38.179821+0000","last_undegraded":"2024-04-25T13:58:38.179821+0000","last_fullsized":"2024-04-25T13:58:38.179821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:50.414344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.169715+0000","last_change":"2024-04-25T13:57:12.148142+0000","last_active":"2024-04-25T13:58:38.169715+0000","last_peered":"2024-04-25T13:58:38.169715+0000","last_clean":"2024-04-25T13:58:38.169715+0000","last_became_active":"2024-04-25T13:57:12.147738+0000","last_became_peered":"2024-04-25T13:57:12.147738+0000","last_unstale":"2024-04-25T13:58:38.169715+0000","last_undegraded":"2024-04-25T13:58:38.169715+0000","last_fullsized":"2024-04-25T13:58:38.169715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:42.014605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.174951+0000","last_change":"2024-04-25T13:57:12.160059+0000","last_active":"2024-04-25T13:58:38.174951+0000","last_peered":"2024-04-25T13:58:38.174951+0000","last_clean":"2024-04-25T13:58:38.174951+0000","last_became_active":"2024-04-25T13:57:12.159685+0000","last_became_peered":"2024-04-25T13:57:12.159685+0000","last_unstale":"2024-04-25T13:58:38.174951+0000","last_undegraded":"2024-04-25T13:58:38.174951+0000","last_fullsized":"2024-04-25T13:58:38.174951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:48.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.153349+0000","last_change":"2024-04-25T13:57:12.157950+0000","last_active":"2024-04-25T13:58:38.153349+0000","last_peered":"2024-04-25T13:58:38.153349+0000","last_clean":"2024-04-25T13:58:38.153349+0000","last_became_active":"2024-04-25T13:57:12.157612+0000","last_became_peered":"2024-04-25T13:57:12.157612+0000","last_unstale":"2024-04-25T13:58:38.153349+0000","last_undegraded":"2024-04-25T13:58:38.153349+0000","last_fullsized":"2024-04-25T13:58:38.153349+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:55.521381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.222728+0000","last_change":"2024-04-25T13:58:09.222728+0000","last_active":"2024-04-25T13:58:09.222728+0000","last_peered":"2024-04-25T13:58:09.222728+0000","last_clean":"2024-04-25T13:58:09.222728+0000","last_became_active":"2024-04-25T13:58:09.210819+0000","last_became_peered":"2024-04-25T13:58:09.210819+0000","last_unstale":"2024-04-25T13:58:09.222728+0000","last_undegraded":"2024-04-25T13:58:09.222728+0000","last_fullsized":"2024-04-25T13:58:09.222728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:00.606838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155336+0000","last_change":"2024-04-25T13:57:12.151301+0000","last_active":"2024-04-25T13:58:38.155336+0000","last_peered":"2024-04-25T13:58:38.155336+0000","last_clean":"2024-04-25T13:58:38.155336+0000","last_became_active":"2024-04-25T13:57:12.150941+0000","last_became_peered":"2024-04-25T13:57:12.150941+0000","last_unstale":"2024-04-25T13:58:38.155336+0000","last_undegraded":"2024-04-25T13:58:38.155336+0000","last_fullsized":"2024-04-25T13:58:38.155336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:22.259683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213379+0000","last_change":"2024-04-25T13:57:11.147314+0000","last_active":"2024-04-25T13:58:09.213379+0000","last_peered":"2024-04-25T13:58:09.213379+0000","last_clean":"2024-04-25T13:58:09.213379+0000","last_became_active":"2024-04-25T13:57:11.146894+0000","last_became_peered":"2024-04-25T13:57:11.146894+0000","last_unstale":"2024-04-25T13:58:09.213379+0000","last_undegraded":"2024-04-25T13:58:09.213379+0000","last_fullsized":"2024-04-25T13:58:09.213379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:55.927007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178922+0000","last_change":"2024-04-25T13:57:12.149186+0000","last_active":"2024-04-25T13:58:38.178922+0000","last_peered":"2024-04-25T13:58:38.178922+0000","last_clean":"2024-04-25T13:58:38.178922+0000","last_became_active":"2024-04-25T13:57:12.148155+0000","last_became_peered":"2024-04-25T13:57:12.148155+0000","last_unstale":"2024-04-25T13:58:38.178922+0000","last_undegraded":"2024-04-25T13:58:38.178922+0000","last_fullsized":"2024-04-25T13:58:38.178922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209681+0000","last_change":"2024-04-25T13:57:11.140086+0000","last_active":"2024-04-25T13:58:09.209681+0000","last_peered":"2024-04-25T13:58:09.209681+0000","last_clean":"2024-04-25T13:58:09.209681+0000","last_became_active":"2024-04-25T13:57:11.139751+0000","last_became_peered":"2024-04-25T13:57:11.139751+0000","last_unstale":"2024-04-25T13:58:09.209681+0000","last_undegraded":"2024-04-25T13:58:09.209681+0000","last_fullsized":"2024-04-25T13:58:09.209681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.552503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179369+0000","last_change":"2024-04-25T13:57:12.157029+0000","last_active":"2024-04-25T13:58:38.179369+0000","last_peered":"2024-04-25T13:58:38.179369+0000","last_clean":"2024-04-25T13:58:38.179369+0000","last_became_active":"2024-04-25T13:57:12.156668+0000","last_became_peered":"2024-04-25T13:57:12.156668+0000","last_unstale":"2024-04-25T13:58:38.179369+0000","last_undegraded":"2024-04-25T13:58:38.179369+0000","last_fullsized":"2024-04-25T13:58:38.179369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208828+0000","last_change":"2024-04-25T13:57:11.140592+0000","last_active":"2024-04-25T13:58:09.208828+0000","last_peered":"2024-04-25T13:58:09.208828+0000","last_clean":"2024-04-25T13:58:09.208828+0000","last_became_active":"2024-04-25T13:57:11.140244+0000","last_became_peered":"2024-04-25T13:57:11.140244+0000","last_unstale":"2024-04-25T13:58:09.208828+0000","last_undegraded":"2024-04-25T13:58:09.208828+0000","last_fullsized":"2024-04-25T13:58:09.208828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:53.071092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.174853+0000","last_change":"2024-04-25T13:58:09.326284+0000","last_active":"2024-04-25T13:58:38.174853+0000","last_peered":"2024-04-25T13:58:38.174853+0000","last_clean":"2024-04-25T13:58:38.174853+0000","last_became_active":"2024-04-25T13:58:09.325888+0000","last_became_peered":"2024-04-25T13:58:09.325888+0000","last_unstale":"2024-04-25T13:58:38.174853+0000","last_undegraded":"2024-04-25T13:58:38.174853+0000","last_fullsized":"2024-04-25T13:58:38.174853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:39.151793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":160,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.323118+0000","last_change":"2024-04-25T13:58:09.221709+0000","last_active":"2024-04-25T13:59:11.323118+0000","last_peered":"2024-04-25T13:59:11.323118+0000","last_clean":"2024-04-25T13:59:11.323118+0000","last_became_active":"2024-04-25T13:58:09.215262+0000","last_became_peered":"2024-04-25T13:58:09.215262+0000","last_unstale":"2024-04-25T13:59:11.323118+0000","last_undegraded":"2024-04-25T13:59:11.323118+0000","last_fullsized":"2024-04-25T13:59:11.323118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:05.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155581+0000","last_change":"2024-04-25T13:57:12.148787+0000","last_active":"2024-04-25T13:58:38.155581+0000","last_peered":"2024-04-25T13:58:38.155581+0000","last_clean":"2024-04-25T13:58:38.155581+0000","last_became_active":"2024-04-25T13:57:12.148424+0000","last_became_peered":"2024-04-25T13:57:12.148424+0000","last_unstale":"2024-04-25T13:58:38.155581+0000","last_undegraded":"2024-04-25T13:58:38.155581+0000","last_fullsized":"2024-04-25T13:58:38.155581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:11.921873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324717+0000","last_change":"2024-04-25T13:57:11.142424+0000","last_active":"2024-04-25T13:58:09.324717+0000","last_peered":"2024-04-25T13:58:09.324717+0000","last_clean":"2024-04-25T13:58:09.324717+0000","last_became_active":"2024-04-25T13:57:11.142047+0000","last_became_peered":"2024-04-25T13:57:11.142047+0000","last_unstale":"2024-04-25T13:58:09.324717+0000","last_undegraded":"2024-04-25T13:58:09.324717+0000","last_fullsized":"2024-04-25T13:58:09.324717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:13.455149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163379+0000","last_change":"2024-04-25T13:57:12.155940+0000","last_active":"2024-04-25T13:58:38.163379+0000","last_peered":"2024-04-25T13:58:38.163379+0000","last_clean":"2024-04-25T13:58:38.163379+0000","last_became_active":"2024-04-25T13:57:12.155503+0000","last_became_peered":"2024-04-25T13:57:12.155503+0000","last_unstale":"2024-04-25T13:58:38.163379+0000","last_undegraded":"2024-04-25T13:58:38.163379+0000","last_fullsized":"2024-04-25T13:58:38.163379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:48.345437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209853+0000","last_change":"2024-04-25T13:57:11.131160+0000","last_active":"2024-04-25T13:58:09.209853+0000","last_peered":"2024-04-25T13:58:09.209853+0000","last_clean":"2024-04-25T13:58:09.209853+0000","last_became_active":"2024-04-25T13:57:11.130753+0000","last_became_peered":"2024-04-25T13:57:11.130753+0000","last_unstale":"2024-04-25T13:58:09.209853+0000","last_undegraded":"2024-04-25T13:58:09.209853+0000","last_fullsized":"2024-04-25T13:58:09.209853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:48.297546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.156739+0000","last_change":"2024-04-25T13:57:12.155779+0000","last_active":"2024-04-25T13:58:38.156739+0000","last_peered":"2024-04-25T13:58:38.156739+0000","last_clean":"2024-04-25T13:58:38.156739+0000","last_became_active":"2024-04-25T13:57:12.154682+0000","last_became_peered":"2024-04-25T13:57:12.154682+0000","last_unstale":"2024-04-25T13:58:38.156739+0000","last_undegraded":"2024-04-25T13:58:38.156739+0000","last_fullsized":"2024-04-25T13:58:38.156739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:20.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211839+0000","last_change":"2024-04-25T13:57:11.140211+0000","last_active":"2024-04-25T13:58:09.211839+0000","last_peered":"2024-04-25T13:58:09.211839+0000","last_clean":"2024-04-25T13:58:09.211839+0000","last_became_active":"2024-04-25T13:57:11.139891+0000","last_became_peered":"2024-04-25T13:57:11.139891+0000","last_unstale":"2024-04-25T13:58:09.211839+0000","last_undegraded":"2024-04-25T13:58:09.211839+0000","last_fullsized":"2024-04-25T13:58:09.211839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:21.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.159292+0000","last_change":"2024-04-25T13:57:12.153886+0000","last_active":"2024-04-25T13:58:38.159292+0000","last_peered":"2024-04-25T13:58:38.159292+0000","last_clean":"2024-04-25T13:58:38.159292+0000","last_became_active":"2024-04-25T13:57:12.153500+0000","last_became_peered":"2024-04-25T13:57:12.153500+0000","last_unstale":"2024-04-25T13:58:38.159292+0000","last_undegraded":"2024-04-25T13:58:38.159292+0000","last_fullsized":"2024-04-25T13:58:38.159292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:53.549745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:09.318464+0000","last_change":"2024-04-25T13:57:11.138829+0000","last_active":"2024-04-25T13:59:09.318464+0000","last_peered":"2024-04-25T13:59:09.318464+0000","last_clean":"2024-04-25T13:59:09.318464+0000","last_became_active":"2024-04-25T13:57:11.138461+0000","last_became_peered":"2024-04-25T13:57:11.138461+0000","last_unstale":"2024-04-25T13:59:09.318464+0000","last_undegraded":"2024-04-25T13:59:09.318464+0000","last_fullsized":"2024-04-25T13:59:09.318464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.134061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177713+0000","last_change":"2024-04-25T13:57:12.157394+0000","last_active":"2024-04-25T13:58:38.177713+0000","last_peered":"2024-04-25T13:58:38.177713+0000","last_clean":"2024-04-25T13:58:38.177713+0000","last_became_active":"2024-04-25T13:57:12.156408+0000","last_became_peered":"2024-04-25T13:57:12.156408+0000","last_unstale":"2024-04-25T13:58:38.177713+0000","last_undegraded":"2024-04-25T13:58:38.177713+0000","last_fullsized":"2024-04-25T13:58:38.177713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:36.741367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208928+0000","last_change":"2024-04-25T13:57:11.142846+0000","last_active":"2024-04-25T13:58:09.208928+0000","last_peered":"2024-04-25T13:58:09.208928+0000","last_clean":"2024-04-25T13:58:09.208928+0000","last_became_active":"2024-04-25T13:57:11.142387+0000","last_became_peered":"2024-04-25T13:57:11.142387+0000","last_unstale":"2024-04-25T13:58:09.208928+0000","last_undegraded":"2024-04-25T13:58:09.208928+0000","last_fullsized":"2024-04-25T13:58:09.208928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.164994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163650+0000","last_change":"2024-04-25T13:57:12.145617+0000","last_active":"2024-04-25T13:58:38.163650+0000","last_peered":"2024-04-25T13:58:38.163650+0000","last_clean":"2024-04-25T13:58:38.163650+0000","last_became_active":"2024-04-25T13:57:12.145263+0000","last_became_peered":"2024-04-25T13:57:12.145263+0000","last_unstale":"2024-04-25T13:58:38.163650+0000","last_undegraded":"2024-04-25T13:58:38.163650+0000","last_fullsized":"2024-04-25T13:58:38.163650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.886538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212101+0000","last_change":"2024-04-25T13:57:11.136972+0000","last_active":"2024-04-25T13:58:09.212101+0000","last_peered":"2024-04-25T13:58:09.212101+0000","last_clean":"2024-04-25T13:58:09.212101+0000","last_became_active":"2024-04-25T13:57:11.136392+0000","last_became_peered":"2024-04-25T13:57:11.136392+0000","last_unstale":"2024-04-25T13:58:09.212101+0000","last_undegraded":"2024-04-25T13:58:09.212101+0000","last_fullsized":"2024-04-25T13:58:09.212101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:45.841670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177590+0000","last_change":"2024-04-25T13:57:12.144905+0000","last_active":"2024-04-25T13:58:38.177590+0000","last_peered":"2024-04-25T13:58:38.177590+0000","last_clean":"2024-04-25T13:58:38.177590+0000","last_became_active":"2024-04-25T13:57:12.144404+0000","last_became_peered":"2024-04-25T13:57:12.144404+0000","last_unstale":"2024-04-25T13:58:38.177590+0000","last_undegraded":"2024-04-25T13:58:38.177590+0000","last_fullsized":"2024-04-25T13:58:38.177590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.186492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'31","reported_seq":291,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:10.306989+0000","last_change":"2024-04-25T13:57:11.139041+0000","last_active":"2024-04-25T13:59:10.306989+0000","last_peered":"2024-04-25T13:59:10.306989+0000","last_clean":"2024-04-25T13:59:10.306989+0000","last_became_active":"2024-04-25T13:57:11.138629+0000","last_became_peered":"2024-04-25T13:57:11.138629+0000","last_unstale":"2024-04-25T13:59:10.306989+0000","last_undegraded":"2024-04-25T13:59:10.306989+0000","last_fullsized":"2024-04-25T13:59:10.306989+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:04.863015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":180,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177642+0000","last_change":"2024-04-25T13:57:12.160897+0000","last_active":"2024-04-25T13:58:38.177642+0000","last_peered":"2024-04-25T13:58:38.177642+0000","last_clean":"2024-04-25T13:58:38.177642+0000","last_became_active":"2024-04-25T13:57:12.160322+0000","last_became_peered":"2024-04-25T13:57:12.160322+0000","last_unstale":"2024-04-25T13:58:38.177642+0000","last_undegraded":"2024-04-25T13:58:38.177642+0000","last_fullsized":"2024-04-25T13:58:38.177642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:26.180324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216914+0000","last_change":"2024-04-25T13:57:11.137104+0000","last_active":"2024-04-25T13:58:09.216914+0000","last_peered":"2024-04-25T13:58:09.216914+0000","last_clean":"2024-04-25T13:58:09.216914+0000","last_became_active":"2024-04-25T13:57:11.136720+0000","last_became_peered":"2024-04-25T13:57:11.136720+0000","last_unstale":"2024-04-25T13:58:09.216914+0000","last_undegraded":"2024-04-25T13:58:09.216914+0000","last_fullsized":"2024-04-25T13:58:09.216914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:41.262776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177140+0000","last_change":"2024-04-25T13:57:12.147831+0000","last_active":"2024-04-25T13:58:38.177140+0000","last_peered":"2024-04-25T13:58:38.177140+0000","last_clean":"2024-04-25T13:58:38.177140+0000","last_became_active":"2024-04-25T13:57:12.147270+0000","last_became_peered":"2024-04-25T13:57:12.147270+0000","last_unstale":"2024-04-25T13:58:38.177140+0000","last_undegraded":"2024-04-25T13:58:38.177140+0000","last_fullsized":"2024-04-25T13:58:38.177140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:31.574661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211663+0000","last_change":"2024-04-25T13:57:11.138254+0000","last_active":"2024-04-25T13:58:09.211663+0000","last_peered":"2024-04-25T13:58:09.211663+0000","last_clean":"2024-04-25T13:58:09.211663+0000","last_became_active":"2024-04-25T13:57:11.137924+0000","last_became_peered":"2024-04-25T13:57:11.137924+0000","last_unstale":"2024-04-25T13:58:09.211663+0000","last_undegraded":"2024-04-25T13:58:09.211663+0000","last_fullsized":"2024-04-25T13:58:09.211663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:27.746203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179497+0000","last_change":"2024-04-25T13:57:12.145743+0000","last_active":"2024-04-25T13:58:38.179497+0000","last_peered":"2024-04-25T13:58:38.179497+0000","last_clean":"2024-04-25T13:58:38.179497+0000","last_became_active":"2024-04-25T13:57:12.145337+0000","last_became_peered":"2024-04-25T13:57:12.145337+0000","last_unstale":"2024-04-25T13:58:38.179497+0000","last_undegraded":"2024-04-25T13:58:38.179497+0000","last_fullsized":"2024-04-25T13:58:38.179497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.692900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'6","reported_seq":146,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.269043+0000","last_change":"2024-04-25T13:57:11.138059+0000","last_active":"2024-04-25T13:59:11.269043+0000","last_peered":"2024-04-25T13:59:11.269043+0000","last_clean":"2024-04-25T13:59:11.269043+0000","last_became_active":"2024-04-25T13:57:11.137698+0000","last_became_peered":"2024-04-25T13:57:11.137698+0000","last_unstale":"2024-04-25T13:59:11.269043+0000","last_undegraded":"2024-04-25T13:59:11.269043+0000","last_fullsized":"2024-04-25T13:59:11.269043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:03.595126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.171572+0000","last_change":"2024-04-25T13:57:12.155507+0000","last_active":"2024-04-25T13:58:38.171572+0000","last_peered":"2024-04-25T13:58:38.171572+0000","last_clean":"2024-04-25T13:58:38.171572+0000","last_became_active":"2024-04-25T13:57:12.155175+0000","last_became_peered":"2024-04-25T13:57:12.155175+0000","last_unstale":"2024-04-25T13:58:38.171572+0000","last_undegraded":"2024-04-25T13:58:38.171572+0000","last_fullsized":"2024-04-25T13:58:38.171572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:21.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'15","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208247+0000","last_change":"2024-04-25T13:57:11.141672+0000","last_active":"2024-04-25T13:58:09.208247+0000","last_peered":"2024-04-25T13:58:09.208247+0000","last_clean":"2024-04-25T13:58:09.208247+0000","last_became_active":"2024-04-25T13:57:11.140957+0000","last_became_peered":"2024-04-25T13:57:11.140957+0000","last_unstale":"2024-04-25T13:58:09.208247+0000","last_undegraded":"2024-04-25T13:58:09.208247+0000","last_fullsized":"2024-04-25T13:58:09.208247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:47.362949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.165834+0000","last_change":"2024-04-25T13:57:12.147881+0000","last_active":"2024-04-25T13:58:38.165834+0000","last_peered":"2024-04-25T13:58:38.165834+0000","last_clean":"2024-04-25T13:58:38.165834+0000","last_became_active":"2024-04-25T13:57:12.146200+0000","last_became_peered":"2024-04-25T13:57:12.146200+0000","last_unstale":"2024-04-25T13:58:38.165834+0000","last_undegraded":"2024-04-25T13:58:38.165834+0000","last_fullsized":"2024-04-25T13:58:38.165834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:51.711045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209588+0000","last_change":"2024-04-25T13:57:11.139574+0000","last_active":"2024-04-25T13:58:09.209588+0000","last_peered":"2024-04-25T13:58:09.209588+0000","last_clean":"2024-04-25T13:58:09.209588+0000","last_became_active":"2024-04-25T13:57:11.139232+0000","last_became_peered":"2024-04-25T13:57:11.139232+0000","last_unstale":"2024-04-25T13:58:09.209588+0000","last_undegraded":"2024-04-25T13:58:09.209588+0000","last_fullsized":"2024-04-25T13:58:09.209588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:08.590270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.162405+0000","last_change":"2024-04-25T13:57:12.147122+0000","last_active":"2024-04-25T13:58:38.162405+0000","last_peered":"2024-04-25T13:58:38.162405+0000","last_clean":"2024-04-25T13:58:38.162405+0000","last_became_active":"2024-04-25T13:57:12.146708+0000","last_became_peered":"2024-04-25T13:57:12.146708+0000","last_unstale":"2024-04-25T13:58:38.162405+0000","last_undegraded":"2024-04-25T13:58:38.162405+0000","last_fullsized":"2024-04-25T13:58:38.162405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:47.858290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.205903+0000","last_change":"2024-04-25T13:57:11.146288+0000","last_active":"2024-04-25T13:58:09.205903+0000","last_peered":"2024-04-25T13:58:09.205903+0000","last_clean":"2024-04-25T13:58:09.205903+0000","last_became_active":"2024-04-25T13:57:11.145851+0000","last_became_peered":"2024-04-25T13:57:11.145851+0000","last_unstale":"2024-04-25T13:58:09.205903+0000","last_undegraded":"2024-04-25T13:58:09.205903+0000","last_fullsized":"2024-04-25T13:58:09.205903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155416+0000","last_change":"2024-04-25T13:57:12.154576+0000","last_active":"2024-04-25T13:58:38.155416+0000","last_peered":"2024-04-25T13:58:38.155416+0000","last_clean":"2024-04-25T13:58:38.155416+0000","last_became_active":"2024-04-25T13:57:12.154254+0000","last_became_peered":"2024-04-25T13:57:12.154254+0000","last_unstale":"2024-04-25T13:58:38.155416+0000","last_undegraded":"2024-04-25T13:58:38.155416+0000","last_fullsized":"2024-04-25T13:58:38.155416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:33.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208334+0000","last_change":"2024-04-25T13:57:11.139941+0000","last_active":"2024-04-25T13:58:09.208334+0000","last_peered":"2024-04-25T13:58:09.208334+0000","last_clean":"2024-04-25T13:58:09.208334+0000","last_became_active":"2024-04-25T13:57:11.139614+0000","last_became_peered":"2024-04-25T13:57:11.139614+0000","last_unstale":"2024-04-25T13:58:09.208334+0000","last_undegraded":"2024-04-25T13:58:09.208334+0000","last_fullsized":"2024-04-25T13:58:09.208334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:39.656085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164620+0000","last_change":"2024-04-25T13:57:12.155935+0000","last_active":"2024-04-25T13:58:38.164620+0000","last_peered":"2024-04-25T13:58:38.164620+0000","last_clean":"2024-04-25T13:58:38.164620+0000","last_became_active":"2024-04-25T13:57:12.155261+0000","last_became_peered":"2024-04-25T13:57:12.155261+0000","last_unstale":"2024-04-25T13:58:38.164620+0000","last_undegraded":"2024-04-25T13:58:38.164620+0000","last_fullsized":"2024-04-25T13:58:38.164620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:54.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.156931+0000","last_change":"2024-04-25T13:58:09.217792+0000","last_active":"2024-04-25T13:58:38.156931+0000","last_peered":"2024-04-25T13:58:38.156931+0000","last_clean":"2024-04-25T13:58:38.156931+0000","last_became_active":"2024-04-25T13:58:09.217386+0000","last_became_peered":"2024-04-25T13:58:09.217386+0000","last_unstale":"2024-04-25T13:58:38.156931+0000","last_undegraded":"2024-04-25T13:58:38.156931+0000","last_fullsized":"2024-04-25T13:58:38.156931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:13.833587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.268314+0000","last_change":"2024-04-25T13:57:11.135226+0000","last_active":"2024-04-25T13:59:11.268314+0000","last_peered":"2024-04-25T13:59:11.268314+0000","last_clean":"2024-04-25T13:59:11.268314+0000","last_became_active":"2024-04-25T13:57:11.134886+0000","last_became_peered":"2024-04-25T13:57:11.134886+0000","last_unstale":"2024-04-25T13:59:11.268314+0000","last_undegraded":"2024-04-25T13:59:11.268314+0000","last_fullsized":"2024-04-25T13:59:11.268314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:00.153597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.173674+0000","last_change":"2024-04-25T13:57:12.149758+0000","last_active":"2024-04-25T13:58:38.173674+0000","last_peered":"2024-04-25T13:58:38.173674+0000","last_clean":"2024-04-25T13:58:38.173674+0000","last_became_active":"2024-04-25T13:57:12.149428+0000","last_became_peered":"2024-04-25T13:57:12.149428+0000","last_unstale":"2024-04-25T13:58:38.173674+0000","last_undegraded":"2024-04-25T13:58:38.173674+0000","last_fullsized":"2024-04-25T13:58:38.173674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.739074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207722+0000","last_change":"2024-04-25T13:57:11.135968+0000","last_active":"2024-04-25T13:58:09.207722+0000","last_peered":"2024-04-25T13:58:09.207722+0000","last_clean":"2024-04-25T13:58:09.207722+0000","last_became_active":"2024-04-25T13:57:11.135577+0000","last_became_peered":"2024-04-25T13:57:11.135577+0000","last_unstale":"2024-04-25T13:58:09.207722+0000","last_undegraded":"2024-04-25T13:58:09.207722+0000","last_fullsized":"2024-04-25T13:58:09.207722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:00.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208486+0000","last_change":"2024-04-25T13:55:57.666399+0000","last_active":"2024-04-25T13:58:09.208486+0000","last_peered":"2024-04-25T13:58:09.208486+0000","last_clean":"2024-04-25T13:58:09.208486+0000","last_became_active":"2024-04-25T13:55:57.660470+0000","last_became_peered":"2024-04-25T13:55:57.660470+0000","last_unstale":"2024-04-25T13:58:09.208486+0000","last_undegraded":"2024-04-25T13:58:09.208486+0000","last_fullsized":"2024-04-25T13:58:09.208486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_clean_scrub_stamp":"2024-04-25T13:52:33.081441+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:22.788632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163242+0000","last_change":"2024-04-25T13:57:12.141923+0000","last_active":"2024-04-25T13:58:38.163242+0000","last_peered":"2024-04-25T13:58:38.163242+0000","last_clean":"2024-04-25T13:58:38.163242+0000","last_became_active":"2024-04-25T13:57:12.140946+0000","last_became_peered":"2024-04-25T13:57:12.140946+0000","last_unstale":"2024-04-25T13:58:38.163242+0000","last_undegraded":"2024-04-25T13:58:38.163242+0000","last_fullsized":"2024-04-25T13:58:38.163242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:49.294598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208387+0000","last_change":"2024-04-25T13:57:11.139851+0000","last_active":"2024-04-25T13:58:09.208387+0000","last_peered":"2024-04-25T13:58:09.208387+0000","last_clean":"2024-04-25T13:58:09.208387+0000","last_became_active":"2024-04-25T13:57:11.139476+0000","last_became_peered":"2024-04-25T13:57:11.139476+0000","last_unstale":"2024-04-25T13:58:09.208387+0000","last_undegraded":"2024-04-25T13:58:09.208387+0000","last_fullsized":"2024-04-25T13:58:09.208387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:29.435917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.220107+0000","last_change":"2024-04-25T13:58:09.219619+0000","last_active":"2024-04-25T13:58:09.220107+0000","last_peered":"2024-04-25T13:58:09.220107+0000","last_clean":"2024-04-25T13:58:09.220107+0000","last_became_active":"2024-04-25T13:58:09.211808+0000","last_became_peered":"2024-04-25T13:58:09.211808+0000","last_unstale":"2024-04-25T13:58:09.220107+0000","last_undegraded":"2024-04-25T13:58:09.220107+0000","last_fullsized":"2024-04-25T13:58:09.220107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:20.208191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.157529+0000","last_change":"2024-04-25T13:57:12.136204+0000","last_active":"2024-04-25T13:58:38.157529+0000","last_peered":"2024-04-25T13:58:38.157529+0000","last_clean":"2024-04-25T13:58:38.157529+0000","last_became_active":"2024-04-25T13:57:12.135798+0000","last_became_peered":"2024-04-25T13:57:12.135798+0000","last_unstale":"2024-04-25T13:58:38.157529+0000","last_undegraded":"2024-04-25T13:58:38.157529+0000","last_fullsized":"2024-04-25T13:58:38.157529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:36.894774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.206151+0000","last_change":"2024-04-25T13:57:11.138699+0000","last_active":"2024-04-25T13:58:09.206151+0000","last_peered":"2024-04-25T13:58:09.206151+0000","last_clean":"2024-04-25T13:58:09.206151+0000","last_became_active":"2024-04-25T13:57:11.138335+0000","last_became_peered":"2024-04-25T13:57:11.138335+0000","last_unstale":"2024-04-25T13:58:09.206151+0000","last_undegraded":"2024-04-25T13:58:09.206151+0000","last_fullsized":"2024-04-25T13:58:09.206151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:44.413228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.171184+0000","last_change":"2024-04-25T13:57:12.155851+0000","last_active":"2024-04-25T13:58:38.171184+0000","last_peered":"2024-04-25T13:58:38.171184+0000","last_clean":"2024-04-25T13:58:38.171184+0000","last_became_active":"2024-04-25T13:57:12.155416+0000","last_became_peered":"2024-04-25T13:57:12.155416+0000","last_unstale":"2024-04-25T13:58:38.171184+0000","last_undegraded":"2024-04-25T13:58:38.171184+0000","last_fullsized":"2024-04-25T13:58:38.171184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:22.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207744+0000","last_change":"2024-04-25T13:57:11.147465+0000","last_active":"2024-04-25T13:58:09.207744+0000","last_peered":"2024-04-25T13:58:09.207744+0000","last_clean":"2024-04-25T13:58:09.207744+0000","last_became_active":"2024-04-25T13:57:11.147096+0000","last_became_peered":"2024-04-25T13:57:11.147096+0000","last_unstale":"2024-04-25T13:58:09.207744+0000","last_undegraded":"2024-04-25T13:58:09.207744+0000","last_fullsized":"2024-04-25T13:58:09.207744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.229356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163274+0000","last_change":"2024-04-25T13:57:12.146446+0000","last_active":"2024-04-25T13:58:38.163274+0000","last_peered":"2024-04-25T13:58:38.163274+0000","last_clean":"2024-04-25T13:58:38.163274+0000","last_became_active":"2024-04-25T13:57:12.146129+0000","last_became_peered":"2024-04-25T13:57:12.146129+0000","last_unstale":"2024-04-25T13:58:38.163274+0000","last_undegraded":"2024-04-25T13:58:38.163274+0000","last_fullsized":"2024-04-25T13:58:38.163274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.977663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325014+0000","last_change":"2024-04-25T13:57:11.145127+0000","last_active":"2024-04-25T13:58:09.325014+0000","last_peered":"2024-04-25T13:58:09.325014+0000","last_clean":"2024-04-25T13:58:09.325014+0000","last_became_active":"2024-04-25T13:57:11.144822+0000","last_became_peered":"2024-04-25T13:57:11.144822+0000","last_unstale":"2024-04-25T13:58:09.325014+0000","last_undegraded":"2024-04-25T13:58:09.325014+0000","last_fullsized":"2024-04-25T13:58:09.325014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:00.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.121092+0000","last_change":"2024-04-25T13:57:12.156184+0000","last_active":"2024-04-25T13:58:38.121092+0000","last_peered":"2024-04-25T13:58:38.121092+0000","last_clean":"2024-04-25T13:58:38.121092+0000","last_became_active":"2024-04-25T13:57:12.155855+0000","last_became_peered":"2024-04-25T13:57:12.155855+0000","last_unstale":"2024-04-25T13:58:38.121092+0000","last_undegraded":"2024-04-25T13:58:38.121092+0000","last_fullsized":"2024-04-25T13:58:38.121092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213952+0000","last_change":"2024-04-25T13:57:11.137616+0000","last_active":"2024-04-25T13:58:09.213952+0000","last_peered":"2024-04-25T13:58:09.213952+0000","last_clean":"2024-04-25T13:58:09.213952+0000","last_became_active":"2024-04-25T13:57:11.137041+0000","last_became_peered":"2024-04-25T13:57:11.137041+0000","last_unstale":"2024-04-25T13:58:09.213952+0000","last_undegraded":"2024-04-25T13:58:09.213952+0000","last_fullsized":"2024-04-25T13:58:09.213952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:54.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.159060+0000","last_change":"2024-04-25T13:57:12.151223+0000","last_active":"2024-04-25T13:58:38.159060+0000","last_peered":"2024-04-25T13:58:38.159060+0000","last_clean":"2024-04-25T13:58:38.159060+0000","last_became_active":"2024-04-25T13:57:12.150871+0000","last_became_peered":"2024-04-25T13:57:12.150871+0000","last_unstale":"2024-04-25T13:58:38.159060+0000","last_undegraded":"2024-04-25T13:58:38.159060+0000","last_fullsized":"2024-04-25T13:58:38.159060+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:17.957728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216414+0000","last_change":"2024-04-25T13:57:11.136524+0000","last_active":"2024-04-25T13:58:09.216414+0000","last_peered":"2024-04-25T13:58:09.216414+0000","last_clean":"2024-04-25T13:58:09.216414+0000","last_became_active":"2024-04-25T13:57:11.136152+0000","last_became_peered":"2024-04-25T13:57:11.136152+0000","last_unstale":"2024-04-25T13:58:09.216414+0000","last_undegraded":"2024-04-25T13:58:09.216414+0000","last_fullsized":"2024-04-25T13:58:09.216414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:23.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160886+0000","last_change":"2024-04-25T13:57:12.157641+0000","last_active":"2024-04-25T13:58:38.160886+0000","last_peered":"2024-04-25T13:58:38.160886+0000","last_clean":"2024-04-25T13:58:38.160886+0000","last_became_active":"2024-04-25T13:57:12.157281+0000","last_became_peered":"2024-04-25T13:57:12.157281+0000","last_unstale":"2024-04-25T13:58:38.160886+0000","last_undegraded":"2024-04-25T13:58:38.160886+0000","last_fullsized":"2024-04-25T13:58:38.160886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:28.245071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208565+0000","last_change":"2024-04-25T13:57:11.139036+0000","last_active":"2024-04-25T13:58:09.208565+0000","last_peered":"2024-04-25T13:58:09.208565+0000","last_clean":"2024-04-25T13:58:09.208565+0000","last_became_active":"2024-04-25T13:57:11.138681+0000","last_became_peered":"2024-04-25T13:57:11.138681+0000","last_unstale":"2024-04-25T13:58:09.208565+0000","last_undegraded":"2024-04-25T13:58:09.208565+0000","last_fullsized":"2024-04-25T13:58:09.208565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:48.584313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180469+0000","last_change":"2024-04-25T13:57:12.153072+0000","last_active":"2024-04-25T13:58:38.180469+0000","last_peered":"2024-04-25T13:58:38.180469+0000","last_clean":"2024-04-25T13:58:38.180469+0000","last_became_active":"2024-04-25T13:57:12.151556+0000","last_became_peered":"2024-04-25T13:57:12.151556+0000","last_unstale":"2024-04-25T13:58:38.180469+0000","last_undegraded":"2024-04-25T13:58:38.180469+0000","last_fullsized":"2024-04-25T13:58:38.180469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:20.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217008+0000","last_change":"2024-04-25T13:57:11.148823+0000","last_active":"2024-04-25T13:58:09.217008+0000","last_peered":"2024-04-25T13:58:09.217008+0000","last_clean":"2024-04-25T13:58:09.217008+0000","last_became_active":"2024-04-25T13:57:11.148436+0000","last_became_peered":"2024-04-25T13:57:11.148436+0000","last_unstale":"2024-04-25T13:58:09.217008+0000","last_undegraded":"2024-04-25T13:58:09.217008+0000","last_fullsized":"2024-04-25T13:58:09.217008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:28.496184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180714+0000","last_change":"2024-04-25T13:57:12.149082+0000","last_active":"2024-04-25T13:58:38.180714+0000","last_peered":"2024-04-25T13:58:38.180714+0000","last_clean":"2024-04-25T13:58:38.180714+0000","last_became_active":"2024-04-25T13:57:12.148260+0000","last_became_peered":"2024-04-25T13:57:12.148260+0000","last_unstale":"2024-04-25T13:58:38.180714+0000","last_undegraded":"2024-04-25T13:58:38.180714+0000","last_fullsized":"2024-04-25T13:58:38.180714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:49.733114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211832+0000","last_change":"2024-04-25T13:57:11.136736+0000","last_active":"2024-04-25T13:58:09.211832+0000","last_peered":"2024-04-25T13:58:09.211832+0000","last_clean":"2024-04-25T13:58:09.211832+0000","last_became_active":"2024-04-25T13:57:11.135831+0000","last_became_peered":"2024-04-25T13:57:11.135831+0000","last_unstale":"2024-04-25T13:58:09.211832+0000","last_undegraded":"2024-04-25T13:58:09.211832+0000","last_fullsized":"2024-04-25T13:58:09.211832+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:49.654642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.175566+0000","last_change":"2024-04-25T13:57:12.149797+0000","last_active":"2024-04-25T13:58:38.175566+0000","last_peered":"2024-04-25T13:58:38.175566+0000","last_clean":"2024-04-25T13:58:38.175566+0000","last_became_active":"2024-04-25T13:57:12.149413+0000","last_became_peered":"2024-04-25T13:57:12.149413+0000","last_unstale":"2024-04-25T13:58:38.175566+0000","last_undegraded":"2024-04-25T13:58:38.175566+0000","last_fullsized":"2024-04-25T13:58:38.175566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:58.517208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.328809+0000","last_change":"2024-04-25T13:58:09.328809+0000","last_active":"2024-04-25T13:58:09.328809+0000","last_peered":"2024-04-25T13:58:09.328809+0000","last_clean":"2024-04-25T13:58:09.328809+0000","last_became_active":"2024-04-25T13:58:09.326068+0000","last_became_peered":"2024-04-25T13:58:09.326068+0000","last_unstale":"2024-04-25T13:58:09.328809+0000","last_undegraded":"2024-04-25T13:58:09.328809+0000","last_fullsized":"2024-04-25T13:58:09.328809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:14.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.176620+0000","last_change":"2024-04-25T13:57:12.150247+0000","last_active":"2024-04-25T13:58:38.176620+0000","last_peered":"2024-04-25T13:58:38.176620+0000","last_clean":"2024-04-25T13:58:38.176620+0000","last_became_active":"2024-04-25T13:57:12.149882+0000","last_became_peered":"2024-04-25T13:57:12.149882+0000","last_unstale":"2024-04-25T13:58:38.176620+0000","last_undegraded":"2024-04-25T13:58:38.176620+0000","last_fullsized":"2024-04-25T13:58:38.176620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.624928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160753+0000","last_change":"2024-04-25T13:57:12.154618+0000","last_active":"2024-04-25T13:58:38.160753+0000","last_peered":"2024-04-25T13:58:38.160753+0000","last_clean":"2024-04-25T13:58:38.160753+0000","last_became_active":"2024-04-25T13:57:12.153648+0000","last_became_peered":"2024-04-25T13:57:12.153648+0000","last_unstale":"2024-04-25T13:58:38.160753+0000","last_undegraded":"2024-04-25T13:58:38.160753+0000","last_fullsized":"2024-04-25T13:58:38.160753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:24.010980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160683+0000","last_change":"2024-04-25T13:57:12.143097+0000","last_active":"2024-04-25T13:58:38.160683+0000","last_peered":"2024-04-25T13:58:38.160683+0000","last_clean":"2024-04-25T13:58:38.160683+0000","last_became_active":"2024-04-25T13:57:12.142295+0000","last_became_peered":"2024-04-25T13:57:12.142295+0000","last_unstale":"2024-04-25T13:58:38.160683+0000","last_undegraded":"2024-04-25T13:58:38.160683+0000","last_fullsized":"2024-04-25T13:58:38.160683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.907643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160759+0000","last_change":"2024-04-25T13:57:12.155659+0000","last_active":"2024-04-25T13:58:38.160759+0000","last_peered":"2024-04-25T13:58:38.160759+0000","last_clean":"2024-04-25T13:58:38.160759+0000","last_became_active":"2024-04-25T13:57:12.155308+0000","last_became_peered":"2024-04-25T13:57:12.155308+0000","last_unstale":"2024-04-25T13:58:38.160759+0000","last_undegraded":"2024-04-25T13:58:38.160759+0000","last_fullsized":"2024-04-25T13:58:38.160759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:12.361660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177243+0000","last_change":"2024-04-25T13:58:09.212066+0000","last_active":"2024-04-25T13:58:38.177243+0000","last_peered":"2024-04-25T13:58:38.177243+0000","last_clean":"2024-04-25T13:58:38.177243+0000","last_became_active":"2024-04-25T13:58:09.211708+0000","last_became_peered":"2024-04-25T13:58:09.211708+0000","last_unstale":"2024-04-25T13:58:38.177243+0000","last_undegraded":"2024-04-25T13:58:38.177243+0000","last_fullsized":"2024-04-25T13:58:38.177243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:34.585999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163940+0000","last_change":"2024-04-25T13:57:12.142134+0000","last_active":"2024-04-25T13:58:38.163940+0000","last_peered":"2024-04-25T13:58:38.163940+0000","last_clean":"2024-04-25T13:58:38.163940+0000","last_became_active":"2024-04-25T13:57:12.141776+0000","last_became_peered":"2024-04-25T13:57:12.141776+0000","last_unstale":"2024-04-25T13:58:38.163940+0000","last_undegraded":"2024-04-25T13:58:38.163940+0000","last_fullsized":"2024-04-25T13:58:38.163940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:07.621761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158589+0000","last_change":"2024-04-25T13:58:09.213725+0000","last_active":"2024-04-25T13:58:38.158589+0000","last_peered":"2024-04-25T13:58:38.158589+0000","last_clean":"2024-04-25T13:58:38.158589+0000","last_became_active":"2024-04-25T13:58:09.213229+0000","last_became_peered":"2024-04-25T13:58:09.213229+0000","last_unstale":"2024-04-25T13:58:38.158589+0000","last_undegraded":"2024-04-25T13:58:38.158589+0000","last_fullsized":"2024-04-25T13:58:38.158589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:14.728468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179484+0000","last_change":"2024-04-25T13:57:12.147818+0000","last_active":"2024-04-25T13:58:38.179484+0000","last_peered":"2024-04-25T13:58:38.179484+0000","last_clean":"2024-04-25T13:58:38.179484+0000","last_became_active":"2024-04-25T13:57:12.146764+0000","last_became_peered":"2024-04-25T13:57:12.146764+0000","last_unstale":"2024-04-25T13:58:38.179484+0000","last_undegraded":"2024-04-25T13:58:38.179484+0000","last_fullsized":"2024-04-25T13:58:38.179484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.750115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158703+0000","last_change":"2024-04-25T13:57:12.152591+0000","last_active":"2024-04-25T13:58:38.158703+0000","last_peered":"2024-04-25T13:58:38.158703+0000","last_clean":"2024-04-25T13:58:38.158703+0000","last_became_active":"2024-04-25T13:57:12.152198+0000","last_became_peered":"2024-04-25T13:57:12.152198+0000","last_unstale":"2024-04-25T13:58:38.158703+0000","last_undegraded":"2024-04-25T13:58:38.158703+0000","last_fullsized":"2024-04-25T13:58:38.158703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:05.868774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.157434+0000","last_change":"2024-04-25T13:57:12.154924+0000","last_active":"2024-04-25T13:58:38.157434+0000","last_peered":"2024-04-25T13:58:38.157434+0000","last_clean":"2024-04-25T13:58:38.157434+0000","last_became_active":"2024-04-25T13:57:12.154549+0000","last_became_peered":"2024-04-25T13:57:12.154549+0000","last_unstale":"2024-04-25T13:58:38.157434+0000","last_undegraded":"2024-04-25T13:58:38.157434+0000","last_fullsized":"2024-04-25T13:58:38.157434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:49.418436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155858+0000","last_change":"2024-04-25T13:58:09.216933+0000","last_active":"2024-04-25T13:58:38.155858+0000","last_peered":"2024-04-25T13:58:38.155858+0000","last_clean":"2024-04-25T13:58:38.155858+0000","last_became_active":"2024-04-25T13:58:09.216615+0000","last_became_peered":"2024-04-25T13:58:09.216615+0000","last_unstale":"2024-04-25T13:58:38.155858+0000","last_undegraded":"2024-04-25T13:58:38.155858+0000","last_fullsized":"2024-04-25T13:58:38.155858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:25.597023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180794+0000","last_change":"2024-04-25T13:57:12.158672+0000","last_active":"2024-04-25T13:58:38.180794+0000","last_peered":"2024-04-25T13:58:38.180794+0000","last_clean":"2024-04-25T13:58:38.180794+0000","last_became_active":"2024-04-25T13:57:12.158391+0000","last_became_peered":"2024-04-25T13:57:12.158391+0000","last_unstale":"2024-04-25T13:58:38.180794+0000","last_undegraded":"2024-04-25T13:58:38.180794+0000","last_fullsized":"2024-04-25T13:58:38.180794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.369907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178211+0000","last_change":"2024-04-25T13:57:12.139310+0000","last_active":"2024-04-25T13:58:38.178211+0000","last_peered":"2024-04-25T13:58:38.178211+0000","last_clean":"2024-04-25T13:58:38.178211+0000","last_became_active":"2024-04-25T13:57:12.138942+0000","last_became_peered":"2024-04-25T13:57:12.138942+0000","last_unstale":"2024-04-25T13:58:38.178211+0000","last_undegraded":"2024-04-25T13:58:38.178211+0000","last_fullsized":"2024-04-25T13:58:38.178211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:01.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.162465+0000","last_change":"2024-04-25T13:57:12.147451+0000","last_active":"2024-04-25T13:58:38.162465+0000","last_peered":"2024-04-25T13:58:38.162465+0000","last_clean":"2024-04-25T13:58:38.162465+0000","last_became_active":"2024-04-25T13:57:12.146746+0000","last_became_peered":"2024-04-25T13:57:12.146746+0000","last_unstale":"2024-04-25T13:58:38.162465+0000","last_undegraded":"2024-04-25T13:58:38.162465+0000","last_fullsized":"2024-04-25T13:58:38.162465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247568,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66277,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":728,"num_read_kb":435,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1617,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":207767,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":835584,"data_stored":402153,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1054857,"data_compressed":43506,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":401785,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":839680,"data_stored":401683,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":993726,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":401174,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":466884,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":402445,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":467612,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":401485,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":403605,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":993460,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":65917,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66645,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63713,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:59:14.839 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:59:14.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:14 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2615397218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:14.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:14 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/475791378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:14.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:14 smithi129 ceph-mon[32569]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T13:59:14.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:14 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2082016280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:14.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.983+0000 7fc62cb7a640 1 -- 172.21.15.78:0/2288888394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628102f50 msgr2=0x7fc628103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.983+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/2288888394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628102f50 0x7fc628103360 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc618009900 tx=0x7fc61802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.983+0000 7fc62cb7a640 1 -- 172.21.15.78:0/2288888394 shutdown_connections 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.983+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/2288888394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628104450 0x7fc62810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.984+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/2288888394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc628103930 0x7fc628103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.984+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/2288888394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628102f50 0x7fc628103360 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.984+0000 7fc62cb7a640 1 -- 172.21.15.78:0/2288888394 >> 172.21.15.78:0/2288888394 conn(0x7fc6280fd330 msgr2=0x7fc6280ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.984+0000 7fc62cb7a640 1 -- 172.21.15.78:0/2288888394 shutdown_connections 2024-04-25T13:59:14.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.984+0000 7fc62cb7a640 1 -- 172.21.15.78:0/2288888394 wait complete. 2024-04-25T13:59:14.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.984+0000 7fc62cb7a640 1 Processor -- start 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.984+0000 7fc62cb7a640 1 -- start start 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62cb7a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc628102f50 0x7fc6281021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62cb7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628103930 0x7fc62819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62cb7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628104450 0x7fc6281a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62cb7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc62810e550 con 0x7fc628104450 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62cb7a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc62810e3d0 con 0x7fc628103930 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62cb7a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc62810e6d0 con 0x7fc628102f50 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628104450 0x7fc6281a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628104450 0x7fc6281a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43156/0 (socket says 172.21.15.78:43156) 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 -- 172.21.15.78:0/177730750 learned_addr learned my addr 172.21.15.78:0/177730750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc625d9b640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628103930 0x7fc62819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62659c640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc628102f50 0x7fc6281021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 -- 172.21.15.78:0/177730750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc628102f50 msgr2=0x7fc6281021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc628102f50 0x7fc6281021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 -- 172.21.15.78:0/177730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628103930 msgr2=0x7fc62819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628103930 0x7fc62819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 -- 172.21.15.78:0/177730750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6281a1d70 con 0x7fc628104450 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc626d9d640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628104450 0x7fc6281a1550 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc62000eaa0 tx=0x7fc62000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc6177fe640 1 -- 172.21.15.78:0/177730750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620004d20 con 0x7fc628104450 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc6177fe640 1 -- 172.21.15.78:0/177730750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc620005020 con 0x7fc628104450 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc6177fe640 1 -- 172.21.15.78:0/177730750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6200116b0 con 0x7fc628104450 2024-04-25T13:59:14.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.985+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6281a2060 con 0x7fc628104450 2024-04-25T13:59:14.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.986+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6281a2630 con 0x7fc628104450 2024-04-25T13:59:14.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.987+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6281078f0 con 0x7fc628104450 2024-04-25T13:59:14.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.988+0000 7fc6177fe640 1 -- 172.21.15.78:0/177730750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc6200051d0 con 0x7fc628104450 2024-04-25T13:59:14.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.989+0000 7fc6177fe640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6100afb70 0x7fc6100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:14.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.989+0000 7fc62659c640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6100afb70 0x7fc6100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:14.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.989+0000 7fc6177fe640 1 -- 172.21.15.78:0/177730750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc6200df110 con 0x7fc628104450 2024-04-25T13:59:14.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.989+0000 7fc62659c640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6100afb70 0x7fc6100b2080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc618009230 tx=0x7fc618031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:14.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:14.993+0000 7fc6177fe640 1 -- 172.21.15.78:0/177730750 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc620016030 con 0x7fc628104450 2024-04-25T13:59:15.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:14 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2615397218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:15.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:14 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/475791378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:15.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:14 smithi112 ceph-mon[32617]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T13:59:15.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:14 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2082016280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.169+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc628062fe0 con 0x7fc6100afb70 2024-04-25T13:59:15.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.176+0000 7fc6177fe640 1 -- 172.21.15.78:0/177730750 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259096 (secure 0 0 0) 0x7fc628062fe0 con 0x7fc6100afb70 2024-04-25T13:59:15.179 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T13:59:15.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6100afb70 msgr2=0x7fc6100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6100afb70 0x7fc6100b2080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc618009230 tx=0x7fc618031400 comp rx=0 tx=0).stop 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628104450 msgr2=0x7fc6281a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628104450 0x7fc6281a1550 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc62000eaa0 tx=0x7fc62000eea0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 shutdown_connections 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6100afb70 0x7fc6100b2080 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc628104450 0x7fc6281a1550 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc628103930 0x7fc62819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 --2- 172.21.15.78:0/177730750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc628102f50 0x7fc6281021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 >> 172.21.15.78:0/177730750 conn(0x7fc6280fd330 msgr2=0x7fc628104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 shutdown_connections 2024-04-25T13:59:15.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.179+0000 7fc62cb7a640 1 -- 172.21.15.78:0/177730750 wait complete. 2024-04-25T13:59:15.190 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-04-25T13:59:14.159924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656879,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":804,"num_read_kb":499,"num_write":295,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953457,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371852,"kb_used_data":10220,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569972,"statfs":{"total":1151940427776,"available":1151559651328,"internally_reserved":0,"allocated":10465280,"data_stored":6790869,"data_compressed":182799,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":36,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003429"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.169510+0000","last_change":"2024-04-25T13:57:12.147384+0000","last_active":"2024-04-25T13:58:38.169510+0000","last_peered":"2024-04-25T13:58:38.169510+0000","last_clean":"2024-04-25T13:58:38.169510+0000","last_became_active":"2024-04-25T13:57:12.146349+0000","last_became_peered":"2024-04-25T13:57:12.146349+0000","last_unstale":"2024-04-25T13:58:38.169510+0000","last_undegraded":"2024-04-25T13:58:38.169510+0000","last_fullsized":"2024-04-25T13:58:38.169510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.151563+0000","last_change":"2024-04-25T13:57:12.156698+0000","last_active":"2024-04-25T13:58:38.151563+0000","last_peered":"2024-04-25T13:58:38.151563+0000","last_clean":"2024-04-25T13:58:38.151563+0000","last_became_active":"2024-04-25T13:57:12.155790+0000","last_became_peered":"2024-04-25T13:57:12.155790+0000","last_unstale":"2024-04-25T13:58:38.151563+0000","last_undegraded":"2024-04-25T13:58:38.151563+0000","last_fullsized":"2024-04-25T13:58:38.151563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:25.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178850+0000","last_change":"2024-04-25T13:57:12.149458+0000","last_active":"2024-04-25T13:58:38.178850+0000","last_peered":"2024-04-25T13:58:38.178850+0000","last_clean":"2024-04-25T13:58:38.178850+0000","last_became_active":"2024-04-25T13:57:12.148179+0000","last_became_peered":"2024-04-25T13:57:12.148179+0000","last_unstale":"2024-04-25T13:58:38.178850+0000","last_undegraded":"2024-04-25T13:58:38.178850+0000","last_fullsized":"2024-04-25T13:58:38.178850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:00.502798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.167668+0000","last_change":"2024-04-25T13:57:12.145776+0000","last_active":"2024-04-25T13:58:38.167668+0000","last_peered":"2024-04-25T13:58:38.167668+0000","last_clean":"2024-04-25T13:58:38.167668+0000","last_became_active":"2024-04-25T13:57:12.145038+0000","last_became_peered":"2024-04-25T13:57:12.145038+0000","last_unstale":"2024-04-25T13:58:38.167668+0000","last_undegraded":"2024-04-25T13:58:38.167668+0000","last_fullsized":"2024-04-25T13:58:38.167668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:52.742746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.148319+0000","last_change":"2024-04-25T13:57:12.160273+0000","last_active":"2024-04-25T13:58:38.148319+0000","last_peered":"2024-04-25T13:58:38.148319+0000","last_clean":"2024-04-25T13:58:38.148319+0000","last_became_active":"2024-04-25T13:57:12.159805+0000","last_became_peered":"2024-04-25T13:57:12.159805+0000","last_unstale":"2024-04-25T13:58:38.148319+0000","last_undegraded":"2024-04-25T13:58:38.148319+0000","last_fullsized":"2024-04-25T13:58:38.148319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:15.044610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158942+0000","last_change":"2024-04-25T13:57:12.151597+0000","last_active":"2024-04-25T13:58:38.158942+0000","last_peered":"2024-04-25T13:58:38.158942+0000","last_clean":"2024-04-25T13:58:38.158942+0000","last_became_active":"2024-04-25T13:57:12.151203+0000","last_became_peered":"2024-04-25T13:57:12.151203+0000","last_unstale":"2024-04-25T13:58:38.158942+0000","last_undegraded":"2024-04-25T13:58:38.158942+0000","last_fullsized":"2024-04-25T13:58:38.158942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.645264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.176553+0000","last_change":"2024-04-25T13:57:12.144959+0000","last_active":"2024-04-25T13:58:38.176553+0000","last_peered":"2024-04-25T13:58:38.176553+0000","last_clean":"2024-04-25T13:58:38.176553+0000","last_became_active":"2024-04-25T13:57:12.144587+0000","last_became_peered":"2024-04-25T13:57:12.144587+0000","last_unstale":"2024-04-25T13:58:38.176553+0000","last_undegraded":"2024-04-25T13:58:38.176553+0000","last_fullsized":"2024-04-25T13:58:38.176553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:45.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.154354+0000","last_change":"2024-04-25T13:57:12.157016+0000","last_active":"2024-04-25T13:58:38.154354+0000","last_peered":"2024-04-25T13:58:38.154354+0000","last_clean":"2024-04-25T13:58:38.154354+0000","last_became_active":"2024-04-25T13:57:12.156209+0000","last_became_peered":"2024-04-25T13:57:12.156209+0000","last_unstale":"2024-04-25T13:58:38.154354+0000","last_undegraded":"2024-04-25T13:58:38.154354+0000","last_fullsized":"2024-04-25T13:58:38.154354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.172499+0000","last_change":"2024-04-25T13:57:12.146696+0000","last_active":"2024-04-25T13:58:38.172499+0000","last_peered":"2024-04-25T13:58:38.172499+0000","last_clean":"2024-04-25T13:58:38.172499+0000","last_became_active":"2024-04-25T13:57:12.146317+0000","last_became_peered":"2024-04-25T13:57:12.146317+0000","last_unstale":"2024-04-25T13:58:38.172499+0000","last_undegraded":"2024-04-25T13:58:38.172499+0000","last_fullsized":"2024-04-25T13:58:38.172499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.078941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164792+0000","last_change":"2024-04-25T13:57:12.155100+0000","last_active":"2024-04-25T13:58:38.164792+0000","last_peered":"2024-04-25T13:58:38.164792+0000","last_clean":"2024-04-25T13:58:38.164792+0000","last_became_active":"2024-04-25T13:57:12.154602+0000","last_became_peered":"2024-04-25T13:57:12.154602+0000","last_unstale":"2024-04-25T13:58:38.164792+0000","last_undegraded":"2024-04-25T13:58:38.164792+0000","last_fullsized":"2024-04-25T13:58:38.164792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.837117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180815+0000","last_change":"2024-04-25T13:57:12.149386+0000","last_active":"2024-04-25T13:58:38.180815+0000","last_peered":"2024-04-25T13:58:38.180815+0000","last_clean":"2024-04-25T13:58:38.180815+0000","last_became_active":"2024-04-25T13:57:12.148710+0000","last_became_peered":"2024-04-25T13:57:12.148710+0000","last_unstale":"2024-04-25T13:58:38.180815+0000","last_undegraded":"2024-04-25T13:58:38.180815+0000","last_fullsized":"2024-04-25T13:58:38.180815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:17.655260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.167445+0000","last_change":"2024-04-25T13:57:12.156103+0000","last_active":"2024-04-25T13:58:38.167445+0000","last_peered":"2024-04-25T13:58:38.167445+0000","last_clean":"2024-04-25T13:58:38.167445+0000","last_became_active":"2024-04-25T13:57:12.155738+0000","last_became_peered":"2024-04-25T13:57:12.155738+0000","last_unstale":"2024-04-25T13:58:38.167445+0000","last_undegraded":"2024-04-25T13:58:38.167445+0000","last_fullsized":"2024-04-25T13:58:38.167445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:38.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160381+0000","last_change":"2024-04-25T13:57:12.154873+0000","last_active":"2024-04-25T13:58:38.160381+0000","last_peered":"2024-04-25T13:58:38.160381+0000","last_clean":"2024-04-25T13:58:38.160381+0000","last_became_active":"2024-04-25T13:57:12.154523+0000","last_became_peered":"2024-04-25T13:57:12.154523+0000","last_unstale":"2024-04-25T13:58:38.160381+0000","last_undegraded":"2024-04-25T13:58:38.160381+0000","last_fullsized":"2024-04-25T13:58:38.160381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:28.376005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164599+0000","last_change":"2024-04-25T13:57:12.156536+0000","last_active":"2024-04-25T13:58:38.164599+0000","last_peered":"2024-04-25T13:58:38.164599+0000","last_clean":"2024-04-25T13:58:38.164599+0000","last_became_active":"2024-04-25T13:57:12.156143+0000","last_became_peered":"2024-04-25T13:57:12.156143+0000","last_unstale":"2024-04-25T13:58:38.164599+0000","last_undegraded":"2024-04-25T13:58:38.164599+0000","last_fullsized":"2024-04-25T13:58:38.164599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:35.210059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163271+0000","last_change":"2024-04-25T13:57:12.144477+0000","last_active":"2024-04-25T13:58:38.163271+0000","last_peered":"2024-04-25T13:58:38.163271+0000","last_clean":"2024-04-25T13:58:38.163271+0000","last_became_active":"2024-04-25T13:57:12.144115+0000","last_became_peered":"2024-04-25T13:57:12.144115+0000","last_unstale":"2024-04-25T13:58:38.163271+0000","last_undegraded":"2024-04-25T13:58:38.163271+0000","last_fullsized":"2024-04-25T13:58:38.163271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:30.733713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179821+0000","last_change":"2024-04-25T13:57:12.148837+0000","last_active":"2024-04-25T13:58:38.179821+0000","last_peered":"2024-04-25T13:58:38.179821+0000","last_clean":"2024-04-25T13:58:38.179821+0000","last_became_active":"2024-04-25T13:57:12.148494+0000","last_became_peered":"2024-04-25T13:57:12.148494+0000","last_unstale":"2024-04-25T13:58:38.179821+0000","last_undegraded":"2024-04-25T13:58:38.179821+0000","last_fullsized":"2024-04-25T13:58:38.179821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:50.414344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.169715+0000","last_change":"2024-04-25T13:57:12.148142+0000","last_active":"2024-04-25T13:58:38.169715+0000","last_peered":"2024-04-25T13:58:38.169715+0000","last_clean":"2024-04-25T13:58:38.169715+0000","last_became_active":"2024-04-25T13:57:12.147738+0000","last_became_peered":"2024-04-25T13:57:12.147738+0000","last_unstale":"2024-04-25T13:58:38.169715+0000","last_undegraded":"2024-04-25T13:58:38.169715+0000","last_fullsized":"2024-04-25T13:58:38.169715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:42.014605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.174951+0000","last_change":"2024-04-25T13:57:12.160059+0000","last_active":"2024-04-25T13:58:38.174951+0000","last_peered":"2024-04-25T13:58:38.174951+0000","last_clean":"2024-04-25T13:58:38.174951+0000","last_became_active":"2024-04-25T13:57:12.159685+0000","last_became_peered":"2024-04-25T13:57:12.159685+0000","last_unstale":"2024-04-25T13:58:38.174951+0000","last_undegraded":"2024-04-25T13:58:38.174951+0000","last_fullsized":"2024-04-25T13:58:38.174951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:48.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.153349+0000","last_change":"2024-04-25T13:57:12.157950+0000","last_active":"2024-04-25T13:58:38.153349+0000","last_peered":"2024-04-25T13:58:38.153349+0000","last_clean":"2024-04-25T13:58:38.153349+0000","last_became_active":"2024-04-25T13:57:12.157612+0000","last_became_peered":"2024-04-25T13:57:12.157612+0000","last_unstale":"2024-04-25T13:58:38.153349+0000","last_undegraded":"2024-04-25T13:58:38.153349+0000","last_fullsized":"2024-04-25T13:58:38.153349+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:55.521381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.222728+0000","last_change":"2024-04-25T13:58:09.222728+0000","last_active":"2024-04-25T13:58:09.222728+0000","last_peered":"2024-04-25T13:58:09.222728+0000","last_clean":"2024-04-25T13:58:09.222728+0000","last_became_active":"2024-04-25T13:58:09.210819+0000","last_became_peered":"2024-04-25T13:58:09.210819+0000","last_unstale":"2024-04-25T13:58:09.222728+0000","last_undegraded":"2024-04-25T13:58:09.222728+0000","last_fullsized":"2024-04-25T13:58:09.222728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:00.606838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155336+0000","last_change":"2024-04-25T13:57:12.151301+0000","last_active":"2024-04-25T13:58:38.155336+0000","last_peered":"2024-04-25T13:58:38.155336+0000","last_clean":"2024-04-25T13:58:38.155336+0000","last_became_active":"2024-04-25T13:57:12.150941+0000","last_became_peered":"2024-04-25T13:57:12.150941+0000","last_unstale":"2024-04-25T13:58:38.155336+0000","last_undegraded":"2024-04-25T13:58:38.155336+0000","last_fullsized":"2024-04-25T13:58:38.155336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:22.259683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213379+0000","last_change":"2024-04-25T13:57:11.147314+0000","last_active":"2024-04-25T13:58:09.213379+0000","last_peered":"2024-04-25T13:58:09.213379+0000","last_clean":"2024-04-25T13:58:09.213379+0000","last_became_active":"2024-04-25T13:57:11.146894+0000","last_became_peered":"2024-04-25T13:57:11.146894+0000","last_unstale":"2024-04-25T13:58:09.213379+0000","last_undegraded":"2024-04-25T13:58:09.213379+0000","last_fullsized":"2024-04-25T13:58:09.213379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:55.927007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178922+0000","last_change":"2024-04-25T13:57:12.149186+0000","last_active":"2024-04-25T13:58:38.178922+0000","last_peered":"2024-04-25T13:58:38.178922+0000","last_clean":"2024-04-25T13:58:38.178922+0000","last_became_active":"2024-04-25T13:57:12.148155+0000","last_became_peered":"2024-04-25T13:57:12.148155+0000","last_unstale":"2024-04-25T13:58:38.178922+0000","last_undegraded":"2024-04-25T13:58:38.178922+0000","last_fullsized":"2024-04-25T13:58:38.178922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209681+0000","last_change":"2024-04-25T13:57:11.140086+0000","last_active":"2024-04-25T13:58:09.209681+0000","last_peered":"2024-04-25T13:58:09.209681+0000","last_clean":"2024-04-25T13:58:09.209681+0000","last_became_active":"2024-04-25T13:57:11.139751+0000","last_became_peered":"2024-04-25T13:57:11.139751+0000","last_unstale":"2024-04-25T13:58:09.209681+0000","last_undegraded":"2024-04-25T13:58:09.209681+0000","last_fullsized":"2024-04-25T13:58:09.209681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.552503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179369+0000","last_change":"2024-04-25T13:57:12.157029+0000","last_active":"2024-04-25T13:58:38.179369+0000","last_peered":"2024-04-25T13:58:38.179369+0000","last_clean":"2024-04-25T13:58:38.179369+0000","last_became_active":"2024-04-25T13:57:12.156668+0000","last_became_peered":"2024-04-25T13:57:12.156668+0000","last_unstale":"2024-04-25T13:58:38.179369+0000","last_undegraded":"2024-04-25T13:58:38.179369+0000","last_fullsized":"2024-04-25T13:58:38.179369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208828+0000","last_change":"2024-04-25T13:57:11.140592+0000","last_active":"2024-04-25T13:58:09.208828+0000","last_peered":"2024-04-25T13:58:09.208828+0000","last_clean":"2024-04-25T13:58:09.208828+0000","last_became_active":"2024-04-25T13:57:11.140244+0000","last_became_peered":"2024-04-25T13:57:11.140244+0000","last_unstale":"2024-04-25T13:58:09.208828+0000","last_undegraded":"2024-04-25T13:58:09.208828+0000","last_fullsized":"2024-04-25T13:58:09.208828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:53.071092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.174853+0000","last_change":"2024-04-25T13:58:09.326284+0000","last_active":"2024-04-25T13:58:38.174853+0000","last_peered":"2024-04-25T13:58:38.174853+0000","last_clean":"2024-04-25T13:58:38.174853+0000","last_became_active":"2024-04-25T13:58:09.325888+0000","last_became_peered":"2024-04-25T13:58:09.325888+0000","last_unstale":"2024-04-25T13:58:38.174853+0000","last_undegraded":"2024-04-25T13:58:38.174853+0000","last_fullsized":"2024-04-25T13:58:38.174853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:39.151793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":160,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.323118+0000","last_change":"2024-04-25T13:58:09.221709+0000","last_active":"2024-04-25T13:59:11.323118+0000","last_peered":"2024-04-25T13:59:11.323118+0000","last_clean":"2024-04-25T13:59:11.323118+0000","last_became_active":"2024-04-25T13:58:09.215262+0000","last_became_peered":"2024-04-25T13:58:09.215262+0000","last_unstale":"2024-04-25T13:59:11.323118+0000","last_undegraded":"2024-04-25T13:59:11.323118+0000","last_fullsized":"2024-04-25T13:59:11.323118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:05.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155581+0000","last_change":"2024-04-25T13:57:12.148787+0000","last_active":"2024-04-25T13:58:38.155581+0000","last_peered":"2024-04-25T13:58:38.155581+0000","last_clean":"2024-04-25T13:58:38.155581+0000","last_became_active":"2024-04-25T13:57:12.148424+0000","last_became_peered":"2024-04-25T13:57:12.148424+0000","last_unstale":"2024-04-25T13:58:38.155581+0000","last_undegraded":"2024-04-25T13:58:38.155581+0000","last_fullsized":"2024-04-25T13:58:38.155581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:11.921873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324717+0000","last_change":"2024-04-25T13:57:11.142424+0000","last_active":"2024-04-25T13:58:09.324717+0000","last_peered":"2024-04-25T13:58:09.324717+0000","last_clean":"2024-04-25T13:58:09.324717+0000","last_became_active":"2024-04-25T13:57:11.142047+0000","last_became_peered":"2024-04-25T13:57:11.142047+0000","last_unstale":"2024-04-25T13:58:09.324717+0000","last_undegraded":"2024-04-25T13:58:09.324717+0000","last_fullsized":"2024-04-25T13:58:09.324717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:13.455149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163379+0000","last_change":"2024-04-25T13:57:12.155940+0000","last_active":"2024-04-25T13:58:38.163379+0000","last_peered":"2024-04-25T13:58:38.163379+0000","last_clean":"2024-04-25T13:58:38.163379+0000","last_became_active":"2024-04-25T13:57:12.155503+0000","last_became_peered":"2024-04-25T13:57:12.155503+0000","last_unstale":"2024-04-25T13:58:38.163379+0000","last_undegraded":"2024-04-25T13:58:38.163379+0000","last_fullsized":"2024-04-25T13:58:38.163379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:48.345437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209853+0000","last_change":"2024-04-25T13:57:11.131160+0000","last_active":"2024-04-25T13:58:09.209853+0000","last_peered":"2024-04-25T13:58:09.209853+0000","last_clean":"2024-04-25T13:58:09.209853+0000","last_became_active":"2024-04-25T13:57:11.130753+0000","last_became_peered":"2024-04-25T13:57:11.130753+0000","last_unstale":"2024-04-25T13:58:09.209853+0000","last_undegraded":"2024-04-25T13:58:09.209853+0000","last_fullsized":"2024-04-25T13:58:09.209853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:48.297546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.156739+0000","last_change":"2024-04-25T13:57:12.155779+0000","last_active":"2024-04-25T13:58:38.156739+0000","last_peered":"2024-04-25T13:58:38.156739+0000","last_clean":"2024-04-25T13:58:38.156739+0000","last_became_active":"2024-04-25T13:57:12.154682+0000","last_became_peered":"2024-04-25T13:57:12.154682+0000","last_unstale":"2024-04-25T13:58:38.156739+0000","last_undegraded":"2024-04-25T13:58:38.156739+0000","last_fullsized":"2024-04-25T13:58:38.156739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:20.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211839+0000","last_change":"2024-04-25T13:57:11.140211+0000","last_active":"2024-04-25T13:58:09.211839+0000","last_peered":"2024-04-25T13:58:09.211839+0000","last_clean":"2024-04-25T13:58:09.211839+0000","last_became_active":"2024-04-25T13:57:11.139891+0000","last_became_peered":"2024-04-25T13:57:11.139891+0000","last_unstale":"2024-04-25T13:58:09.211839+0000","last_undegraded":"2024-04-25T13:58:09.211839+0000","last_fullsized":"2024-04-25T13:58:09.211839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:21.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.159292+0000","last_change":"2024-04-25T13:57:12.153886+0000","last_active":"2024-04-25T13:58:38.159292+0000","last_peered":"2024-04-25T13:58:38.159292+0000","last_clean":"2024-04-25T13:58:38.159292+0000","last_became_active":"2024-04-25T13:57:12.153500+0000","last_became_peered":"2024-04-25T13:57:12.153500+0000","last_unstale":"2024-04-25T13:58:38.159292+0000","last_undegraded":"2024-04-25T13:58:38.159292+0000","last_fullsized":"2024-04-25T13:58:38.159292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:53.549745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:09.318464+0000","last_change":"2024-04-25T13:57:11.138829+0000","last_active":"2024-04-25T13:59:09.318464+0000","last_peered":"2024-04-25T13:59:09.318464+0000","last_clean":"2024-04-25T13:59:09.318464+0000","last_became_active":"2024-04-25T13:57:11.138461+0000","last_became_peered":"2024-04-25T13:57:11.138461+0000","last_unstale":"2024-04-25T13:59:09.318464+0000","last_undegraded":"2024-04-25T13:59:09.318464+0000","last_fullsized":"2024-04-25T13:59:09.318464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.134061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177713+0000","last_change":"2024-04-25T13:57:12.157394+0000","last_active":"2024-04-25T13:58:38.177713+0000","last_peered":"2024-04-25T13:58:38.177713+0000","last_clean":"2024-04-25T13:58:38.177713+0000","last_became_active":"2024-04-25T13:57:12.156408+0000","last_became_peered":"2024-04-25T13:57:12.156408+0000","last_unstale":"2024-04-25T13:58:38.177713+0000","last_undegraded":"2024-04-25T13:58:38.177713+0000","last_fullsized":"2024-04-25T13:58:38.177713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:36.741367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208928+0000","last_change":"2024-04-25T13:57:11.142846+0000","last_active":"2024-04-25T13:58:09.208928+0000","last_peered":"2024-04-25T13:58:09.208928+0000","last_clean":"2024-04-25T13:58:09.208928+0000","last_became_active":"2024-04-25T13:57:11.142387+0000","last_became_peered":"2024-04-25T13:57:11.142387+0000","last_unstale":"2024-04-25T13:58:09.208928+0000","last_undegraded":"2024-04-25T13:58:09.208928+0000","last_fullsized":"2024-04-25T13:58:09.208928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.164994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163650+0000","last_change":"2024-04-25T13:57:12.145617+0000","last_active":"2024-04-25T13:58:38.163650+0000","last_peered":"2024-04-25T13:58:38.163650+0000","last_clean":"2024-04-25T13:58:38.163650+0000","last_became_active":"2024-04-25T13:57:12.145263+0000","last_became_peered":"2024-04-25T13:57:12.145263+0000","last_unstale":"2024-04-25T13:58:38.163650+0000","last_undegraded":"2024-04-25T13:58:38.163650+0000","last_fullsized":"2024-04-25T13:58:38.163650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.886538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212101+0000","last_change":"2024-04-25T13:57:11.136972+0000","last_active":"2024-04-25T13:58:09.212101+0000","last_peered":"2024-04-25T13:58:09.212101+0000","last_clean":"2024-04-25T13:58:09.212101+0000","last_became_active":"2024-04-25T13:57:11.136392+0000","last_became_peered":"2024-04-25T13:57:11.136392+0000","last_unstale":"2024-04-25T13:58:09.212101+0000","last_undegraded":"2024-04-25T13:58:09.212101+0000","last_fullsized":"2024-04-25T13:58:09.212101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:45.841670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177590+0000","last_change":"2024-04-25T13:57:12.144905+0000","last_active":"2024-04-25T13:58:38.177590+0000","last_peered":"2024-04-25T13:58:38.177590+0000","last_clean":"2024-04-25T13:58:38.177590+0000","last_became_active":"2024-04-25T13:57:12.144404+0000","last_became_peered":"2024-04-25T13:57:12.144404+0000","last_unstale":"2024-04-25T13:58:38.177590+0000","last_undegraded":"2024-04-25T13:58:38.177590+0000","last_fullsized":"2024-04-25T13:58:38.177590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.186492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'31","reported_seq":291,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:10.306989+0000","last_change":"2024-04-25T13:57:11.139041+0000","last_active":"2024-04-25T13:59:10.306989+0000","last_peered":"2024-04-25T13:59:10.306989+0000","last_clean":"2024-04-25T13:59:10.306989+0000","last_became_active":"2024-04-25T13:57:11.138629+0000","last_became_peered":"2024-04-25T13:57:11.138629+0000","last_unstale":"2024-04-25T13:59:10.306989+0000","last_undegraded":"2024-04-25T13:59:10.306989+0000","last_fullsized":"2024-04-25T13:59:10.306989+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:04.863015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":180,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177642+0000","last_change":"2024-04-25T13:57:12.160897+0000","last_active":"2024-04-25T13:58:38.177642+0000","last_peered":"2024-04-25T13:58:38.177642+0000","last_clean":"2024-04-25T13:58:38.177642+0000","last_became_active":"2024-04-25T13:57:12.160322+0000","last_became_peered":"2024-04-25T13:57:12.160322+0000","last_unstale":"2024-04-25T13:58:38.177642+0000","last_undegraded":"2024-04-25T13:58:38.177642+0000","last_fullsized":"2024-04-25T13:58:38.177642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:26.180324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216914+0000","last_change":"2024-04-25T13:57:11.137104+0000","last_active":"2024-04-25T13:58:09.216914+0000","last_peered":"2024-04-25T13:58:09.216914+0000","last_clean":"2024-04-25T13:58:09.216914+0000","last_became_active":"2024-04-25T13:57:11.136720+0000","last_became_peered":"2024-04-25T13:57:11.136720+0000","last_unstale":"2024-04-25T13:58:09.216914+0000","last_undegraded":"2024-04-25T13:58:09.216914+0000","last_fullsized":"2024-04-25T13:58:09.216914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:41.262776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177140+0000","last_change":"2024-04-25T13:57:12.147831+0000","last_active":"2024-04-25T13:58:38.177140+0000","last_peered":"2024-04-25T13:58:38.177140+0000","last_clean":"2024-04-25T13:58:38.177140+0000","last_became_active":"2024-04-25T13:57:12.147270+0000","last_became_peered":"2024-04-25T13:57:12.147270+0000","last_unstale":"2024-04-25T13:58:38.177140+0000","last_undegraded":"2024-04-25T13:58:38.177140+0000","last_fullsized":"2024-04-25T13:58:38.177140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:31.574661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211663+0000","last_change":"2024-04-25T13:57:11.138254+0000","last_active":"2024-04-25T13:58:09.211663+0000","last_peered":"2024-04-25T13:58:09.211663+0000","last_clean":"2024-04-25T13:58:09.211663+0000","last_became_active":"2024-04-25T13:57:11.137924+0000","last_became_peered":"2024-04-25T13:57:11.137924+0000","last_unstale":"2024-04-25T13:58:09.211663+0000","last_undegraded":"2024-04-25T13:58:09.211663+0000","last_fullsized":"2024-04-25T13:58:09.211663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:27.746203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179497+0000","last_change":"2024-04-25T13:57:12.145743+0000","last_active":"2024-04-25T13:58:38.179497+0000","last_peered":"2024-04-25T13:58:38.179497+0000","last_clean":"2024-04-25T13:58:38.179497+0000","last_became_active":"2024-04-25T13:57:12.145337+0000","last_became_peered":"2024-04-25T13:57:12.145337+0000","last_unstale":"2024-04-25T13:58:38.179497+0000","last_undegraded":"2024-04-25T13:58:38.179497+0000","last_fullsized":"2024-04-25T13:58:38.179497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.692900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'6","reported_seq":146,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.269043+0000","last_change":"2024-04-25T13:57:11.138059+0000","last_active":"2024-04-25T13:59:11.269043+0000","last_peered":"2024-04-25T13:59:11.269043+0000","last_clean":"2024-04-25T13:59:11.269043+0000","last_became_active":"2024-04-25T13:57:11.137698+0000","last_became_peered":"2024-04-25T13:57:11.137698+0000","last_unstale":"2024-04-25T13:59:11.269043+0000","last_undegraded":"2024-04-25T13:59:11.269043+0000","last_fullsized":"2024-04-25T13:59:11.269043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:03.595126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.171572+0000","last_change":"2024-04-25T13:57:12.155507+0000","last_active":"2024-04-25T13:58:38.171572+0000","last_peered":"2024-04-25T13:58:38.171572+0000","last_clean":"2024-04-25T13:58:38.171572+0000","last_became_active":"2024-04-25T13:57:12.155175+0000","last_became_peered":"2024-04-25T13:57:12.155175+0000","last_unstale":"2024-04-25T13:58:38.171572+0000","last_undegraded":"2024-04-25T13:58:38.171572+0000","last_fullsized":"2024-04-25T13:58:38.171572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:21.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'15","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208247+0000","last_change":"2024-04-25T13:57:11.141672+0000","last_active":"2024-04-25T13:58:09.208247+0000","last_peered":"2024-04-25T13:58:09.208247+0000","last_clean":"2024-04-25T13:58:09.208247+0000","last_became_active":"2024-04-25T13:57:11.140957+0000","last_became_peered":"2024-04-25T13:57:11.140957+0000","last_unstale":"2024-04-25T13:58:09.208247+0000","last_undegraded":"2024-04-25T13:58:09.208247+0000","last_fullsized":"2024-04-25T13:58:09.208247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:47.362949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.165834+0000","last_change":"2024-04-25T13:57:12.147881+0000","last_active":"2024-04-25T13:58:38.165834+0000","last_peered":"2024-04-25T13:58:38.165834+0000","last_clean":"2024-04-25T13:58:38.165834+0000","last_became_active":"2024-04-25T13:57:12.146200+0000","last_became_peered":"2024-04-25T13:57:12.146200+0000","last_unstale":"2024-04-25T13:58:38.165834+0000","last_undegraded":"2024-04-25T13:58:38.165834+0000","last_fullsized":"2024-04-25T13:58:38.165834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:51.711045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209588+0000","last_change":"2024-04-25T13:57:11.139574+0000","last_active":"2024-04-25T13:58:09.209588+0000","last_peered":"2024-04-25T13:58:09.209588+0000","last_clean":"2024-04-25T13:58:09.209588+0000","last_became_active":"2024-04-25T13:57:11.139232+0000","last_became_peered":"2024-04-25T13:57:11.139232+0000","last_unstale":"2024-04-25T13:58:09.209588+0000","last_undegraded":"2024-04-25T13:58:09.209588+0000","last_fullsized":"2024-04-25T13:58:09.209588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:08.590270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.162405+0000","last_change":"2024-04-25T13:57:12.147122+0000","last_active":"2024-04-25T13:58:38.162405+0000","last_peered":"2024-04-25T13:58:38.162405+0000","last_clean":"2024-04-25T13:58:38.162405+0000","last_became_active":"2024-04-25T13:57:12.146708+0000","last_became_peered":"2024-04-25T13:57:12.146708+0000","last_unstale":"2024-04-25T13:58:38.162405+0000","last_undegraded":"2024-04-25T13:58:38.162405+0000","last_fullsized":"2024-04-25T13:58:38.162405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:47.858290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.205903+0000","last_change":"2024-04-25T13:57:11.146288+0000","last_active":"2024-04-25T13:58:09.205903+0000","last_peered":"2024-04-25T13:58:09.205903+0000","last_clean":"2024-04-25T13:58:09.205903+0000","last_became_active":"2024-04-25T13:57:11.145851+0000","last_became_peered":"2024-04-25T13:57:11.145851+0000","last_unstale":"2024-04-25T13:58:09.205903+0000","last_undegraded":"2024-04-25T13:58:09.205903+0000","last_fullsized":"2024-04-25T13:58:09.205903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155416+0000","last_change":"2024-04-25T13:57:12.154576+0000","last_active":"2024-04-25T13:58:38.155416+0000","last_peered":"2024-04-25T13:58:38.155416+0000","last_clean":"2024-04-25T13:58:38.155416+0000","last_became_active":"2024-04-25T13:57:12.154254+0000","last_became_peered":"2024-04-25T13:57:12.154254+0000","last_unstale":"2024-04-25T13:58:38.155416+0000","last_undegraded":"2024-04-25T13:58:38.155416+0000","last_fullsized":"2024-04-25T13:58:38.155416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:33.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208334+0000","last_change":"2024-04-25T13:57:11.139941+0000","last_active":"2024-04-25T13:58:09.208334+0000","last_peered":"2024-04-25T13:58:09.208334+0000","last_clean":"2024-04-25T13:58:09.208334+0000","last_became_active":"2024-04-25T13:57:11.139614+0000","last_became_peered":"2024-04-25T13:57:11.139614+0000","last_unstale":"2024-04-25T13:58:09.208334+0000","last_undegraded":"2024-04-25T13:58:09.208334+0000","last_fullsized":"2024-04-25T13:58:09.208334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:39.656085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164620+0000","last_change":"2024-04-25T13:57:12.155935+0000","last_active":"2024-04-25T13:58:38.164620+0000","last_peered":"2024-04-25T13:58:38.164620+0000","last_clean":"2024-04-25T13:58:38.164620+0000","last_became_active":"2024-04-25T13:57:12.155261+0000","last_became_peered":"2024-04-25T13:57:12.155261+0000","last_unstale":"2024-04-25T13:58:38.164620+0000","last_undegraded":"2024-04-25T13:58:38.164620+0000","last_fullsized":"2024-04-25T13:58:38.164620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:54.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.156931+0000","last_change":"2024-04-25T13:58:09.217792+0000","last_active":"2024-04-25T13:58:38.156931+0000","last_peered":"2024-04-25T13:58:38.156931+0000","last_clean":"2024-04-25T13:58:38.156931+0000","last_became_active":"2024-04-25T13:58:09.217386+0000","last_became_peered":"2024-04-25T13:58:09.217386+0000","last_unstale":"2024-04-25T13:58:38.156931+0000","last_undegraded":"2024-04-25T13:58:38.156931+0000","last_fullsized":"2024-04-25T13:58:38.156931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:13.833587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.268314+0000","last_change":"2024-04-25T13:57:11.135226+0000","last_active":"2024-04-25T13:59:11.268314+0000","last_peered":"2024-04-25T13:59:11.268314+0000","last_clean":"2024-04-25T13:59:11.268314+0000","last_became_active":"2024-04-25T13:57:11.134886+0000","last_became_peered":"2024-04-25T13:57:11.134886+0000","last_unstale":"2024-04-25T13:59:11.268314+0000","last_undegraded":"2024-04-25T13:59:11.268314+0000","last_fullsized":"2024-04-25T13:59:11.268314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:00.153597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.173674+0000","last_change":"2024-04-25T13:57:12.149758+0000","last_active":"2024-04-25T13:58:38.173674+0000","last_peered":"2024-04-25T13:58:38.173674+0000","last_clean":"2024-04-25T13:58:38.173674+0000","last_became_active":"2024-04-25T13:57:12.149428+0000","last_became_peered":"2024-04-25T13:57:12.149428+0000","last_unstale":"2024-04-25T13:58:38.173674+0000","last_undegraded":"2024-04-25T13:58:38.173674+0000","last_fullsized":"2024-04-25T13:58:38.173674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.739074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207722+0000","last_change":"2024-04-25T13:57:11.135968+0000","last_active":"2024-04-25T13:58:09.207722+0000","last_peered":"2024-04-25T13:58:09.207722+0000","last_clean":"2024-04-25T13:58:09.207722+0000","last_became_active":"2024-04-25T13:57:11.135577+0000","last_became_peered":"2024-04-25T13:57:11.135577+0000","last_unstale":"2024-04-25T13:58:09.207722+0000","last_undegraded":"2024-04-25T13:58:09.207722+0000","last_fullsized":"2024-04-25T13:58:09.207722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:00.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208486+0000","last_change":"2024-04-25T13:55:57.666399+0000","last_active":"2024-04-25T13:58:09.208486+0000","last_peered":"2024-04-25T13:58:09.208486+0000","last_clean":"2024-04-25T13:58:09.208486+0000","last_became_active":"2024-04-25T13:55:57.660470+0000","last_became_peered":"2024-04-25T13:55:57.660470+0000","last_unstale":"2024-04-25T13:58:09.208486+0000","last_undegraded":"2024-04-25T13:58:09.208486+0000","last_fullsized":"2024-04-25T13:58:09.208486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_clean_scrub_stamp":"2024-04-25T13:52:33.081441+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:22.788632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163242+0000","last_change":"2024-04-25T13:57:12.141923+0000","last_active":"2024-04-25T13:58:38.163242+0000","last_peered":"2024-04-25T13:58:38.163242+0000","last_clean":"2024-04-25T13:58:38.163242+0000","last_became_active":"2024-04-25T13:57:12.140946+0000","last_became_peered":"2024-04-25T13:57:12.140946+0000","last_unstale":"2024-04-25T13:58:38.163242+0000","last_undegraded":"2024-04-25T13:58:38.163242+0000","last_fullsized":"2024-04-25T13:58:38.163242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:49.294598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208387+0000","last_change":"2024-04-25T13:57:11.139851+0000","last_active":"2024-04-25T13:58:09.208387+0000","last_peered":"2024-04-25T13:58:09.208387+0000","last_clean":"2024-04-25T13:58:09.208387+0000","last_became_active":"2024-04-25T13:57:11.139476+0000","last_became_peered":"2024-04-25T13:57:11.139476+0000","last_unstale":"2024-04-25T13:58:09.208387+0000","last_undegraded":"2024-04-25T13:58:09.208387+0000","last_fullsized":"2024-04-25T13:58:09.208387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:29.435917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.220107+0000","last_change":"2024-04-25T13:58:09.219619+0000","last_active":"2024-04-25T13:58:09.220107+0000","last_peered":"2024-04-25T13:58:09.220107+0000","last_clean":"2024-04-25T13:58:09.220107+0000","last_became_active":"2024-04-25T13:58:09.211808+0000","last_became_peered":"2024-04-25T13:58:09.211808+0000","last_unstale":"2024-04-25T13:58:09.220107+0000","last_undegraded":"2024-04-25T13:58:09.220107+0000","last_fullsized":"2024-04-25T13:58:09.220107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:20.208191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.157529+0000","last_change":"2024-04-25T13:57:12.136204+0000","last_active":"2024-04-25T13:58:38.157529+0000","last_peered":"2024-04-25T13:58:38.157529+0000","last_clean":"2024-04-25T13:58:38.157529+0000","last_became_active":"2024-04-25T13:57:12.135798+0000","last_became_peered":"2024-04-25T13:57:12.135798+0000","last_unstale":"2024-04-25T13:58:38.157529+0000","last_undegraded":"2024-04-25T13:58:38.157529+0000","last_fullsized":"2024-04-25T13:58:38.157529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:36.894774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.206151+0000","last_change":"2024-04-25T13:57:11.138699+0000","last_active":"2024-04-25T13:58:09.206151+0000","last_peered":"2024-04-25T13:58:09.206151+0000","last_clean":"2024-04-25T13:58:09.206151+0000","last_became_active":"2024-04-25T13:57:11.138335+0000","last_became_peered":"2024-04-25T13:57:11.138335+0000","last_unstale":"2024-04-25T13:58:09.206151+0000","last_undegraded":"2024-04-25T13:58:09.206151+0000","last_fullsized":"2024-04-25T13:58:09.206151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:44.413228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.171184+0000","last_change":"2024-04-25T13:57:12.155851+0000","last_active":"2024-04-25T13:58:38.171184+0000","last_peered":"2024-04-25T13:58:38.171184+0000","last_clean":"2024-04-25T13:58:38.171184+0000","last_became_active":"2024-04-25T13:57:12.155416+0000","last_became_peered":"2024-04-25T13:57:12.155416+0000","last_unstale":"2024-04-25T13:58:38.171184+0000","last_undegraded":"2024-04-25T13:58:38.171184+0000","last_fullsized":"2024-04-25T13:58:38.171184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:22.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207744+0000","last_change":"2024-04-25T13:57:11.147465+0000","last_active":"2024-04-25T13:58:09.207744+0000","last_peered":"2024-04-25T13:58:09.207744+0000","last_clean":"2024-04-25T13:58:09.207744+0000","last_became_active":"2024-04-25T13:57:11.147096+0000","last_became_peered":"2024-04-25T13:57:11.147096+0000","last_unstale":"2024-04-25T13:58:09.207744+0000","last_undegraded":"2024-04-25T13:58:09.207744+0000","last_fullsized":"2024-04-25T13:58:09.207744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.229356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163274+0000","last_change":"2024-04-25T13:57:12.146446+0000","last_active":"2024-04-25T13:58:38.163274+0000","last_peered":"2024-04-25T13:58:38.163274+0000","last_clean":"2024-04-25T13:58:38.163274+0000","last_became_active":"2024-04-25T13:57:12.146129+0000","last_became_peered":"2024-04-25T13:57:12.146129+0000","last_unstale":"2024-04-25T13:58:38.163274+0000","last_undegraded":"2024-04-25T13:58:38.163274+0000","last_fullsized":"2024-04-25T13:58:38.163274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.977663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325014+0000","last_change":"2024-04-25T13:57:11.145127+0000","last_active":"2024-04-25T13:58:09.325014+0000","last_peered":"2024-04-25T13:58:09.325014+0000","last_clean":"2024-04-25T13:58:09.325014+0000","last_became_active":"2024-04-25T13:57:11.144822+0000","last_became_peered":"2024-04-25T13:57:11.144822+0000","last_unstale":"2024-04-25T13:58:09.325014+0000","last_undegraded":"2024-04-25T13:58:09.325014+0000","last_fullsized":"2024-04-25T13:58:09.325014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:00.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.121092+0000","last_change":"2024-04-25T13:57:12.156184+0000","last_active":"2024-04-25T13:58:38.121092+0000","last_peered":"2024-04-25T13:58:38.121092+0000","last_clean":"2024-04-25T13:58:38.121092+0000","last_became_active":"2024-04-25T13:57:12.155855+0000","last_became_peered":"2024-04-25T13:57:12.155855+0000","last_unstale":"2024-04-25T13:58:38.121092+0000","last_undegraded":"2024-04-25T13:58:38.121092+0000","last_fullsized":"2024-04-25T13:58:38.121092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213952+0000","last_change":"2024-04-25T13:57:11.137616+0000","last_active":"2024-04-25T13:58:09.213952+0000","last_peered":"2024-04-25T13:58:09.213952+0000","last_clean":"2024-04-25T13:58:09.213952+0000","last_became_active":"2024-04-25T13:57:11.137041+0000","last_became_peered":"2024-04-25T13:57:11.137041+0000","last_unstale":"2024-04-25T13:58:09.213952+0000","last_undegraded":"2024-04-25T13:58:09.213952+0000","last_fullsized":"2024-04-25T13:58:09.213952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:54.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.159060+0000","last_change":"2024-04-25T13:57:12.151223+0000","last_active":"2024-04-25T13:58:38.159060+0000","last_peered":"2024-04-25T13:58:38.159060+0000","last_clean":"2024-04-25T13:58:38.159060+0000","last_became_active":"2024-04-25T13:57:12.150871+0000","last_became_peered":"2024-04-25T13:57:12.150871+0000","last_unstale":"2024-04-25T13:58:38.159060+0000","last_undegraded":"2024-04-25T13:58:38.159060+0000","last_fullsized":"2024-04-25T13:58:38.159060+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:17.957728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216414+0000","last_change":"2024-04-25T13:57:11.136524+0000","last_active":"2024-04-25T13:58:09.216414+0000","last_peered":"2024-04-25T13:58:09.216414+0000","last_clean":"2024-04-25T13:58:09.216414+0000","last_became_active":"2024-04-25T13:57:11.136152+0000","last_became_peered":"2024-04-25T13:57:11.136152+0000","last_unstale":"2024-04-25T13:58:09.216414+0000","last_undegraded":"2024-04-25T13:58:09.216414+0000","last_fullsized":"2024-04-25T13:58:09.216414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:23.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160886+0000","last_change":"2024-04-25T13:57:12.157641+0000","last_active":"2024-04-25T13:58:38.160886+0000","last_peered":"2024-04-25T13:58:38.160886+0000","last_clean":"2024-04-25T13:58:38.160886+0000","last_became_active":"2024-04-25T13:57:12.157281+0000","last_became_peered":"2024-04-25T13:57:12.157281+0000","last_unstale":"2024-04-25T13:58:38.160886+0000","last_undegraded":"2024-04-25T13:58:38.160886+0000","last_fullsized":"2024-04-25T13:58:38.160886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:28.245071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208565+0000","last_change":"2024-04-25T13:57:11.139036+0000","last_active":"2024-04-25T13:58:09.208565+0000","last_peered":"2024-04-25T13:58:09.208565+0000","last_clean":"2024-04-25T13:58:09.208565+0000","last_became_active":"2024-04-25T13:57:11.138681+0000","last_became_peered":"2024-04-25T13:57:11.138681+0000","last_unstale":"2024-04-25T13:58:09.208565+0000","last_undegraded":"2024-04-25T13:58:09.208565+0000","last_fullsized":"2024-04-25T13:58:09.208565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:48.584313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180469+0000","last_change":"2024-04-25T13:57:12.153072+0000","last_active":"2024-04-25T13:58:38.180469+0000","last_peered":"2024-04-25T13:58:38.180469+0000","last_clean":"2024-04-25T13:58:38.180469+0000","last_became_active":"2024-04-25T13:57:12.151556+0000","last_became_peered":"2024-04-25T13:57:12.151556+0000","last_unstale":"2024-04-25T13:58:38.180469+0000","last_undegraded":"2024-04-25T13:58:38.180469+0000","last_fullsized":"2024-04-25T13:58:38.180469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:20.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217008+0000","last_change":"2024-04-25T13:57:11.148823+0000","last_active":"2024-04-25T13:58:09.217008+0000","last_peered":"2024-04-25T13:58:09.217008+0000","last_clean":"2024-04-25T13:58:09.217008+0000","last_became_active":"2024-04-25T13:57:11.148436+0000","last_became_peered":"2024-04-25T13:57:11.148436+0000","last_unstale":"2024-04-25T13:58:09.217008+0000","last_undegraded":"2024-04-25T13:58:09.217008+0000","last_fullsized":"2024-04-25T13:58:09.217008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:28.496184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180714+0000","last_change":"2024-04-25T13:57:12.149082+0000","last_active":"2024-04-25T13:58:38.180714+0000","last_peered":"2024-04-25T13:58:38.180714+0000","last_clean":"2024-04-25T13:58:38.180714+0000","last_became_active":"2024-04-25T13:57:12.148260+0000","last_became_peered":"2024-04-25T13:57:12.148260+0000","last_unstale":"2024-04-25T13:58:38.180714+0000","last_undegraded":"2024-04-25T13:58:38.180714+0000","last_fullsized":"2024-04-25T13:58:38.180714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:49.733114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211832+0000","last_change":"2024-04-25T13:57:11.136736+0000","last_active":"2024-04-25T13:58:09.211832+0000","last_peered":"2024-04-25T13:58:09.211832+0000","last_clean":"2024-04-25T13:58:09.211832+0000","last_became_active":"2024-04-25T13:57:11.135831+0000","last_became_peered":"2024-04-25T13:57:11.135831+0000","last_unstale":"2024-04-25T13:58:09.211832+0000","last_undegraded":"2024-04-25T13:58:09.211832+0000","last_fullsized":"2024-04-25T13:58:09.211832+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:49.654642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.175566+0000","last_change":"2024-04-25T13:57:12.149797+0000","last_active":"2024-04-25T13:58:38.175566+0000","last_peered":"2024-04-25T13:58:38.175566+0000","last_clean":"2024-04-25T13:58:38.175566+0000","last_became_active":"2024-04-25T13:57:12.149413+0000","last_became_peered":"2024-04-25T13:57:12.149413+0000","last_unstale":"2024-04-25T13:58:38.175566+0000","last_undegraded":"2024-04-25T13:58:38.175566+0000","last_fullsized":"2024-04-25T13:58:38.175566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:58.517208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.328809+0000","last_change":"2024-04-25T13:58:09.328809+0000","last_active":"2024-04-25T13:58:09.328809+0000","last_peered":"2024-04-25T13:58:09.328809+0000","last_clean":"2024-04-25T13:58:09.328809+0000","last_became_active":"2024-04-25T13:58:09.326068+0000","last_became_peered":"2024-04-25T13:58:09.326068+0000","last_unstale":"2024-04-25T13:58:09.328809+0000","last_undegraded":"2024-04-25T13:58:09.328809+0000","last_fullsized":"2024-04-25T13:58:09.328809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:14.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.176620+0000","last_change":"2024-04-25T13:57:12.150247+0000","last_active":"2024-04-25T13:58:38.176620+0000","last_peered":"2024-04-25T13:58:38.176620+0000","last_clean":"2024-04-25T13:58:38.176620+0000","last_became_active":"2024-04-25T13:57:12.149882+0000","last_became_peered":"2024-04-25T13:57:12.149882+0000","last_unstale":"2024-04-25T13:58:38.176620+0000","last_undegraded":"2024-04-25T13:58:38.176620+0000","last_fullsized":"2024-04-25T13:58:38.176620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.624928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160753+0000","last_change":"2024-04-25T13:57:12.154618+0000","last_active":"2024-04-25T13:58:38.160753+0000","last_peered":"2024-04-25T13:58:38.160753+0000","last_clean":"2024-04-25T13:58:38.160753+0000","last_became_active":"2024-04-25T13:57:12.153648+0000","last_became_peered":"2024-04-25T13:57:12.153648+0000","last_unstale":"2024-04-25T13:58:38.160753+0000","last_undegraded":"2024-04-25T13:58:38.160753+0000","last_fullsized":"2024-04-25T13:58:38.160753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:24.010980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160683+0000","last_change":"2024-04-25T13:57:12.143097+0000","last_active":"2024-04-25T13:58:38.160683+0000","last_peered":"2024-04-25T13:58:38.160683+0000","last_clean":"2024-04-25T13:58:38.160683+0000","last_became_active":"2024-04-25T13:57:12.142295+0000","last_became_peered":"2024-04-25T13:57:12.142295+0000","last_unstale":"2024-04-25T13:58:38.160683+0000","last_undegraded":"2024-04-25T13:58:38.160683+0000","last_fullsized":"2024-04-25T13:58:38.160683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.907643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160759+0000","last_change":"2024-04-25T13:57:12.155659+0000","last_active":"2024-04-25T13:58:38.160759+0000","last_peered":"2024-04-25T13:58:38.160759+0000","last_clean":"2024-04-25T13:58:38.160759+0000","last_became_active":"2024-04-25T13:57:12.155308+0000","last_became_peered":"2024-04-25T13:57:12.155308+0000","last_unstale":"2024-04-25T13:58:38.160759+0000","last_undegraded":"2024-04-25T13:58:38.160759+0000","last_fullsized":"2024-04-25T13:58:38.160759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:12.361660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177243+0000","last_change":"2024-04-25T13:58:09.212066+0000","last_active":"2024-04-25T13:58:38.177243+0000","last_peered":"2024-04-25T13:58:38.177243+0000","last_clean":"2024-04-25T13:58:38.177243+0000","last_became_active":"2024-04-25T13:58:09.211708+0000","last_became_peered":"2024-04-25T13:58:09.211708+0000","last_unstale":"2024-04-25T13:58:38.177243+0000","last_undegraded":"2024-04-25T13:58:38.177243+0000","last_fullsized":"2024-04-25T13:58:38.177243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:34.585999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163940+0000","last_change":"2024-04-25T13:57:12.142134+0000","last_active":"2024-04-25T13:58:38.163940+0000","last_peered":"2024-04-25T13:58:38.163940+0000","last_clean":"2024-04-25T13:58:38.163940+0000","last_became_active":"2024-04-25T13:57:12.141776+0000","last_became_peered":"2024-04-25T13:57:12.141776+0000","last_unstale":"2024-04-25T13:58:38.163940+0000","last_undegraded":"2024-04-25T13:58:38.163940+0000","last_fullsized":"2024-04-25T13:58:38.163940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:07.621761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158589+0000","last_change":"2024-04-25T13:58:09.213725+0000","last_active":"2024-04-25T13:58:38.158589+0000","last_peered":"2024-04-25T13:58:38.158589+0000","last_clean":"2024-04-25T13:58:38.158589+0000","last_became_active":"2024-04-25T13:58:09.213229+0000","last_became_peered":"2024-04-25T13:58:09.213229+0000","last_unstale":"2024-04-25T13:58:38.158589+0000","last_undegraded":"2024-04-25T13:58:38.158589+0000","last_fullsized":"2024-04-25T13:58:38.158589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:14.728468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179484+0000","last_change":"2024-04-25T13:57:12.147818+0000","last_active":"2024-04-25T13:58:38.179484+0000","last_peered":"2024-04-25T13:58:38.179484+0000","last_clean":"2024-04-25T13:58:38.179484+0000","last_became_active":"2024-04-25T13:57:12.146764+0000","last_became_peered":"2024-04-25T13:57:12.146764+0000","last_unstale":"2024-04-25T13:58:38.179484+0000","last_undegraded":"2024-04-25T13:58:38.179484+0000","last_fullsized":"2024-04-25T13:58:38.179484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.750115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158703+0000","last_change":"2024-04-25T13:57:12.152591+0000","last_active":"2024-04-25T13:58:38.158703+0000","last_peered":"2024-04-25T13:58:38.158703+0000","last_clean":"2024-04-25T13:58:38.158703+0000","last_became_active":"2024-04-25T13:57:12.152198+0000","last_became_peered":"2024-04-25T13:57:12.152198+0000","last_unstale":"2024-04-25T13:58:38.158703+0000","last_undegraded":"2024-04-25T13:58:38.158703+0000","last_fullsized":"2024-04-25T13:58:38.158703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:05.868774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.157434+0000","last_change":"2024-04-25T13:57:12.154924+0000","last_active":"2024-04-25T13:58:38.157434+0000","last_peered":"2024-04-25T13:58:38.157434+0000","last_clean":"2024-04-25T13:58:38.157434+0000","last_became_active":"2024-04-25T13:57:12.154549+0000","last_became_peered":"2024-04-25T13:57:12.154549+0000","last_unstale":"2024-04-25T13:58:38.157434+0000","last_undegraded":"2024-04-25T13:58:38.157434+0000","last_fullsized":"2024-04-25T13:58:38.157434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:49.418436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155858+0000","last_change":"2024-04-25T13:58:09.216933+0000","last_active":"2024-04-25T13:58:38.155858+0000","last_peered":"2024-04-25T13:58:38.155858+0000","last_clean":"2024-04-25T13:58:38.155858+0000","last_became_active":"2024-04-25T13:58:09.216615+0000","last_became_peered":"2024-04-25T13:58:09.216615+0000","last_unstale":"2024-04-25T13:58:38.155858+0000","last_undegraded":"2024-04-25T13:58:38.155858+0000","last_fullsized":"2024-04-25T13:58:38.155858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:25.597023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180794+0000","last_change":"2024-04-25T13:57:12.158672+0000","last_active":"2024-04-25T13:58:38.180794+0000","last_peered":"2024-04-25T13:58:38.180794+0000","last_clean":"2024-04-25T13:58:38.180794+0000","last_became_active":"2024-04-25T13:57:12.158391+0000","last_became_peered":"2024-04-25T13:57:12.158391+0000","last_unstale":"2024-04-25T13:58:38.180794+0000","last_undegraded":"2024-04-25T13:58:38.180794+0000","last_fullsized":"2024-04-25T13:58:38.180794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.369907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178211+0000","last_change":"2024-04-25T13:57:12.139310+0000","last_active":"2024-04-25T13:58:38.178211+0000","last_peered":"2024-04-25T13:58:38.178211+0000","last_clean":"2024-04-25T13:58:38.178211+0000","last_became_active":"2024-04-25T13:57:12.138942+0000","last_became_peered":"2024-04-25T13:57:12.138942+0000","last_unstale":"2024-04-25T13:58:38.178211+0000","last_undegraded":"2024-04-25T13:58:38.178211+0000","last_fullsized":"2024-04-25T13:58:38.178211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:01.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.162465+0000","last_change":"2024-04-25T13:57:12.147451+0000","last_active":"2024-04-25T13:58:38.162465+0000","last_peered":"2024-04-25T13:58:38.162465+0000","last_clean":"2024-04-25T13:58:38.162465+0000","last_became_active":"2024-04-25T13:57:12.146746+0000","last_became_peered":"2024-04-25T13:57:12.146746+0000","last_unstale":"2024-04-25T13:58:38.162465+0000","last_undegraded":"2024-04-25T13:58:38.162465+0000","last_fullsized":"2024-04-25T13:58:38.162465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247568,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66277,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":728,"num_read_kb":435,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1617,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":207767,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":835584,"data_stored":402153,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1054857,"data_compressed":43506,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":401785,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":839680,"data_stored":401683,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":993726,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":401174,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":466884,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":402445,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":467612,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":401485,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":403605,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":993460,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":65917,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66645,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63713,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:59:15.197 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-25T13:59:15.198 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:59:15.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 -- 172.21.15.78:0/1324634068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84103c00 msgr2=0x7f9f84104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/1324634068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84103c00 0x7f9f84104010 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009900 tx=0x7f9f7802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:15.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 -- 172.21.15.78:0/1324634068 shutdown_connections 2024-04-25T13:59:15.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/1324634068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f84108900 0x7f9f8410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/1324634068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84103c00 0x7f9f84104010 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/1324634068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f841029c0 0x7f9f841036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 -- 172.21.15.78:0/1324634068 >> 172.21.15.78:0/1324634068 conn(0x7f9f840fdbd0 msgr2=0x7f9f84100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 -- 172.21.15.78:0/1324634068 shutdown_connections 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.339+0000 7f9f8bee7640 1 -- 172.21.15.78:0/1324634068 wait complete. 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 Processor -- start 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 -- start start 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f841029c0 0x7f9f8406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f84103c00 0x7f9f8419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84108900 0x7f9f841a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8410e4f0 con 0x7f9f84108900 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8410e370 con 0x7f9f84103c00 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8bee7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8410e670 con 0x7f9f841029c0 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84108900 0x7f9f841a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84108900 0x7f9f841a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43174/0 (socket says 172.21.15.78:43174) 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 -- 172.21.15.78:0/971565201 learned_addr learned my addr 172.21.15.78:0/971565201 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:15.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f89c83640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f841029c0 0x7f9f8406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f89482640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f84103c00 0x7f9f8419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 -- 172.21.15.78:0/971565201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f841029c0 msgr2=0x7f9f8406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f841029c0 0x7f9f8406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 -- 172.21.15.78:0/971565201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f84103c00 msgr2=0x7f9f8419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f84103c00 0x7f9f8419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.340+0000 7f9f8a484640 1 -- 172.21.15.78:0/971565201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f841a1d20 con 0x7f9f84108900 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.341+0000 7f9f8a484640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84108900 0x7f9f841a1500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00eaa0 tx=0x7f9f7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.341+0000 7f9f76ffd640 1 -- 172.21.15.78:0/971565201 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00cdb0 con 0x7f9f84108900 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.341+0000 7f9f76ffd640 1 -- 172.21.15.78:0/971565201 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f7c005020 con 0x7f9f84108900 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.341+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f841a1fb0 con 0x7f9f84108900 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.341+0000 7f9f76ffd640 1 -- 172.21.15.78:0/971565201 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0116d0 con 0x7f9f84108900 2024-04-25T13:59:15.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.341+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f841a2470 con 0x7f9f84108900 2024-04-25T13:59:15.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.342+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f84107710 con 0x7f9f84108900 2024-04-25T13:59:15.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.343+0000 7f9f76ffd640 1 -- 172.21.15.78:0/971565201 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9f7c0051d0 con 0x7f9f84108900 2024-04-25T13:59:15.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.344+0000 7f9f76ffd640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f700afaf0 0x7f9f700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.344+0000 7f9f89c83640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f700afaf0 0x7f9f700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.344+0000 7f9f76ffd640 1 -- 172.21.15.78:0/971565201 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9f7c0df160 con 0x7f9f84108900 2024-04-25T13:59:15.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.344+0000 7f9f89c83640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f700afaf0 0x7f9f700b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f841035c0 tx=0x7f9f8000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:15.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.347+0000 7f9f76ffd640 1 -- 172.21.15.78:0/971565201 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f7c016030 con 0x7f9f84108900 2024-04-25T13:59:15.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.525+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 --> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f84062fe0 con 0x7f9f700afaf0 2024-04-25T13:59:15.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.532+0000 7f9f76ffd640 1 -- 172.21.15.78:0/971565201 <== mgr.14152 v2:172.21.15.78:6800/3547239980 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259096 (secure 0 0 0) 0x7f9f84062fe0 con 0x7f9f700afaf0 2024-04-25T13:59:15.537 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:15.539 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T13:59:15.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f700afaf0 msgr2=0x7f9f700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f700afaf0 0x7f9f700b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f841035c0 tx=0x7f9f8000b390 comp rx=0 tx=0).stop 2024-04-25T13:59:15.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84108900 msgr2=0x7f9f841a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84108900 0x7f9f841a1500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00eaa0 tx=0x7f9f7c00ef60 comp rx=0 tx=0).stop 2024-04-25T13:59:15.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 shutdown_connections 2024-04-25T13:59:15.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f700afaf0 0x7f9f700b2000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f84108900 0x7f9f841a1500 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f84103c00 0x7f9f8419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 --2- 172.21.15.78:0/971565201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f841029c0 0x7f9f8406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 >> 172.21.15.78:0/971565201 conn(0x7f9f840fdbd0 msgr2=0x7f9f841048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:15.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.535+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 shutdown_connections 2024-04-25T13:59:15.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.536+0000 7f9f8bee7640 1 -- 172.21.15.78:0/971565201 wait complete. 2024-04-25T13:59:15.546 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-04-25T13:59:14.159924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656879,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":804,"num_read_kb":499,"num_write":295,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953457,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371852,"kb_used_data":10220,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569972,"statfs":{"total":1151940427776,"available":1151559651328,"internally_reserved":0,"allocated":10465280,"data_stored":6790869,"data_compressed":182799,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":36,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003429"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.169510+0000","last_change":"2024-04-25T13:57:12.147384+0000","last_active":"2024-04-25T13:58:38.169510+0000","last_peered":"2024-04-25T13:58:38.169510+0000","last_clean":"2024-04-25T13:58:38.169510+0000","last_became_active":"2024-04-25T13:57:12.146349+0000","last_became_peered":"2024-04-25T13:57:12.146349+0000","last_unstale":"2024-04-25T13:58:38.169510+0000","last_undegraded":"2024-04-25T13:58:38.169510+0000","last_fullsized":"2024-04-25T13:58:38.169510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.507731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.151563+0000","last_change":"2024-04-25T13:57:12.156698+0000","last_active":"2024-04-25T13:58:38.151563+0000","last_peered":"2024-04-25T13:58:38.151563+0000","last_clean":"2024-04-25T13:58:38.151563+0000","last_became_active":"2024-04-25T13:57:12.155790+0000","last_became_peered":"2024-04-25T13:57:12.155790+0000","last_unstale":"2024-04-25T13:58:38.151563+0000","last_undegraded":"2024-04-25T13:58:38.151563+0000","last_fullsized":"2024-04-25T13:58:38.151563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:25.312883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178850+0000","last_change":"2024-04-25T13:57:12.149458+0000","last_active":"2024-04-25T13:58:38.178850+0000","last_peered":"2024-04-25T13:58:38.178850+0000","last_clean":"2024-04-25T13:58:38.178850+0000","last_became_active":"2024-04-25T13:57:12.148179+0000","last_became_peered":"2024-04-25T13:57:12.148179+0000","last_unstale":"2024-04-25T13:58:38.178850+0000","last_undegraded":"2024-04-25T13:58:38.178850+0000","last_fullsized":"2024-04-25T13:58:38.178850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:00.502798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.167668+0000","last_change":"2024-04-25T13:57:12.145776+0000","last_active":"2024-04-25T13:58:38.167668+0000","last_peered":"2024-04-25T13:58:38.167668+0000","last_clean":"2024-04-25T13:58:38.167668+0000","last_became_active":"2024-04-25T13:57:12.145038+0000","last_became_peered":"2024-04-25T13:57:12.145038+0000","last_unstale":"2024-04-25T13:58:38.167668+0000","last_undegraded":"2024-04-25T13:58:38.167668+0000","last_fullsized":"2024-04-25T13:58:38.167668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:52.742746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.148319+0000","last_change":"2024-04-25T13:57:12.160273+0000","last_active":"2024-04-25T13:58:38.148319+0000","last_peered":"2024-04-25T13:58:38.148319+0000","last_clean":"2024-04-25T13:58:38.148319+0000","last_became_active":"2024-04-25T13:57:12.159805+0000","last_became_peered":"2024-04-25T13:57:12.159805+0000","last_unstale":"2024-04-25T13:58:38.148319+0000","last_undegraded":"2024-04-25T13:58:38.148319+0000","last_fullsized":"2024-04-25T13:58:38.148319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:15.044610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158942+0000","last_change":"2024-04-25T13:57:12.151597+0000","last_active":"2024-04-25T13:58:38.158942+0000","last_peered":"2024-04-25T13:58:38.158942+0000","last_clean":"2024-04-25T13:58:38.158942+0000","last_became_active":"2024-04-25T13:57:12.151203+0000","last_became_peered":"2024-04-25T13:57:12.151203+0000","last_unstale":"2024-04-25T13:58:38.158942+0000","last_undegraded":"2024-04-25T13:58:38.158942+0000","last_fullsized":"2024-04-25T13:58:38.158942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.645264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.176553+0000","last_change":"2024-04-25T13:57:12.144959+0000","last_active":"2024-04-25T13:58:38.176553+0000","last_peered":"2024-04-25T13:58:38.176553+0000","last_clean":"2024-04-25T13:58:38.176553+0000","last_became_active":"2024-04-25T13:57:12.144587+0000","last_became_peered":"2024-04-25T13:57:12.144587+0000","last_unstale":"2024-04-25T13:58:38.176553+0000","last_undegraded":"2024-04-25T13:58:38.176553+0000","last_fullsized":"2024-04-25T13:58:38.176553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:45.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.154354+0000","last_change":"2024-04-25T13:57:12.157016+0000","last_active":"2024-04-25T13:58:38.154354+0000","last_peered":"2024-04-25T13:58:38.154354+0000","last_clean":"2024-04-25T13:58:38.154354+0000","last_became_active":"2024-04-25T13:57:12.156209+0000","last_became_peered":"2024-04-25T13:57:12.156209+0000","last_unstale":"2024-04-25T13:58:38.154354+0000","last_undegraded":"2024-04-25T13:58:38.154354+0000","last_fullsized":"2024-04-25T13:58:38.154354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:36.175763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.172499+0000","last_change":"2024-04-25T13:57:12.146696+0000","last_active":"2024-04-25T13:58:38.172499+0000","last_peered":"2024-04-25T13:58:38.172499+0000","last_clean":"2024-04-25T13:58:38.172499+0000","last_became_active":"2024-04-25T13:57:12.146317+0000","last_became_peered":"2024-04-25T13:57:12.146317+0000","last_unstale":"2024-04-25T13:58:38.172499+0000","last_undegraded":"2024-04-25T13:58:38.172499+0000","last_fullsized":"2024-04-25T13:58:38.172499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.078941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164792+0000","last_change":"2024-04-25T13:57:12.155100+0000","last_active":"2024-04-25T13:58:38.164792+0000","last_peered":"2024-04-25T13:58:38.164792+0000","last_clean":"2024-04-25T13:58:38.164792+0000","last_became_active":"2024-04-25T13:57:12.154602+0000","last_became_peered":"2024-04-25T13:57:12.154602+0000","last_unstale":"2024-04-25T13:58:38.164792+0000","last_undegraded":"2024-04-25T13:58:38.164792+0000","last_fullsized":"2024-04-25T13:58:38.164792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.837117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180815+0000","last_change":"2024-04-25T13:57:12.149386+0000","last_active":"2024-04-25T13:58:38.180815+0000","last_peered":"2024-04-25T13:58:38.180815+0000","last_clean":"2024-04-25T13:58:38.180815+0000","last_became_active":"2024-04-25T13:57:12.148710+0000","last_became_peered":"2024-04-25T13:57:12.148710+0000","last_unstale":"2024-04-25T13:58:38.180815+0000","last_undegraded":"2024-04-25T13:58:38.180815+0000","last_fullsized":"2024-04-25T13:58:38.180815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:17.655260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.167445+0000","last_change":"2024-04-25T13:57:12.156103+0000","last_active":"2024-04-25T13:58:38.167445+0000","last_peered":"2024-04-25T13:58:38.167445+0000","last_clean":"2024-04-25T13:58:38.167445+0000","last_became_active":"2024-04-25T13:57:12.155738+0000","last_became_peered":"2024-04-25T13:57:12.155738+0000","last_unstale":"2024-04-25T13:58:38.167445+0000","last_undegraded":"2024-04-25T13:58:38.167445+0000","last_fullsized":"2024-04-25T13:58:38.167445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:38.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160381+0000","last_change":"2024-04-25T13:57:12.154873+0000","last_active":"2024-04-25T13:58:38.160381+0000","last_peered":"2024-04-25T13:58:38.160381+0000","last_clean":"2024-04-25T13:58:38.160381+0000","last_became_active":"2024-04-25T13:57:12.154523+0000","last_became_peered":"2024-04-25T13:57:12.154523+0000","last_unstale":"2024-04-25T13:58:38.160381+0000","last_undegraded":"2024-04-25T13:58:38.160381+0000","last_fullsized":"2024-04-25T13:58:38.160381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:28.376005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164599+0000","last_change":"2024-04-25T13:57:12.156536+0000","last_active":"2024-04-25T13:58:38.164599+0000","last_peered":"2024-04-25T13:58:38.164599+0000","last_clean":"2024-04-25T13:58:38.164599+0000","last_became_active":"2024-04-25T13:57:12.156143+0000","last_became_peered":"2024-04-25T13:57:12.156143+0000","last_unstale":"2024-04-25T13:58:38.164599+0000","last_undegraded":"2024-04-25T13:58:38.164599+0000","last_fullsized":"2024-04-25T13:58:38.164599+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:35.210059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163271+0000","last_change":"2024-04-25T13:57:12.144477+0000","last_active":"2024-04-25T13:58:38.163271+0000","last_peered":"2024-04-25T13:58:38.163271+0000","last_clean":"2024-04-25T13:58:38.163271+0000","last_became_active":"2024-04-25T13:57:12.144115+0000","last_became_peered":"2024-04-25T13:57:12.144115+0000","last_unstale":"2024-04-25T13:58:38.163271+0000","last_undegraded":"2024-04-25T13:58:38.163271+0000","last_fullsized":"2024-04-25T13:58:38.163271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:30.733713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179821+0000","last_change":"2024-04-25T13:57:12.148837+0000","last_active":"2024-04-25T13:58:38.179821+0000","last_peered":"2024-04-25T13:58:38.179821+0000","last_clean":"2024-04-25T13:58:38.179821+0000","last_became_active":"2024-04-25T13:57:12.148494+0000","last_became_peered":"2024-04-25T13:57:12.148494+0000","last_unstale":"2024-04-25T13:58:38.179821+0000","last_undegraded":"2024-04-25T13:58:38.179821+0000","last_fullsized":"2024-04-25T13:58:38.179821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:50.414344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.169715+0000","last_change":"2024-04-25T13:57:12.148142+0000","last_active":"2024-04-25T13:58:38.169715+0000","last_peered":"2024-04-25T13:58:38.169715+0000","last_clean":"2024-04-25T13:58:38.169715+0000","last_became_active":"2024-04-25T13:57:12.147738+0000","last_became_peered":"2024-04-25T13:57:12.147738+0000","last_unstale":"2024-04-25T13:58:38.169715+0000","last_undegraded":"2024-04-25T13:58:38.169715+0000","last_fullsized":"2024-04-25T13:58:38.169715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:42.014605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.174951+0000","last_change":"2024-04-25T13:57:12.160059+0000","last_active":"2024-04-25T13:58:38.174951+0000","last_peered":"2024-04-25T13:58:38.174951+0000","last_clean":"2024-04-25T13:58:38.174951+0000","last_became_active":"2024-04-25T13:57:12.159685+0000","last_became_peered":"2024-04-25T13:57:12.159685+0000","last_unstale":"2024-04-25T13:58:38.174951+0000","last_undegraded":"2024-04-25T13:58:38.174951+0000","last_fullsized":"2024-04-25T13:58:38.174951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:48.958812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.153349+0000","last_change":"2024-04-25T13:57:12.157950+0000","last_active":"2024-04-25T13:58:38.153349+0000","last_peered":"2024-04-25T13:58:38.153349+0000","last_clean":"2024-04-25T13:58:38.153349+0000","last_became_active":"2024-04-25T13:57:12.157612+0000","last_became_peered":"2024-04-25T13:57:12.157612+0000","last_unstale":"2024-04-25T13:58:38.153349+0000","last_undegraded":"2024-04-25T13:58:38.153349+0000","last_fullsized":"2024-04-25T13:58:38.153349+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:55.521381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.222728+0000","last_change":"2024-04-25T13:58:09.222728+0000","last_active":"2024-04-25T13:58:09.222728+0000","last_peered":"2024-04-25T13:58:09.222728+0000","last_clean":"2024-04-25T13:58:09.222728+0000","last_became_active":"2024-04-25T13:58:09.210819+0000","last_became_peered":"2024-04-25T13:58:09.210819+0000","last_unstale":"2024-04-25T13:58:09.222728+0000","last_undegraded":"2024-04-25T13:58:09.222728+0000","last_fullsized":"2024-04-25T13:58:09.222728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:00.606838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155336+0000","last_change":"2024-04-25T13:57:12.151301+0000","last_active":"2024-04-25T13:58:38.155336+0000","last_peered":"2024-04-25T13:58:38.155336+0000","last_clean":"2024-04-25T13:58:38.155336+0000","last_became_active":"2024-04-25T13:57:12.150941+0000","last_became_peered":"2024-04-25T13:57:12.150941+0000","last_unstale":"2024-04-25T13:58:38.155336+0000","last_undegraded":"2024-04-25T13:58:38.155336+0000","last_fullsized":"2024-04-25T13:58:38.155336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:22.259683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213379+0000","last_change":"2024-04-25T13:57:11.147314+0000","last_active":"2024-04-25T13:58:09.213379+0000","last_peered":"2024-04-25T13:58:09.213379+0000","last_clean":"2024-04-25T13:58:09.213379+0000","last_became_active":"2024-04-25T13:57:11.146894+0000","last_became_peered":"2024-04-25T13:57:11.146894+0000","last_unstale":"2024-04-25T13:58:09.213379+0000","last_undegraded":"2024-04-25T13:58:09.213379+0000","last_fullsized":"2024-04-25T13:58:09.213379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:55.927007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178922+0000","last_change":"2024-04-25T13:57:12.149186+0000","last_active":"2024-04-25T13:58:38.178922+0000","last_peered":"2024-04-25T13:58:38.178922+0000","last_clean":"2024-04-25T13:58:38.178922+0000","last_became_active":"2024-04-25T13:57:12.148155+0000","last_became_peered":"2024-04-25T13:57:12.148155+0000","last_unstale":"2024-04-25T13:58:38.178922+0000","last_undegraded":"2024-04-25T13:58:38.178922+0000","last_fullsized":"2024-04-25T13:58:38.178922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:05.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209681+0000","last_change":"2024-04-25T13:57:11.140086+0000","last_active":"2024-04-25T13:58:09.209681+0000","last_peered":"2024-04-25T13:58:09.209681+0000","last_clean":"2024-04-25T13:58:09.209681+0000","last_became_active":"2024-04-25T13:57:11.139751+0000","last_became_peered":"2024-04-25T13:57:11.139751+0000","last_unstale":"2024-04-25T13:58:09.209681+0000","last_undegraded":"2024-04-25T13:58:09.209681+0000","last_fullsized":"2024-04-25T13:58:09.209681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.552503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179369+0000","last_change":"2024-04-25T13:57:12.157029+0000","last_active":"2024-04-25T13:58:38.179369+0000","last_peered":"2024-04-25T13:58:38.179369+0000","last_clean":"2024-04-25T13:58:38.179369+0000","last_became_active":"2024-04-25T13:57:12.156668+0000","last_became_peered":"2024-04-25T13:57:12.156668+0000","last_unstale":"2024-04-25T13:58:38.179369+0000","last_undegraded":"2024-04-25T13:58:38.179369+0000","last_fullsized":"2024-04-25T13:58:38.179369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:37.345470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208828+0000","last_change":"2024-04-25T13:57:11.140592+0000","last_active":"2024-04-25T13:58:09.208828+0000","last_peered":"2024-04-25T13:58:09.208828+0000","last_clean":"2024-04-25T13:58:09.208828+0000","last_became_active":"2024-04-25T13:57:11.140244+0000","last_became_peered":"2024-04-25T13:57:11.140244+0000","last_unstale":"2024-04-25T13:58:09.208828+0000","last_undegraded":"2024-04-25T13:58:09.208828+0000","last_fullsized":"2024-04-25T13:58:09.208828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:53.071092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.174853+0000","last_change":"2024-04-25T13:58:09.326284+0000","last_active":"2024-04-25T13:58:38.174853+0000","last_peered":"2024-04-25T13:58:38.174853+0000","last_clean":"2024-04-25T13:58:38.174853+0000","last_became_active":"2024-04-25T13:58:09.325888+0000","last_became_peered":"2024-04-25T13:58:09.325888+0000","last_unstale":"2024-04-25T13:58:38.174853+0000","last_undegraded":"2024-04-25T13:58:38.174853+0000","last_fullsized":"2024-04-25T13:58:38.174853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:39.151793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":160,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.323118+0000","last_change":"2024-04-25T13:58:09.221709+0000","last_active":"2024-04-25T13:59:11.323118+0000","last_peered":"2024-04-25T13:59:11.323118+0000","last_clean":"2024-04-25T13:59:11.323118+0000","last_became_active":"2024-04-25T13:58:09.215262+0000","last_became_peered":"2024-04-25T13:58:09.215262+0000","last_unstale":"2024-04-25T13:59:11.323118+0000","last_undegraded":"2024-04-25T13:59:11.323118+0000","last_fullsized":"2024-04-25T13:59:11.323118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:05.858125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155581+0000","last_change":"2024-04-25T13:57:12.148787+0000","last_active":"2024-04-25T13:58:38.155581+0000","last_peered":"2024-04-25T13:58:38.155581+0000","last_clean":"2024-04-25T13:58:38.155581+0000","last_became_active":"2024-04-25T13:57:12.148424+0000","last_became_peered":"2024-04-25T13:57:12.148424+0000","last_unstale":"2024-04-25T13:58:38.155581+0000","last_undegraded":"2024-04-25T13:58:38.155581+0000","last_fullsized":"2024-04-25T13:58:38.155581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:11.921873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.324717+0000","last_change":"2024-04-25T13:57:11.142424+0000","last_active":"2024-04-25T13:58:09.324717+0000","last_peered":"2024-04-25T13:58:09.324717+0000","last_clean":"2024-04-25T13:58:09.324717+0000","last_became_active":"2024-04-25T13:57:11.142047+0000","last_became_peered":"2024-04-25T13:57:11.142047+0000","last_unstale":"2024-04-25T13:58:09.324717+0000","last_undegraded":"2024-04-25T13:58:09.324717+0000","last_fullsized":"2024-04-25T13:58:09.324717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:13.455149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163379+0000","last_change":"2024-04-25T13:57:12.155940+0000","last_active":"2024-04-25T13:58:38.163379+0000","last_peered":"2024-04-25T13:58:38.163379+0000","last_clean":"2024-04-25T13:58:38.163379+0000","last_became_active":"2024-04-25T13:57:12.155503+0000","last_became_peered":"2024-04-25T13:57:12.155503+0000","last_unstale":"2024-04-25T13:58:38.163379+0000","last_undegraded":"2024-04-25T13:58:38.163379+0000","last_fullsized":"2024-04-25T13:58:38.163379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:48.345437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209853+0000","last_change":"2024-04-25T13:57:11.131160+0000","last_active":"2024-04-25T13:58:09.209853+0000","last_peered":"2024-04-25T13:58:09.209853+0000","last_clean":"2024-04-25T13:58:09.209853+0000","last_became_active":"2024-04-25T13:57:11.130753+0000","last_became_peered":"2024-04-25T13:57:11.130753+0000","last_unstale":"2024-04-25T13:58:09.209853+0000","last_undegraded":"2024-04-25T13:58:09.209853+0000","last_fullsized":"2024-04-25T13:58:09.209853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:48.297546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.156739+0000","last_change":"2024-04-25T13:57:12.155779+0000","last_active":"2024-04-25T13:58:38.156739+0000","last_peered":"2024-04-25T13:58:38.156739+0000","last_clean":"2024-04-25T13:58:38.156739+0000","last_became_active":"2024-04-25T13:57:12.154682+0000","last_became_peered":"2024-04-25T13:57:12.154682+0000","last_unstale":"2024-04-25T13:58:38.156739+0000","last_undegraded":"2024-04-25T13:58:38.156739+0000","last_fullsized":"2024-04-25T13:58:38.156739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:20.605425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211839+0000","last_change":"2024-04-25T13:57:11.140211+0000","last_active":"2024-04-25T13:58:09.211839+0000","last_peered":"2024-04-25T13:58:09.211839+0000","last_clean":"2024-04-25T13:58:09.211839+0000","last_became_active":"2024-04-25T13:57:11.139891+0000","last_became_peered":"2024-04-25T13:57:11.139891+0000","last_unstale":"2024-04-25T13:58:09.211839+0000","last_undegraded":"2024-04-25T13:58:09.211839+0000","last_fullsized":"2024-04-25T13:58:09.211839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:21.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.159292+0000","last_change":"2024-04-25T13:57:12.153886+0000","last_active":"2024-04-25T13:58:38.159292+0000","last_peered":"2024-04-25T13:58:38.159292+0000","last_clean":"2024-04-25T13:58:38.159292+0000","last_became_active":"2024-04-25T13:57:12.153500+0000","last_became_peered":"2024-04-25T13:57:12.153500+0000","last_unstale":"2024-04-25T13:58:38.159292+0000","last_undegraded":"2024-04-25T13:58:38.159292+0000","last_fullsized":"2024-04-25T13:58:38.159292+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:53.549745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:09.318464+0000","last_change":"2024-04-25T13:57:11.138829+0000","last_active":"2024-04-25T13:59:09.318464+0000","last_peered":"2024-04-25T13:59:09.318464+0000","last_clean":"2024-04-25T13:59:09.318464+0000","last_became_active":"2024-04-25T13:57:11.138461+0000","last_became_peered":"2024-04-25T13:57:11.138461+0000","last_unstale":"2024-04-25T13:59:09.318464+0000","last_undegraded":"2024-04-25T13:59:09.318464+0000","last_fullsized":"2024-04-25T13:59:09.318464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.134061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177713+0000","last_change":"2024-04-25T13:57:12.157394+0000","last_active":"2024-04-25T13:58:38.177713+0000","last_peered":"2024-04-25T13:58:38.177713+0000","last_clean":"2024-04-25T13:58:38.177713+0000","last_became_active":"2024-04-25T13:57:12.156408+0000","last_became_peered":"2024-04-25T13:57:12.156408+0000","last_unstale":"2024-04-25T13:58:38.177713+0000","last_undegraded":"2024-04-25T13:58:38.177713+0000","last_fullsized":"2024-04-25T13:58:38.177713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:36.741367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208928+0000","last_change":"2024-04-25T13:57:11.142846+0000","last_active":"2024-04-25T13:58:09.208928+0000","last_peered":"2024-04-25T13:58:09.208928+0000","last_clean":"2024-04-25T13:58:09.208928+0000","last_became_active":"2024-04-25T13:57:11.142387+0000","last_became_peered":"2024-04-25T13:57:11.142387+0000","last_unstale":"2024-04-25T13:58:09.208928+0000","last_undegraded":"2024-04-25T13:58:09.208928+0000","last_fullsized":"2024-04-25T13:58:09.208928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.164994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163650+0000","last_change":"2024-04-25T13:57:12.145617+0000","last_active":"2024-04-25T13:58:38.163650+0000","last_peered":"2024-04-25T13:58:38.163650+0000","last_clean":"2024-04-25T13:58:38.163650+0000","last_became_active":"2024-04-25T13:57:12.145263+0000","last_became_peered":"2024-04-25T13:57:12.145263+0000","last_unstale":"2024-04-25T13:58:38.163650+0000","last_undegraded":"2024-04-25T13:58:38.163650+0000","last_fullsized":"2024-04-25T13:58:38.163650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:43.886538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.212101+0000","last_change":"2024-04-25T13:57:11.136972+0000","last_active":"2024-04-25T13:58:09.212101+0000","last_peered":"2024-04-25T13:58:09.212101+0000","last_clean":"2024-04-25T13:58:09.212101+0000","last_became_active":"2024-04-25T13:57:11.136392+0000","last_became_peered":"2024-04-25T13:57:11.136392+0000","last_unstale":"2024-04-25T13:58:09.212101+0000","last_undegraded":"2024-04-25T13:58:09.212101+0000","last_fullsized":"2024-04-25T13:58:09.212101+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:45.841670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177590+0000","last_change":"2024-04-25T13:57:12.144905+0000","last_active":"2024-04-25T13:58:38.177590+0000","last_peered":"2024-04-25T13:58:38.177590+0000","last_clean":"2024-04-25T13:58:38.177590+0000","last_became_active":"2024-04-25T13:57:12.144404+0000","last_became_peered":"2024-04-25T13:57:12.144404+0000","last_unstale":"2024-04-25T13:58:38.177590+0000","last_undegraded":"2024-04-25T13:58:38.177590+0000","last_fullsized":"2024-04-25T13:58:38.177590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.186492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'31","reported_seq":291,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:10.306989+0000","last_change":"2024-04-25T13:57:11.139041+0000","last_active":"2024-04-25T13:59:10.306989+0000","last_peered":"2024-04-25T13:59:10.306989+0000","last_clean":"2024-04-25T13:59:10.306989+0000","last_became_active":"2024-04-25T13:57:11.138629+0000","last_became_peered":"2024-04-25T13:57:11.138629+0000","last_unstale":"2024-04-25T13:59:10.306989+0000","last_undegraded":"2024-04-25T13:59:10.306989+0000","last_fullsized":"2024-04-25T13:59:10.306989+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:04.863015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":180,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177642+0000","last_change":"2024-04-25T13:57:12.160897+0000","last_active":"2024-04-25T13:58:38.177642+0000","last_peered":"2024-04-25T13:58:38.177642+0000","last_clean":"2024-04-25T13:58:38.177642+0000","last_became_active":"2024-04-25T13:57:12.160322+0000","last_became_peered":"2024-04-25T13:57:12.160322+0000","last_unstale":"2024-04-25T13:58:38.177642+0000","last_undegraded":"2024-04-25T13:58:38.177642+0000","last_fullsized":"2024-04-25T13:58:38.177642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:26.180324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216914+0000","last_change":"2024-04-25T13:57:11.137104+0000","last_active":"2024-04-25T13:58:09.216914+0000","last_peered":"2024-04-25T13:58:09.216914+0000","last_clean":"2024-04-25T13:58:09.216914+0000","last_became_active":"2024-04-25T13:57:11.136720+0000","last_became_peered":"2024-04-25T13:57:11.136720+0000","last_unstale":"2024-04-25T13:58:09.216914+0000","last_undegraded":"2024-04-25T13:58:09.216914+0000","last_fullsized":"2024-04-25T13:58:09.216914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:41.262776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177140+0000","last_change":"2024-04-25T13:57:12.147831+0000","last_active":"2024-04-25T13:58:38.177140+0000","last_peered":"2024-04-25T13:58:38.177140+0000","last_clean":"2024-04-25T13:58:38.177140+0000","last_became_active":"2024-04-25T13:57:12.147270+0000","last_became_peered":"2024-04-25T13:57:12.147270+0000","last_unstale":"2024-04-25T13:58:38.177140+0000","last_undegraded":"2024-04-25T13:58:38.177140+0000","last_fullsized":"2024-04-25T13:58:38.177140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:31.574661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211663+0000","last_change":"2024-04-25T13:57:11.138254+0000","last_active":"2024-04-25T13:58:09.211663+0000","last_peered":"2024-04-25T13:58:09.211663+0000","last_clean":"2024-04-25T13:58:09.211663+0000","last_became_active":"2024-04-25T13:57:11.137924+0000","last_became_peered":"2024-04-25T13:57:11.137924+0000","last_unstale":"2024-04-25T13:58:09.211663+0000","last_undegraded":"2024-04-25T13:58:09.211663+0000","last_fullsized":"2024-04-25T13:58:09.211663+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:27.746203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179497+0000","last_change":"2024-04-25T13:57:12.145743+0000","last_active":"2024-04-25T13:58:38.179497+0000","last_peered":"2024-04-25T13:58:38.179497+0000","last_clean":"2024-04-25T13:58:38.179497+0000","last_became_active":"2024-04-25T13:57:12.145337+0000","last_became_peered":"2024-04-25T13:57:12.145337+0000","last_unstale":"2024-04-25T13:58:38.179497+0000","last_undegraded":"2024-04-25T13:58:38.179497+0000","last_fullsized":"2024-04-25T13:58:38.179497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.692900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'6","reported_seq":146,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.269043+0000","last_change":"2024-04-25T13:57:11.138059+0000","last_active":"2024-04-25T13:59:11.269043+0000","last_peered":"2024-04-25T13:59:11.269043+0000","last_clean":"2024-04-25T13:59:11.269043+0000","last_became_active":"2024-04-25T13:57:11.137698+0000","last_became_peered":"2024-04-25T13:57:11.137698+0000","last_unstale":"2024-04-25T13:59:11.269043+0000","last_undegraded":"2024-04-25T13:59:11.269043+0000","last_fullsized":"2024-04-25T13:59:11.269043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:03.595126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.171572+0000","last_change":"2024-04-25T13:57:12.155507+0000","last_active":"2024-04-25T13:58:38.171572+0000","last_peered":"2024-04-25T13:58:38.171572+0000","last_clean":"2024-04-25T13:58:38.171572+0000","last_became_active":"2024-04-25T13:57:12.155175+0000","last_became_peered":"2024-04-25T13:57:12.155175+0000","last_unstale":"2024-04-25T13:58:38.171572+0000","last_undegraded":"2024-04-25T13:58:38.171572+0000","last_fullsized":"2024-04-25T13:58:38.171572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:21.294649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'15","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208247+0000","last_change":"2024-04-25T13:57:11.141672+0000","last_active":"2024-04-25T13:58:09.208247+0000","last_peered":"2024-04-25T13:58:09.208247+0000","last_clean":"2024-04-25T13:58:09.208247+0000","last_became_active":"2024-04-25T13:57:11.140957+0000","last_became_peered":"2024-04-25T13:57:11.140957+0000","last_unstale":"2024-04-25T13:58:09.208247+0000","last_undegraded":"2024-04-25T13:58:09.208247+0000","last_fullsized":"2024-04-25T13:58:09.208247+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:47.362949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.165834+0000","last_change":"2024-04-25T13:57:12.147881+0000","last_active":"2024-04-25T13:58:38.165834+0000","last_peered":"2024-04-25T13:58:38.165834+0000","last_clean":"2024-04-25T13:58:38.165834+0000","last_became_active":"2024-04-25T13:57:12.146200+0000","last_became_peered":"2024-04-25T13:57:12.146200+0000","last_unstale":"2024-04-25T13:58:38.165834+0000","last_undegraded":"2024-04-25T13:58:38.165834+0000","last_fullsized":"2024-04-25T13:58:38.165834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:51.711045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.209588+0000","last_change":"2024-04-25T13:57:11.139574+0000","last_active":"2024-04-25T13:58:09.209588+0000","last_peered":"2024-04-25T13:58:09.209588+0000","last_clean":"2024-04-25T13:58:09.209588+0000","last_became_active":"2024-04-25T13:57:11.139232+0000","last_became_peered":"2024-04-25T13:57:11.139232+0000","last_unstale":"2024-04-25T13:58:09.209588+0000","last_undegraded":"2024-04-25T13:58:09.209588+0000","last_fullsized":"2024-04-25T13:58:09.209588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:08.590270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.162405+0000","last_change":"2024-04-25T13:57:12.147122+0000","last_active":"2024-04-25T13:58:38.162405+0000","last_peered":"2024-04-25T13:58:38.162405+0000","last_clean":"2024-04-25T13:58:38.162405+0000","last_became_active":"2024-04-25T13:57:12.146708+0000","last_became_peered":"2024-04-25T13:57:12.146708+0000","last_unstale":"2024-04-25T13:58:38.162405+0000","last_undegraded":"2024-04-25T13:58:38.162405+0000","last_fullsized":"2024-04-25T13:58:38.162405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:47.858290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.205903+0000","last_change":"2024-04-25T13:57:11.146288+0000","last_active":"2024-04-25T13:58:09.205903+0000","last_peered":"2024-04-25T13:58:09.205903+0000","last_clean":"2024-04-25T13:58:09.205903+0000","last_became_active":"2024-04-25T13:57:11.145851+0000","last_became_peered":"2024-04-25T13:57:11.145851+0000","last_unstale":"2024-04-25T13:58:09.205903+0000","last_undegraded":"2024-04-25T13:58:09.205903+0000","last_fullsized":"2024-04-25T13:58:09.205903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.011875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155416+0000","last_change":"2024-04-25T13:57:12.154576+0000","last_active":"2024-04-25T13:58:38.155416+0000","last_peered":"2024-04-25T13:58:38.155416+0000","last_clean":"2024-04-25T13:58:38.155416+0000","last_became_active":"2024-04-25T13:57:12.154254+0000","last_became_peered":"2024-04-25T13:57:12.154254+0000","last_unstale":"2024-04-25T13:58:38.155416+0000","last_undegraded":"2024-04-25T13:58:38.155416+0000","last_fullsized":"2024-04-25T13:58:38.155416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:33.774459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208334+0000","last_change":"2024-04-25T13:57:11.139941+0000","last_active":"2024-04-25T13:58:09.208334+0000","last_peered":"2024-04-25T13:58:09.208334+0000","last_clean":"2024-04-25T13:58:09.208334+0000","last_became_active":"2024-04-25T13:57:11.139614+0000","last_became_peered":"2024-04-25T13:57:11.139614+0000","last_unstale":"2024-04-25T13:58:09.208334+0000","last_undegraded":"2024-04-25T13:58:09.208334+0000","last_fullsized":"2024-04-25T13:58:09.208334+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:39.656085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.164620+0000","last_change":"2024-04-25T13:57:12.155935+0000","last_active":"2024-04-25T13:58:38.164620+0000","last_peered":"2024-04-25T13:58:38.164620+0000","last_clean":"2024-04-25T13:58:38.164620+0000","last_became_active":"2024-04-25T13:57:12.155261+0000","last_became_peered":"2024-04-25T13:57:12.155261+0000","last_unstale":"2024-04-25T13:58:38.164620+0000","last_undegraded":"2024-04-25T13:58:38.164620+0000","last_fullsized":"2024-04-25T13:58:38.164620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:54.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.156931+0000","last_change":"2024-04-25T13:58:09.217792+0000","last_active":"2024-04-25T13:58:38.156931+0000","last_peered":"2024-04-25T13:58:38.156931+0000","last_clean":"2024-04-25T13:58:38.156931+0000","last_became_active":"2024-04-25T13:58:09.217386+0000","last_became_peered":"2024-04-25T13:58:09.217386+0000","last_unstale":"2024-04-25T13:58:38.156931+0000","last_undegraded":"2024-04-25T13:58:38.156931+0000","last_fullsized":"2024-04-25T13:58:38.156931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:13.833587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"77'6","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:59:11.268314+0000","last_change":"2024-04-25T13:57:11.135226+0000","last_active":"2024-04-25T13:59:11.268314+0000","last_peered":"2024-04-25T13:59:11.268314+0000","last_clean":"2024-04-25T13:59:11.268314+0000","last_became_active":"2024-04-25T13:57:11.134886+0000","last_became_peered":"2024-04-25T13:57:11.134886+0000","last_unstale":"2024-04-25T13:59:11.268314+0000","last_undegraded":"2024-04-25T13:59:11.268314+0000","last_fullsized":"2024-04-25T13:59:11.268314+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:00.153597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.173674+0000","last_change":"2024-04-25T13:57:12.149758+0000","last_active":"2024-04-25T13:58:38.173674+0000","last_peered":"2024-04-25T13:58:38.173674+0000","last_clean":"2024-04-25T13:58:38.173674+0000","last_became_active":"2024-04-25T13:57:12.149428+0000","last_became_peered":"2024-04-25T13:57:12.149428+0000","last_unstale":"2024-04-25T13:58:38.173674+0000","last_undegraded":"2024-04-25T13:58:38.173674+0000","last_fullsized":"2024-04-25T13:58:38.173674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:46.739074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207722+0000","last_change":"2024-04-25T13:57:11.135968+0000","last_active":"2024-04-25T13:58:09.207722+0000","last_peered":"2024-04-25T13:58:09.207722+0000","last_clean":"2024-04-25T13:58:09.207722+0000","last_became_active":"2024-04-25T13:57:11.135577+0000","last_became_peered":"2024-04-25T13:57:11.135577+0000","last_unstale":"2024-04-25T13:58:09.207722+0000","last_undegraded":"2024-04-25T13:58:09.207722+0000","last_fullsized":"2024-04-25T13:58:09.207722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:00.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208486+0000","last_change":"2024-04-25T13:55:57.666399+0000","last_active":"2024-04-25T13:58:09.208486+0000","last_peered":"2024-04-25T13:58:09.208486+0000","last_clean":"2024-04-25T13:58:09.208486+0000","last_became_active":"2024-04-25T13:55:57.660470+0000","last_became_peered":"2024-04-25T13:55:57.660470+0000","last_unstale":"2024-04-25T13:58:09.208486+0000","last_undegraded":"2024-04-25T13:58:09.208486+0000","last_fullsized":"2024-04-25T13:58:09.208486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:52:33.081441+0000","last_clean_scrub_stamp":"2024-04-25T13:52:33.081441+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:22.788632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163242+0000","last_change":"2024-04-25T13:57:12.141923+0000","last_active":"2024-04-25T13:58:38.163242+0000","last_peered":"2024-04-25T13:58:38.163242+0000","last_clean":"2024-04-25T13:58:38.163242+0000","last_became_active":"2024-04-25T13:57:12.140946+0000","last_became_peered":"2024-04-25T13:57:12.140946+0000","last_unstale":"2024-04-25T13:58:38.163242+0000","last_undegraded":"2024-04-25T13:58:38.163242+0000","last_fullsized":"2024-04-25T13:58:38.163242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:49.294598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208387+0000","last_change":"2024-04-25T13:57:11.139851+0000","last_active":"2024-04-25T13:58:09.208387+0000","last_peered":"2024-04-25T13:58:09.208387+0000","last_clean":"2024-04-25T13:58:09.208387+0000","last_became_active":"2024-04-25T13:57:11.139476+0000","last_became_peered":"2024-04-25T13:57:11.139476+0000","last_unstale":"2024-04-25T13:58:09.208387+0000","last_undegraded":"2024-04-25T13:58:09.208387+0000","last_fullsized":"2024-04-25T13:58:09.208387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:29.435917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.220107+0000","last_change":"2024-04-25T13:58:09.219619+0000","last_active":"2024-04-25T13:58:09.220107+0000","last_peered":"2024-04-25T13:58:09.220107+0000","last_clean":"2024-04-25T13:58:09.220107+0000","last_became_active":"2024-04-25T13:58:09.211808+0000","last_became_peered":"2024-04-25T13:58:09.211808+0000","last_unstale":"2024-04-25T13:58:09.220107+0000","last_undegraded":"2024-04-25T13:58:09.220107+0000","last_fullsized":"2024-04-25T13:58:09.220107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:20.208191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.157529+0000","last_change":"2024-04-25T13:57:12.136204+0000","last_active":"2024-04-25T13:58:38.157529+0000","last_peered":"2024-04-25T13:58:38.157529+0000","last_clean":"2024-04-25T13:58:38.157529+0000","last_became_active":"2024-04-25T13:57:12.135798+0000","last_became_peered":"2024-04-25T13:57:12.135798+0000","last_unstale":"2024-04-25T13:58:38.157529+0000","last_undegraded":"2024-04-25T13:58:38.157529+0000","last_fullsized":"2024-04-25T13:58:38.157529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:36.894774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'9","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.206151+0000","last_change":"2024-04-25T13:57:11.138699+0000","last_active":"2024-04-25T13:58:09.206151+0000","last_peered":"2024-04-25T13:58:09.206151+0000","last_clean":"2024-04-25T13:58:09.206151+0000","last_became_active":"2024-04-25T13:57:11.138335+0000","last_became_peered":"2024-04-25T13:57:11.138335+0000","last_unstale":"2024-04-25T13:58:09.206151+0000","last_undegraded":"2024-04-25T13:58:09.206151+0000","last_fullsized":"2024-04-25T13:58:09.206151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:44.413228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.171184+0000","last_change":"2024-04-25T13:57:12.155851+0000","last_active":"2024-04-25T13:58:38.171184+0000","last_peered":"2024-04-25T13:58:38.171184+0000","last_clean":"2024-04-25T13:58:38.171184+0000","last_became_active":"2024-04-25T13:57:12.155416+0000","last_became_peered":"2024-04-25T13:57:12.155416+0000","last_unstale":"2024-04-25T13:58:38.171184+0000","last_undegraded":"2024-04-25T13:58:38.171184+0000","last_fullsized":"2024-04-25T13:58:38.171184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:22.406338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.207744+0000","last_change":"2024-04-25T13:57:11.147465+0000","last_active":"2024-04-25T13:58:09.207744+0000","last_peered":"2024-04-25T13:58:09.207744+0000","last_clean":"2024-04-25T13:58:09.207744+0000","last_became_active":"2024-04-25T13:57:11.147096+0000","last_became_peered":"2024-04-25T13:57:11.147096+0000","last_unstale":"2024-04-25T13:58:09.207744+0000","last_undegraded":"2024-04-25T13:58:09.207744+0000","last_fullsized":"2024-04-25T13:58:09.207744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:19.229356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163274+0000","last_change":"2024-04-25T13:57:12.146446+0000","last_active":"2024-04-25T13:58:38.163274+0000","last_peered":"2024-04-25T13:58:38.163274+0000","last_clean":"2024-04-25T13:58:38.163274+0000","last_became_active":"2024-04-25T13:57:12.146129+0000","last_became_peered":"2024-04-25T13:57:12.146129+0000","last_unstale":"2024-04-25T13:58:38.163274+0000","last_undegraded":"2024-04-25T13:58:38.163274+0000","last_fullsized":"2024-04-25T13:58:38.163274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:49.977663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.325014+0000","last_change":"2024-04-25T13:57:11.145127+0000","last_active":"2024-04-25T13:58:09.325014+0000","last_peered":"2024-04-25T13:58:09.325014+0000","last_clean":"2024-04-25T13:58:09.325014+0000","last_became_active":"2024-04-25T13:57:11.144822+0000","last_became_peered":"2024-04-25T13:57:11.144822+0000","last_unstale":"2024-04-25T13:58:09.325014+0000","last_undegraded":"2024-04-25T13:58:09.325014+0000","last_fullsized":"2024-04-25T13:58:09.325014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:00.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.121092+0000","last_change":"2024-04-25T13:57:12.156184+0000","last_active":"2024-04-25T13:58:38.121092+0000","last_peered":"2024-04-25T13:58:38.121092+0000","last_clean":"2024-04-25T13:58:38.121092+0000","last_became_active":"2024-04-25T13:57:12.155855+0000","last_became_peered":"2024-04-25T13:57:12.155855+0000","last_unstale":"2024-04-25T13:58:38.121092+0000","last_undegraded":"2024-04-25T13:58:38.121092+0000","last_fullsized":"2024-04-25T13:58:38.121092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:00.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.213952+0000","last_change":"2024-04-25T13:57:11.137616+0000","last_active":"2024-04-25T13:58:09.213952+0000","last_peered":"2024-04-25T13:58:09.213952+0000","last_clean":"2024-04-25T13:58:09.213952+0000","last_became_active":"2024-04-25T13:57:11.137041+0000","last_became_peered":"2024-04-25T13:57:11.137041+0000","last_unstale":"2024-04-25T13:58:09.213952+0000","last_undegraded":"2024-04-25T13:58:09.213952+0000","last_fullsized":"2024-04-25T13:58:09.213952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:54.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.159060+0000","last_change":"2024-04-25T13:57:12.151223+0000","last_active":"2024-04-25T13:58:38.159060+0000","last_peered":"2024-04-25T13:58:38.159060+0000","last_clean":"2024-04-25T13:58:38.159060+0000","last_became_active":"2024-04-25T13:57:12.150871+0000","last_became_peered":"2024-04-25T13:57:12.150871+0000","last_unstale":"2024-04-25T13:58:38.159060+0000","last_undegraded":"2024-04-25T13:58:38.159060+0000","last_fullsized":"2024-04-25T13:58:38.159060+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:17.957728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.216414+0000","last_change":"2024-04-25T13:57:11.136524+0000","last_active":"2024-04-25T13:58:09.216414+0000","last_peered":"2024-04-25T13:58:09.216414+0000","last_clean":"2024-04-25T13:58:09.216414+0000","last_became_active":"2024-04-25T13:57:11.136152+0000","last_became_peered":"2024-04-25T13:57:11.136152+0000","last_unstale":"2024-04-25T13:58:09.216414+0000","last_undegraded":"2024-04-25T13:58:09.216414+0000","last_fullsized":"2024-04-25T13:58:09.216414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:23.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160886+0000","last_change":"2024-04-25T13:57:12.157641+0000","last_active":"2024-04-25T13:58:38.160886+0000","last_peered":"2024-04-25T13:58:38.160886+0000","last_clean":"2024-04-25T13:58:38.160886+0000","last_became_active":"2024-04-25T13:57:12.157281+0000","last_became_peered":"2024-04-25T13:57:12.157281+0000","last_unstale":"2024-04-25T13:58:38.160886+0000","last_undegraded":"2024-04-25T13:58:38.160886+0000","last_fullsized":"2024-04-25T13:58:38.160886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:28.245071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.208565+0000","last_change":"2024-04-25T13:57:11.139036+0000","last_active":"2024-04-25T13:58:09.208565+0000","last_peered":"2024-04-25T13:58:09.208565+0000","last_clean":"2024-04-25T13:58:09.208565+0000","last_became_active":"2024-04-25T13:57:11.138681+0000","last_became_peered":"2024-04-25T13:57:11.138681+0000","last_unstale":"2024-04-25T13:58:09.208565+0000","last_undegraded":"2024-04-25T13:58:09.208565+0000","last_fullsized":"2024-04-25T13:58:09.208565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:48.584313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180469+0000","last_change":"2024-04-25T13:57:12.153072+0000","last_active":"2024-04-25T13:58:38.180469+0000","last_peered":"2024-04-25T13:58:38.180469+0000","last_clean":"2024-04-25T13:58:38.180469+0000","last_became_active":"2024-04-25T13:57:12.151556+0000","last_became_peered":"2024-04-25T13:57:12.151556+0000","last_unstale":"2024-04-25T13:58:38.180469+0000","last_undegraded":"2024-04-25T13:58:38.180469+0000","last_fullsized":"2024-04-25T13:58:38.180469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:20.144339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.217008+0000","last_change":"2024-04-25T13:57:11.148823+0000","last_active":"2024-04-25T13:58:09.217008+0000","last_peered":"2024-04-25T13:58:09.217008+0000","last_clean":"2024-04-25T13:58:09.217008+0000","last_became_active":"2024-04-25T13:57:11.148436+0000","last_became_peered":"2024-04-25T13:57:11.148436+0000","last_unstale":"2024-04-25T13:58:09.217008+0000","last_undegraded":"2024-04-25T13:58:09.217008+0000","last_fullsized":"2024-04-25T13:58:09.217008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:28.496184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180714+0000","last_change":"2024-04-25T13:57:12.149082+0000","last_active":"2024-04-25T13:58:38.180714+0000","last_peered":"2024-04-25T13:58:38.180714+0000","last_clean":"2024-04-25T13:58:38.180714+0000","last_became_active":"2024-04-25T13:57:12.148260+0000","last_became_peered":"2024-04-25T13:57:12.148260+0000","last_unstale":"2024-04-25T13:58:38.180714+0000","last_undegraded":"2024-04-25T13:58:38.180714+0000","last_fullsized":"2024-04-25T13:58:38.180714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:49.733114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.211832+0000","last_change":"2024-04-25T13:57:11.136736+0000","last_active":"2024-04-25T13:58:09.211832+0000","last_peered":"2024-04-25T13:58:09.211832+0000","last_clean":"2024-04-25T13:58:09.211832+0000","last_became_active":"2024-04-25T13:57:11.135831+0000","last_became_peered":"2024-04-25T13:57:11.135831+0000","last_unstale":"2024-04-25T13:58:09.211832+0000","last_undegraded":"2024-04-25T13:58:09.211832+0000","last_fullsized":"2024-04-25T13:58:09.211832+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:49.654642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.175566+0000","last_change":"2024-04-25T13:57:12.149797+0000","last_active":"2024-04-25T13:58:38.175566+0000","last_peered":"2024-04-25T13:58:38.175566+0000","last_clean":"2024-04-25T13:58:38.175566+0000","last_became_active":"2024-04-25T13:57:12.149413+0000","last_became_peered":"2024-04-25T13:57:12.149413+0000","last_unstale":"2024-04-25T13:58:38.175566+0000","last_undegraded":"2024-04-25T13:58:38.175566+0000","last_fullsized":"2024-04-25T13:58:38.175566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:58.517208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:09.328809+0000","last_change":"2024-04-25T13:58:09.328809+0000","last_active":"2024-04-25T13:58:09.328809+0000","last_peered":"2024-04-25T13:58:09.328809+0000","last_clean":"2024-04-25T13:58:09.328809+0000","last_became_active":"2024-04-25T13:58:09.326068+0000","last_became_peered":"2024-04-25T13:58:09.326068+0000","last_unstale":"2024-04-25T13:58:09.328809+0000","last_undegraded":"2024-04-25T13:58:09.328809+0000","last_fullsized":"2024-04-25T13:58:09.328809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:10.113645+0000","last_clean_scrub_stamp":"2024-04-25T13:57:10.113645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:14.624295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.176620+0000","last_change":"2024-04-25T13:57:12.150247+0000","last_active":"2024-04-25T13:58:38.176620+0000","last_peered":"2024-04-25T13:58:38.176620+0000","last_clean":"2024-04-25T13:58:38.176620+0000","last_became_active":"2024-04-25T13:57:12.149882+0000","last_became_peered":"2024-04-25T13:57:12.149882+0000","last_unstale":"2024-04-25T13:58:38.176620+0000","last_undegraded":"2024-04-25T13:58:38.176620+0000","last_fullsized":"2024-04-25T13:58:38.176620+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.624928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160753+0000","last_change":"2024-04-25T13:57:12.154618+0000","last_active":"2024-04-25T13:58:38.160753+0000","last_peered":"2024-04-25T13:58:38.160753+0000","last_clean":"2024-04-25T13:58:38.160753+0000","last_became_active":"2024-04-25T13:57:12.153648+0000","last_became_peered":"2024-04-25T13:57:12.153648+0000","last_unstale":"2024-04-25T13:58:38.160753+0000","last_undegraded":"2024-04-25T13:58:38.160753+0000","last_fullsized":"2024-04-25T13:58:38.160753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:24.010980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160683+0000","last_change":"2024-04-25T13:57:12.143097+0000","last_active":"2024-04-25T13:58:38.160683+0000","last_peered":"2024-04-25T13:58:38.160683+0000","last_clean":"2024-04-25T13:58:38.160683+0000","last_became_active":"2024-04-25T13:57:12.142295+0000","last_became_peered":"2024-04-25T13:57:12.142295+0000","last_unstale":"2024-04-25T13:58:38.160683+0000","last_undegraded":"2024-04-25T13:58:38.160683+0000","last_fullsized":"2024-04-25T13:58:38.160683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:31.907643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.160759+0000","last_change":"2024-04-25T13:57:12.155659+0000","last_active":"2024-04-25T13:58:38.160759+0000","last_peered":"2024-04-25T13:58:38.160759+0000","last_clean":"2024-04-25T13:58:38.160759+0000","last_became_active":"2024-04-25T13:57:12.155308+0000","last_became_peered":"2024-04-25T13:57:12.155308+0000","last_unstale":"2024-04-25T13:58:38.160759+0000","last_undegraded":"2024-04-25T13:58:38.160759+0000","last_fullsized":"2024-04-25T13:58:38.160759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:12.361660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.177243+0000","last_change":"2024-04-25T13:58:09.212066+0000","last_active":"2024-04-25T13:58:38.177243+0000","last_peered":"2024-04-25T13:58:38.177243+0000","last_clean":"2024-04-25T13:58:38.177243+0000","last_became_active":"2024-04-25T13:58:09.211708+0000","last_became_peered":"2024-04-25T13:58:09.211708+0000","last_unstale":"2024-04-25T13:58:38.177243+0000","last_undegraded":"2024-04-25T13:58:38.177243+0000","last_fullsized":"2024-04-25T13:58:38.177243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:34.585999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.163940+0000","last_change":"2024-04-25T13:57:12.142134+0000","last_active":"2024-04-25T13:58:38.163940+0000","last_peered":"2024-04-25T13:58:38.163940+0000","last_clean":"2024-04-25T13:58:38.163940+0000","last_became_active":"2024-04-25T13:57:12.141776+0000","last_became_peered":"2024-04-25T13:57:12.141776+0000","last_unstale":"2024-04-25T13:58:38.163940+0000","last_undegraded":"2024-04-25T13:58:38.163940+0000","last_fullsized":"2024-04-25T13:58:38.163940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:07.621761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158589+0000","last_change":"2024-04-25T13:58:09.213725+0000","last_active":"2024-04-25T13:58:38.158589+0000","last_peered":"2024-04-25T13:58:38.158589+0000","last_clean":"2024-04-25T13:58:38.158589+0000","last_became_active":"2024-04-25T13:58:09.213229+0000","last_became_peered":"2024-04-25T13:58:09.213229+0000","last_unstale":"2024-04-25T13:58:38.158589+0000","last_undegraded":"2024-04-25T13:58:38.158589+0000","last_fullsized":"2024-04-25T13:58:38.158589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:14.728468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.179484+0000","last_change":"2024-04-25T13:57:12.147818+0000","last_active":"2024-04-25T13:58:38.179484+0000","last_peered":"2024-04-25T13:58:38.179484+0000","last_clean":"2024-04-25T13:58:38.179484+0000","last_became_active":"2024-04-25T13:57:12.146764+0000","last_became_peered":"2024-04-25T13:57:12.146764+0000","last_unstale":"2024-04-25T13:58:38.179484+0000","last_undegraded":"2024-04-25T13:58:38.179484+0000","last_fullsized":"2024-04-25T13:58:38.179484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.750115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.158703+0000","last_change":"2024-04-25T13:57:12.152591+0000","last_active":"2024-04-25T13:58:38.158703+0000","last_peered":"2024-04-25T13:58:38.158703+0000","last_clean":"2024-04-25T13:58:38.158703+0000","last_became_active":"2024-04-25T13:57:12.152198+0000","last_became_peered":"2024-04-25T13:57:12.152198+0000","last_unstale":"2024-04-25T13:58:38.158703+0000","last_undegraded":"2024-04-25T13:58:38.158703+0000","last_fullsized":"2024-04-25T13:58:38.158703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:05.868774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.157434+0000","last_change":"2024-04-25T13:57:12.154924+0000","last_active":"2024-04-25T13:58:38.157434+0000","last_peered":"2024-04-25T13:58:38.157434+0000","last_clean":"2024-04-25T13:58:38.157434+0000","last_became_active":"2024-04-25T13:57:12.154549+0000","last_became_peered":"2024-04-25T13:57:12.154549+0000","last_unstale":"2024-04-25T13:58:38.157434+0000","last_undegraded":"2024-04-25T13:58:38.157434+0000","last_fullsized":"2024-04-25T13:58:38.157434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:49.418436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.155858+0000","last_change":"2024-04-25T13:58:09.216933+0000","last_active":"2024-04-25T13:58:38.155858+0000","last_peered":"2024-04-25T13:58:38.155858+0000","last_clean":"2024-04-25T13:58:38.155858+0000","last_became_active":"2024-04-25T13:58:09.216615+0000","last_became_peered":"2024-04-25T13:58:09.216615+0000","last_unstale":"2024-04-25T13:58:38.155858+0000","last_undegraded":"2024-04-25T13:58:38.155858+0000","last_fullsized":"2024-04-25T13:58:38.155858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:12:25.597023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.180794+0000","last_change":"2024-04-25T13:57:12.158672+0000","last_active":"2024-04-25T13:58:38.180794+0000","last_peered":"2024-04-25T13:58:38.180794+0000","last_clean":"2024-04-25T13:58:38.180794+0000","last_became_active":"2024-04-25T13:57:12.158391+0000","last_became_peered":"2024-04-25T13:57:12.158391+0000","last_unstale":"2024-04-25T13:58:38.180794+0000","last_undegraded":"2024-04-25T13:58:38.180794+0000","last_fullsized":"2024-04-25T13:58:38.180794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.369907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.178211+0000","last_change":"2024-04-25T13:57:12.139310+0000","last_active":"2024-04-25T13:58:38.178211+0000","last_peered":"2024-04-25T13:58:38.178211+0000","last_clean":"2024-04-25T13:58:38.178211+0000","last_became_active":"2024-04-25T13:57:12.138942+0000","last_became_peered":"2024-04-25T13:57:12.138942+0000","last_unstale":"2024-04-25T13:58:38.178211+0000","last_undegraded":"2024-04-25T13:58:38.178211+0000","last_fullsized":"2024-04-25T13:58:38.178211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:01.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T13:58:38.162465+0000","last_change":"2024-04-25T13:57:12.147451+0000","last_active":"2024-04-25T13:58:38.162465+0000","last_peered":"2024-04-25T13:58:38.162465+0000","last_clean":"2024-04-25T13:58:38.162465+0000","last_became_active":"2024-04-25T13:57:12.146746+0000","last_became_peered":"2024-04-25T13:57:12.146746+0000","last_unstale":"2024-04-25T13:58:38.162465+0000","last_undegraded":"2024-04-25T13:58:38.162465+0000","last_fullsized":"2024-04-25T13:58:38.162465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:57:11.115671+0000","last_clean_scrub_stamp":"2024-04-25T13:57:11.115671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247568,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66277,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":728,"num_read_kb":435,"num_write":170,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1617,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":207767,"data_compressed":5577,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":835584,"data_stored":402153,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1054857,"data_compressed":43506,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":401785,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266742,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":839680,"data_stored":401683,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":993726,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626496,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":401174,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":466884,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":402445,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":467612,"data_compressed":7668,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":401485,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":843776,"data_stored":403605,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":905216,"data_stored":993460,"data_compressed":41647,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34189,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":65917,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66645,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63713,"data_compressed":1859,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:59:15.550 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:59:15.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 -- 172.21.15.78:0/1696728916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c108920 msgr2=0x7fe52c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 --2- 172.21.15.78:0/1696728916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c108920 0x7fe52c10ad40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe524009ca0 tx=0x7fe52402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 -- 172.21.15.78:0/1696728916 shutdown_connections 2024-04-25T13:59:15.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 --2- 172.21.15.78:0/1696728916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c108920 0x7fe52c10ad40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 --2- 172.21.15.78:0/1696728916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c103c20 0x7fe52c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 --2- 172.21.15.78:0/1696728916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe52c1029e0 0x7fe52c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 -- 172.21.15.78:0/1696728916 >> 172.21.15.78:0/1696728916 conn(0x7fe52c0fdbf0 msgr2=0x7fe52c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 -- 172.21.15.78:0/1696728916 shutdown_connections 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.695+0000 7fe532dc5640 1 -- 172.21.15.78:0/1696728916 wait complete. 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 Processor -- start 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 -- start start 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe52c1029e0 0x7fe52c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c103c20 0x7fe52c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c108920 0x7fe52c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c10e4f0 con 0x7fe52c103c20 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c10e370 con 0x7fe52c108920 2024-04-25T13:59:15.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe532dc5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c10e670 con 0x7fe52c1029e0 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe52bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c103c20 0x7fe52c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe52bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c103c20 0x7fe52c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43196/0 (socket says 172.21.15.78:43196) 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.696+0000 7fe52bfff640 1 -- 172.21.15.78:0/2186269584 learned_addr learned my addr 172.21.15.78:0/2186269584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe530b61640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe52c1029e0 0x7fe52c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe531362640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c108920 0x7fe52c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe52bfff640 1 -- 172.21.15.78:0/2186269584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe52c1029e0 msgr2=0x7fe52c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe52bfff640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe52c1029e0 0x7fe52c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe52bfff640 1 -- 172.21.15.78:0/2186269584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c108920 msgr2=0x7fe52c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe52bfff640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c108920 0x7fe52c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe52bfff640 1 -- 172.21.15.78:0/2186269584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c1a0bb0 con 0x7fe52c103c20 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe531362640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c108920 0x7fe52c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe52bfff640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c103c20 0x7fe52c199660 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe51800c920 tx=0x7fe51800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe529ffb640 1 -- 172.21.15.78:0/2186269584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518007c70 con 0x7fe52c103c20 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe529ffb640 1 -- 172.21.15.78:0/2186269584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe518007e10 con 0x7fe52c103c20 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe529ffb640 1 -- 172.21.15.78:0/2186269584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518018820 con 0x7fe52c103c20 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c1a0ea0 con 0x7fe52c103c20 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.697+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c1a1360 con 0x7fe52c103c20 2024-04-25T13:59:15.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.698+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52c107730 con 0x7fe52c103c20 2024-04-25T13:59:15.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.700+0000 7fe529ffb640 1 -- 172.21.15.78:0/2186269584 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe51800f450 con 0x7fe52c103c20 2024-04-25T13:59:15.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.700+0000 7fe529ffb640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe51c0afb70 0x7fe51c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:15.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.701+0000 7fe530b61640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe51c0afb70 0x7fe51c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:15.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.701+0000 7fe529ffb640 1 -- 172.21.15.78:0/2186269584 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe5180dedb0 con 0x7fe52c103c20 2024-04-25T13:59:15.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.701+0000 7fe530b61640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe51c0afb70 0x7fe51c0b2080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe520005fd0 tx=0x7fe520005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:15.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.705+0000 7fe529ffb640 1 -- 172.21.15.78:0/2186269584 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe518014030 con 0x7fe52c103c20 2024-04-25T13:59:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.883+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe52c0637d0 con 0x7fe52c103c20 2024-04-25T13:59:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.885+0000 7fe529ffb640 1 -- 172.21.15.78:0/2186269584 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7fe5180a7980 con 0x7fe52c103c20 2024-04-25T13:59:15.885 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:15.886 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:59:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe51c0afb70 msgr2=0x7fe51c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe51c0afb70 0x7fe51c0b2080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe520005fd0 tx=0x7fe520005850 comp rx=0 tx=0).stop 2024-04-25T13:59:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c103c20 msgr2=0x7fe52c199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c103c20 0x7fe52c199660 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe51800c920 tx=0x7fe51800cde0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 shutdown_connections 2024-04-25T13:59:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe52c108920 0x7fe52c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe51c0afb70 0x7fe51c0b2080 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c103c20 0x7fe52c199660 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 --2- 172.21.15.78:0/2186269584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe52c1029e0 0x7fe52c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 >> 172.21.15.78:0/2186269584 conn(0x7fe52c0fdbf0 msgr2=0x7fe52c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 shutdown_connections 2024-04-25T13:59:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:15.887+0000 7fe532dc5640 1 -- 172.21.15.78:0/2186269584 wait complete. 2024-04-25T13:59:15.897 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:15 smithi078 ceph-mon[29163]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:15.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:15 smithi078 ceph-mon[29163]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:15.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:59:15.901 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:59:15.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:15 smithi129 ceph-mon[32569]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:15.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:15 smithi129 ceph-mon[32569]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 -- 172.21.15.78:0/552702410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 msgr2=0x7f73801069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 --2- 172.21.15.78:0/552702410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f73801069f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f737c009900 tx=0x7f737c02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 -- 172.21.15.78:0/552702410 shutdown_connections 2024-04-25T13:59:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 --2- 172.21.15.78:0/552702410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7380109890 0x7f738010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 --2- 172.21.15.78:0/552702410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7380106f30 0x7f7380109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 --2- 172.21.15.78:0/552702410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f73801069f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 -- 172.21.15.78:0/552702410 >> 172.21.15.78:0/552702410 conn(0x7f73800fdbd0 msgr2=0x7f7380100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 -- 172.21.15.78:0/552702410 shutdown_connections 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.012+0000 7f7387e27640 1 -- 172.21.15.78:0/552702410 wait complete. 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 Processor -- start 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 -- start start 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f738019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7380106f30 0x7f738019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7380109890 0x7f73801a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f738010f5e0 con 0x7f73801045d0 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f738010f460 con 0x7f7380106f30 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7387e27640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f738010f760 con 0x7f7380109890 2024-04-25T13:59:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7385bc3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f738019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7385bc3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f738019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43228/0 (socket says 172.21.15.78:43228) 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f7385bc3640 1 -- 172.21.15.78:0/3777586344 learned_addr learned my addr 172.21.15.78:0/3777586344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f73863c4640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7380109890 0x7f73801a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.013+0000 7f73853c2640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7380106f30 0x7f738019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7385bc3640 1 -- 172.21.15.78:0/3777586344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7380109890 msgr2=0x7f73801a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7385bc3640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7380109890 0x7f73801a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7385bc3640 1 -- 172.21.15.78:0/3777586344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7380106f30 msgr2=0x7f738019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7385bc3640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7380106f30 0x7f738019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7385bc3640 1 -- 172.21.15.78:0/3777586344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73801a4310 con 0x7f73801045d0 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7385bc3640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f738019c7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f737c006010 tx=0x7f737c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7376ffd640 1 -- 172.21.15.78:0/3777586344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c03f030 con 0x7f73801045d0 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7376ffd640 1 -- 172.21.15.78:0/3777586344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f737c003aa0 con 0x7f73801045d0 2024-04-25T13:59:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73801a45a0 con 0x7f73801045d0 2024-04-25T13:59:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7376ffd640 1 -- 172.21.15.78:0/3777586344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c035070 con 0x7f73801045d0 2024-04-25T13:59:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.014+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73801a4a90 con 0x7f73801045d0 2024-04-25T13:59:16.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:15 smithi112 ceph-mon[32617]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:16.017 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:15 smithi112 ceph-mon[32617]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.015+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738006c2a0 con 0x7f73801045d0 2024-04-25T13:59:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.016+0000 7f7376ffd640 1 -- 172.21.15.78:0/3777586344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f737c042030 con 0x7f73801045d0 2024-04-25T13:59:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.017+0000 7f7376ffd640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f736c0afaf0 0x7f736c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.017+0000 7f73853c2640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f736c0afaf0 0x7f736c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.017+0000 7f7376ffd640 1 -- 172.21.15.78:0/3777586344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f737c0ff140 con 0x7f73801045d0 2024-04-25T13:59:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.018+0000 7f73853c2640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f736c0afaf0 0x7f736c0b2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f738019d870 tx=0x7f737000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.022+0000 7f7376ffd640 1 -- 172.21.15.78:0/3777586344 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f737c0c76c0 con 0x7f73801045d0 2024-04-25T13:59:16.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.201+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73801a4cf0 con 0x7f73801045d0 2024-04-25T13:59:16.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.202+0000 7f7376ffd640 1 -- 172.21.15.78:0/3777586344 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f737c0cb9c0 con 0x7f73801045d0 2024-04-25T13:59:16.201 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f736c0afaf0 msgr2=0x7f736c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f736c0afaf0 0x7f736c0b2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f738019d870 tx=0x7f737000b390 comp rx=0 tx=0).stop 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 msgr2=0x7f738019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f738019c7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f737c006010 tx=0x7f737c031a50 comp rx=0 tx=0).stop 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 shutdown_connections 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7380109890 0x7f73801a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7380106f30 0x7f738019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f736c0afaf0 0x7f736c0b2000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 --2- 172.21.15.78:0/3777586344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73801045d0 0x7f738019c7d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 >> 172.21.15.78:0/3777586344 conn(0x7f73800fdbd0 msgr2=0x7f73801021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 shutdown_connections 2024-04-25T13:59:16.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.204+0000 7f7387e27640 1 -- 172.21.15.78:0/3777586344 wait complete. 2024-04-25T13:59:16.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:59:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 -- 172.21.15.78:0/2312899391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41045d0 msgr2=0x7ff6d41069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 --2- 172.21.15.78:0/2312899391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41045d0 0x7ff6d41069f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009ca0 tx=0x7ff6c402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 -- 172.21.15.78:0/2312899391 shutdown_connections 2024-04-25T13:59:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 --2- 172.21.15.78:0/2312899391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6d4109890 0x7ff6d410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 --2- 172.21.15.78:0/2312899391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6d4106f30 0x7ff6d4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 --2- 172.21.15.78:0/2312899391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d41045d0 0x7ff6d41069f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 -- 172.21.15.78:0/2312899391 >> 172.21.15.78:0/2312899391 conn(0x7ff6d40fdbd0 msgr2=0x7ff6d4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 -- 172.21.15.78:0/2312899391 shutdown_connections 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 -- 172.21.15.78:0/2312899391 wait complete. 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 Processor -- start 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.327+0000 7ff6d9985640 1 -- start start 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6d41045d0 0x7ff6d419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6d4106f30 0x7ff6d419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d4109890 0x7ff6d41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410f5e0 con 0x7ff6d4109890 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410f460 con 0x7ff6d4106f30 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410f760 con 0x7ff6d41045d0 2024-04-25T13:59:16.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d4109890 0x7ff6d41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d4109890 0x7ff6d41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43246/0 (socket says 172.21.15.78:43246) 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 -- 172.21.15.78:0/1114178624 learned_addr learned my addr 172.21.15.78:0/1114178624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d27fc640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6d4106f30 0x7ff6d419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d2ffd640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6d41045d0 0x7ff6d419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 -- 172.21.15.78:0/1114178624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6d41045d0 msgr2=0x7ff6d419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6d41045d0 0x7ff6d419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 -- 172.21.15.78:0/1114178624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6d4106f30 msgr2=0x7ff6d419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6d4106f30 0x7ff6d419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 -- 172.21.15.78:0/1114178624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d41a20d0 con 0x7ff6d4109890 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d37fe640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d4109890 0x7ff6d41a1a60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00d720 tx=0x7ff6cc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d8983640 1 -- 172.21.15.78:0/1114178624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc017070 con 0x7ff6d4109890 2024-04-25T13:59:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d41a23c0 con 0x7ff6d4109890 2024-04-25T13:59:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d8983640 1 -- 172.21.15.78:0/1114178624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6cc00de30 con 0x7ff6d4109890 2024-04-25T13:59:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d8983640 1 -- 172.21.15.78:0/1114178624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc004d00 con 0x7ff6d4109890 2024-04-25T13:59:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.328+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d41a2960 con 0x7ff6d4109890 2024-04-25T13:59:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.329+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d406c2a0 con 0x7ff6d4109890 2024-04-25T13:59:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.331+0000 7ff6d8983640 1 -- 172.21.15.78:0/1114178624 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff6cc004ea0 con 0x7ff6d4109890 2024-04-25T13:59:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.332+0000 7ff6d8983640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.332+0000 7ff6d2ffd640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.332+0000 7ff6d8983640 1 -- 172.21.15.78:0/1114178624 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff6cc0de990 con 0x7ff6d4109890 2024-04-25T13:59:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.332+0000 7ff6d2ffd640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4031b50 tx=0x7ff6c4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.335+0000 7ff6d8983640 1 -- 172.21.15.78:0/1114178624 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6cc0a7560 con 0x7ff6d4109890 2024-04-25T13:59:16.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.513+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff6d41a2bc0 con 0x7ff6d4109890 2024-04-25T13:59:16.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.513+0000 7ff6d8983640 1 -- 172.21.15.78:0/1114178624 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7ff6cc0a6e60 con 0x7ff6d4109890 2024-04-25T13:59:16.513 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6bc0afaf0 msgr2=0x7ff6bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4031b50 tx=0x7ff6c4004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d4109890 msgr2=0x7ff6d41a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d4109890 0x7ff6d41a1a60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00d720 tx=0x7ff6cc00dbe0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 shutdown_connections 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6bc0afaf0 0x7ff6bc0b2000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6d4109890 0x7ff6d41a1a60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6d4106f30 0x7ff6d419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 --2- 172.21.15.78:0/1114178624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6d41045d0 0x7ff6d419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 >> 172.21.15.78:0/1114178624 conn(0x7ff6d40fdbd0 msgr2=0x7ff6d4105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:16.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 shutdown_connections 2024-04-25T13:59:16.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.516+0000 7ff6d9985640 1 -- 172.21.15.78:0/1114178624 wait complete. 2024-04-25T13:59:16.527 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:59:16.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 -- 172.21.15.78:0/3494882444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc1027b0 msgr2=0x7f8edc102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 --2- 172.21.15.78:0/3494882444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc1027b0 0x7f8edc102c80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009900 tx=0x7f8ecc02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:16.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 -- 172.21.15.78:0/3494882444 shutdown_connections 2024-04-25T13:59:16.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 --2- 172.21.15.78:0/3494882444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8edc107920 0x7f8edc107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 --2- 172.21.15.78:0/3494882444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc1027b0 0x7f8edc102c80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 --2- 172.21.15.78:0/3494882444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8edc101d70 0x7f8edc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 -- 172.21.15.78:0/3494882444 >> 172.21.15.78:0/3494882444 conn(0x7f8edc0fd340 msgr2=0x7f8edc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 -- 172.21.15.78:0/3494882444 shutdown_connections 2024-04-25T13:59:16.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.638+0000 7f8ee3546640 1 -- 172.21.15.78:0/3494882444 wait complete. 2024-04-25T13:59:16.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 Processor -- start 2024-04-25T13:59:16.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 -- start start 2024-04-25T13:59:16.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc101d70 0x7f8edc1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8edc1027b0 0x7f8edc1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8edc107920 0x7f8edc1a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc115bf0 con 0x7f8edc101d70 2024-04-25T13:59:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc115a70 con 0x7f8edc107920 2024-04-25T13:59:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee3546640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc115d70 con 0x7f8edc1027b0 2024-04-25T13:59:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee12e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc101d70 0x7f8edc1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee12e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc101d70 0x7f8edc1a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33718/0 (socket says 172.21.15.78:33718) 2024-04-25T13:59:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.639+0000 7f8ee12e2640 1 -- 172.21.15.78:0/1858795716 learned_addr learned my addr 172.21.15.78:0/1858795716 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee1ae3640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8edc107920 0x7f8edc1a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee0ae1640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8edc1027b0 0x7f8edc1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee12e2640 1 -- 172.21.15.78:0/1858795716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8edc1027b0 msgr2=0x7f8edc1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee12e2640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8edc1027b0 0x7f8edc1a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee12e2640 1 -- 172.21.15.78:0/1858795716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8edc107920 msgr2=0x7f8edc1a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee12e2640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8edc107920 0x7f8edc1a7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee12e2640 1 -- 172.21.15.78:0/1858795716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc1a8370 con 0x7f8edc101d70 2024-04-25T13:59:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee1ae3640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8edc107920 0x7f8edc1a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee12e2640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc101d70 0x7f8edc1a0830 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800e9f0 tx=0x7f8ed800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1858795716 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800cd80 con 0x7f8edc101d70 2024-04-25T13:59:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1858795716 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed80054d0 con 0x7f8edc101d70 2024-04-25T13:59:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc1a8660 con 0x7f8edc101d70 2024-04-25T13:59:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1858795716 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8010690 con 0x7f8edc101d70 2024-04-25T13:59:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.640+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc1a8b60 con 0x7f8edc101d70 2024-04-25T13:59:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.641+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc106730 con 0x7f8edc101d70 2024-04-25T13:59:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.642+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1858795716 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8ed8004e90 con 0x7f8edc101d70 2024-04-25T13:59:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.643+0000 7f8ed27fc640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ec80afaf0 0x7f8ec80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.643+0000 7f8ee0ae1640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ec80afaf0 0x7f8ec80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.643+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1858795716 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8ed80df150 con 0x7f8edc101d70 2024-04-25T13:59:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.643+0000 7f8ee0ae1640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ec80afaf0 0x7f8ec80b2000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc02a930 tx=0x7f8ecc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.647+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1858795716 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ed80a7d20 con 0x7f8edc101d70 2024-04-25T13:59:16.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.824+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8edc0637a0 con 0x7f8edc101d70 2024-04-25T13:59:16.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.825+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1858795716 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f8ed80a7620 con 0x7f8edc101d70 2024-04-25T13:59:16.825 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ec80afaf0 msgr2=0x7f8ec80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ec80afaf0 0x7f8ec80b2000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc02a930 tx=0x7f8ecc000bc0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc101d70 msgr2=0x7f8edc1a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc101d70 0x7f8edc1a0830 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800e9f0 tx=0x7f8ed800edf0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 shutdown_connections 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8edc107920 0x7f8edc1a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8edc1027b0 0x7f8edc1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ec80afaf0 0x7f8ec80b2000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 --2- 172.21.15.78:0/1858795716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8edc101d70 0x7f8edc1a0830 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 >> 172.21.15.78:0/1858795716 conn(0x7f8edc0fd340 msgr2=0x7f8edc103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 shutdown_connections 2024-04-25T13:59:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.827+0000 7f8ee3546640 1 -- 172.21.15.78:0/1858795716 wait complete. 2024-04-25T13:59:16.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:16 smithi078 ceph-mon[29163]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:16.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:16 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2186269584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:16.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:16 smithi078 ceph-mon[29163]: pgmap v341: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T13:59:16.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:16 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3777586344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:16.837 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:16 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1114178624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:16.838 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:59:16.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:16 smithi129 ceph-mon[32569]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:16.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:16 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2186269584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:16.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:16 smithi129 ceph-mon[32569]: pgmap v341: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T13:59:16.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:16 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3777586344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:16.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:16 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1114178624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 -- 172.21.15.78:0/1873161251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64103b10 msgr2=0x7f4a64104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/1873161251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64103b10 0x7f4a64104810 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009900 tx=0x7f4a6002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 -- 172.21.15.78:0/1873161251 shutdown_connections 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/1873161251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a64109a50 0x7f4a6410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/1873161251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a64104d50 0x7f4a64105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/1873161251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64103b10 0x7f4a64104810 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 -- 172.21.15.78:0/1873161251 >> 172.21.15.78:0/1873161251 conn(0x7f4a640fed40 msgr2=0x7f4a641011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 -- 172.21.15.78:0/1873161251 shutdown_connections 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.946+0000 7f4a6c77a640 1 -- 172.21.15.78:0/1873161251 wait complete. 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 Processor -- start 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 -- start start 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a64103b10 0x7f4a6419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a64104d50 0x7f4a6419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64109a50 0x7f4a641a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410f5f0 con 0x7f4a64109a50 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410f470 con 0x7f4a64104d50 2024-04-25T13:59:16.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6c77a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410f770 con 0x7f4a64103b10 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6ad17640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64109a50 0x7f4a641a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6ad17640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64109a50 0x7f4a641a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33726/0 (socket says 172.21.15.78:33726) 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.947+0000 7f4a6ad17640 1 -- 172.21.15.78:0/635374378 learned_addr learned my addr 172.21.15.78:0/635374378 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a69d15640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a64104d50 0x7f4a6419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6a516640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a64103b10 0x7f4a6419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6ad17640 1 -- 172.21.15.78:0/635374378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a64103b10 msgr2=0x7f4a6419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6ad17640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a64103b10 0x7f4a6419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6ad17640 1 -- 172.21.15.78:0/635374378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a64104d50 msgr2=0x7f4a6419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6ad17640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a64104d50 0x7f4a6419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6ad17640 1 -- 172.21.15.78:0/635374378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a641a1cb0 con 0x7f4a64109a50 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a69d15640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a64104d50 0x7f4a6419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6a516640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a64103b10 0x7f4a6419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:16.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6ad17640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64109a50 0x7f4a641a16f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00eb10 tx=0x7f4a5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a5b7fe640 1 -- 172.21.15.78:0/635374378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c004b20 con 0x7f4a64109a50 2024-04-25T13:59:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a5b7fe640 1 -- 172.21.15.78:0/635374378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a5c004cc0 con 0x7f4a64109a50 2024-04-25T13:59:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a641a1fa0 con 0x7f4a64109a50 2024-04-25T13:59:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a5b7fe640 1 -- 172.21.15.78:0/635374378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c010670 con 0x7f4a64109a50 2024-04-25T13:59:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.948+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641a23b0 con 0x7f4a64109a50 2024-04-25T13:59:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.949+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6406a1a0 con 0x7f4a64109a50 2024-04-25T13:59:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.951+0000 7f4a5b7fe640 1 -- 172.21.15.78:0/635374378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4a5c005000 con 0x7f4a64109a50 2024-04-25T13:59:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.952+0000 7f4a5b7fe640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a500afaf0 0x7f4a500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.952+0000 7f4a6a516640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a500afaf0 0x7f4a500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.952+0000 7f4a5b7fe640 1 -- 172.21.15.78:0/635374378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4a5c0df530 con 0x7f4a64109a50 2024-04-25T13:59:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.952+0000 7f4a6a516640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a500afaf0 0x7f4a500b2000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4a60005f70 tx=0x7f4a60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:16.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:16.956+0000 7f4a5b7fe640 1 -- 172.21.15.78:0/635374378 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a5c053ff0 con 0x7f4a64109a50 2024-04-25T13:59:17.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:16 smithi112 ceph-mon[32617]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:59:17.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:16 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2186269584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:17.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:16 smithi112 ceph-mon[32617]: pgmap v341: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T13:59:17.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:16 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3777586344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:59:17.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:16 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1114178624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.169+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a641088b0 con 0x7f4a64109a50 2024-04-25T13:59:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.170+0000 7f4a5b7fe640 1 -- 172.21.15.78:0/635374378 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f4a641088b0 con 0x7f4a64109a50 2024-04-25T13:59:17.170 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:17.170 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T13:59:17.170 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a500afaf0 msgr2=0x7f4a500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a500afaf0 0x7f4a500b2000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4a60005f70 tx=0x7f4a60000bc0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64109a50 msgr2=0x7f4a641a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64109a50 0x7f4a641a16f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00eb10 tx=0x7f4a5c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 shutdown_connections 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a500afaf0 0x7f4a500b2000 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a64109a50 0x7f4a641a16f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a64104d50 0x7f4a6419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 --2- 172.21.15.78:0/635374378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a64103b10 0x7f4a6419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 >> 172.21.15.78:0/635374378 conn(0x7f4a640fed40 msgr2=0x7f4a64105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 shutdown_connections 2024-04-25T13:59:17.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.172+0000 7f4a6c77a640 1 -- 172.21.15.78:0/635374378 wait complete. 2024-04-25T13:59:17.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:59:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 -- 172.21.15.78:0/3611901605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8102a00 msgr2=0x7f96d8103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 --2- 172.21.15.78:0/3611901605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8102a00 0x7f96d8103700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009900 tx=0x7f96c802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 -- 172.21.15.78:0/3611901605 shutdown_connections 2024-04-25T13:59:17.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 --2- 172.21.15.78:0/3611901605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96d8108940 0x7f96d810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 --2- 172.21.15.78:0/3611901605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d8103c40 0x7f96d8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 --2- 172.21.15.78:0/3611901605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8102a00 0x7f96d8103700 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 -- 172.21.15.78:0/3611901605 >> 172.21.15.78:0/3611901605 conn(0x7f96d80fdbf0 msgr2=0x7f96d81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 -- 172.21.15.78:0/3611901605 shutdown_connections 2024-04-25T13:59:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.291+0000 7f96de100640 1 -- 172.21.15.78:0/3611901605 wait complete. 2024-04-25T13:59:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 Processor -- start 2024-04-25T13:59:17.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 -- start start 2024-04-25T13:59:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d8102a00 0x7f96d8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96d8103c40 0x7f96d81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8108940 0x7f96d81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810e560 con 0x7f96d8108940 2024-04-25T13:59:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810e3e0 con 0x7f96d8102a00 2024-04-25T13:59:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96de100640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810e6e0 con 0x7f96d8103c40 2024-04-25T13:59:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8108940 0x7f96d81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8108940 0x7f96d81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33732/0 (socket says 172.21.15.78:33732) 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96d7fff640 1 -- 172.21.15.78:0/3063878098 learned_addr learned my addr 172.21.15.78:0/3063878098 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96d6ffd640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96d8103c40 0x7f96d81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96d77fe640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d8102a00 0x7f96d8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96d7fff640 1 -- 172.21.15.78:0/3063878098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96d8103c40 msgr2=0x7f96d81996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.292+0000 7f96d7fff640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96d8103c40 0x7f96d81996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96d7fff640 1 -- 172.21.15.78:0/3063878098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d8102a00 msgr2=0x7f96d8199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96d7fff640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d8102a00 0x7f96d8199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96d7fff640 1 -- 172.21.15.78:0/3063878098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d81a0cd0 con 0x7f96d8108940 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96d7fff640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8108940 0x7f96d81a0660 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b970 tx=0x7f96d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96d4ff9640 1 -- 172.21.15.78:0/3063878098 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0019070 con 0x7f96d8108940 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96d4ff9640 1 -- 172.21.15.78:0/3063878098 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d0014440 con 0x7f96d8108940 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d81a0fc0 con 0x7f96d8108940 2024-04-25T13:59:17.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96d4ff9640 1 -- 172.21.15.78:0/3063878098 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d00075d0 con 0x7f96d8108940 2024-04-25T13:59:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.293+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d81a1450 con 0x7f96d8108940 2024-04-25T13:59:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.294+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d8107750 con 0x7f96d8108940 2024-04-25T13:59:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.295+0000 7f96d4ff9640 1 -- 172.21.15.78:0/3063878098 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f96d001c070 con 0x7f96d8108940 2024-04-25T13:59:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.296+0000 7f96d4ff9640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96c00afaf0 0x7f96c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.296+0000 7f96d77fe640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96c00afaf0 0x7f96c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.296+0000 7f96d4ff9640 1 -- 172.21.15.78:0/3063878098 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f96d00de610 con 0x7f96d8108940 2024-04-25T13:59:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.297+0000 7f96d77fe640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96c00afaf0 0x7f96c00b2000 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005ab0 tx=0x7f96c8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.300+0000 7f96d4ff9640 1 -- 172.21.15.78:0/3063878098 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96d00a6b90 con 0x7f96d8108940 2024-04-25T13:59:17.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.477+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96d802dd20 con 0x7f96d8108940 2024-04-25T13:59:17.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.479+0000 7f96d4ff9640 1 -- 172.21.15.78:0/3063878098 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f96d00aae90 con 0x7f96d8108940 2024-04-25T13:59:17.479 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:17.479 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96c00afaf0 msgr2=0x7f96c00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96c00afaf0 0x7f96c00b2000 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005ab0 tx=0x7f96c8031400 comp rx=0 tx=0).stop 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8108940 msgr2=0x7f96d81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8108940 0x7f96d81a0660 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b970 tx=0x7f96d000be30 comp rx=0 tx=0).stop 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 shutdown_connections 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96c00afaf0 0x7f96c00b2000 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96d8108940 0x7f96d81a0660 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96d8103c40 0x7f96d81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 --2- 172.21.15.78:0/3063878098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96d8102a00 0x7f96d8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 >> 172.21.15.78:0/3063878098 conn(0x7f96d80fdbf0 msgr2=0x7f96d8104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 shutdown_connections 2024-04-25T13:59:17.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.481+0000 7f96de100640 1 -- 172.21.15.78:0/3063878098 wait complete. 2024-04-25T13:59:17.493 DEBUG:tasks.fwd_scrub:Yielding 2024-04-25T13:59:17.493 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T13:59:17.504 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T13:59:17.504 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T13:59:17.504 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-25T13:59:17.504 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:59:17.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:17.505 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:59:17.507 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T13:59:17.543 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:59:17.544 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T13:59:17.544 INFO:tasks.workunit:timeout=3h 2024-04-25T13:59:17.544 INFO:tasks.workunit:cleanup=True 2024-04-25T13:59:17.544 DEBUG:teuthology.orchestra.run.smithi078:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:17.559 INFO:teuthology.orchestra.run.smithi078.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 13:57:17.514772499 +0000 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 13:57:17.514772499 +0000 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 13:59:10.818909008 +0000 2024-04-25T13:59:17.560 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 13:57:17.514772499 +0000 2024-04-25T13:59:17.560 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T13:59:17.560 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 -- 172.21.15.78:0/1050414803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 msgr2=0x7f38b8104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 --2- 172.21.15.78:0/1050414803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b8104040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac02a410 comp rx=0 tx=0).stop 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 -- 172.21.15.78:0/1050414803 shutdown_connections 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 --2- 172.21.15.78:0/1050414803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b8108930 0x7f38b810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 --2- 172.21.15.78:0/1050414803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b8104040 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 --2- 172.21.15.78:0/1050414803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38b81029f0 0x7f38b81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 -- 172.21.15.78:0/1050414803 >> 172.21.15.78:0/1050414803 conn(0x7f38b80fdc00 msgr2=0x7f38b8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 -- 172.21.15.78:0/1050414803 shutdown_connections 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.619+0000 7f38be8af640 1 -- 172.21.15.78:0/1050414803 wait complete. 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.620+0000 7f38be8af640 1 Processor -- start 2024-04-25T13:59:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.620+0000 7f38be8af640 1 -- start start 2024-04-25T13:59:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.620+0000 7f38be8af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b81029f0 0x7f38b806bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.620+0000 7f38be8af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38be8af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38b8108930 0x7f38b81a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38be8af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810e510 con 0x7f38b8103c30 2024-04-25T13:59:17.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38be8af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810e390 con 0x7f38b81029f0 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38be8af640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810e690 con 0x7f38b8108930 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33748/0 (socket says 172.21.15.78:33748) 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 -- 172.21.15.78:0/2320942023 learned_addr learned my addr 172.21.15.78:0/2320942023 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38bce4c640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38b8108930 0x7f38b81a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b7fff640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b81029f0 0x7f38b806bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 -- 172.21.15.78:0/2320942023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38b8108930 msgr2=0x7f38b81a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38b8108930 0x7f38b81a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 -- 172.21.15.78:0/2320942023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b81029f0 msgr2=0x7f38b806bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b81029f0 0x7f38b806bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 -- 172.21.15.78:0/2320942023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b81a1d90 con 0x7f38b8103c30 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b7fff640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b81029f0 0x7f38b806bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:17.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b77fe640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b819f020 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38ac006010 tx=0x7f38ac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b57fa640 1 -- 172.21.15.78:0/2320942023 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac03f030 con 0x7f38b8103c30 2024-04-25T13:59:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b57fa640 1 -- 172.21.15.78:0/2320942023 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38ac003aa0 con 0x7f38b8103c30 2024-04-25T13:59:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b81a2020 con 0x7f38b8103c30 2024-04-25T13:59:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38b57fa640 1 -- 172.21.15.78:0/2320942023 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac035070 con 0x7f38b8103c30 2024-04-25T13:59:17.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.621+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b81a2510 con 0x7f38b8103c30 2024-04-25T13:59:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 Processor -- start 2024-04-25T13:59:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 -- start start 2024-04-25T13:59:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b57fa640 1 -- 172.21.15.78:0/2320942023 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f38ac042030 con 0x7f38b8103c30 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf20730 0x56421d2e9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56421d247b00 con 0x56421cf18550 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56421d1e3920 con 0x56421cf20730 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38c5767740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56421d243ac0 con 0x56421cf41410 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2e2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33754/0 (socket says 172.21.15.78:33754) 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38bce4c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b7fff640 1 -- 172.21.15.78:0/3274047029 learned_addr learned my addr 172.21.15.78:0/3274047029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b77fe640 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf20730 0x56421d2e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b7fff640 1 -- 172.21.15.78:0/3274047029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 msgr2=0x56421d2e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b7fff640 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2e24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.624+0000 7f38b7fff640 1 -- 172.21.15.78:0/3274047029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf20730 msgr2=0x56421d2e9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f38b7fff640 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf20730 0x56421d2e9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f38b7fff640 1 -- 172.21.15.78:0/3274047029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56421d0217e0 con 0x56421cf18550 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f38b7fff640 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2e2a20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009cb0 tx=0x7f38a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ba9c289ddc3e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f38b57fa640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38a00afaf0 0x7f38a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f38bce4c640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38a00afaf0 0x7f38a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f389e7fc640 1 -- 172.21.15.78:0/3274047029 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8004020 con 0x56421cf18550 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f389e7fc640 1 -- 172.21.15.78:0/3274047029 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a80041c0 con 0x56421cf18550 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f389e7fc640 1 -- 172.21.15.78:0/3274047029 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8012720 con 0x56421cf18550 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.625+0000 7f38b57fa640 1 -- 172.21.15.78:0/2320942023 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f38ac0ff240 con 0x7f38b8103c30 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 -- 172.21.15.78:0/3274047029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 msgr2=0x56421d2e2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38bce4c640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38a00afaf0 0x7f38a00b2000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x56421d2e3670 tx=0x7f38b00096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2e2a20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009cb0 tx=0x7f38a80117b0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 -- 172.21.15.78:0/3274047029 shutdown_connections 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf20730 0x56421d2e9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2e2a20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 --2- 172.21.15.78:0/3274047029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 -- 172.21.15.78:0/3274047029 >> 172.21.15.78:0/3274047029 conn(0x56421cf35560 msgr2=0x56421d2dde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 -- 172.21.15.78:0/3274047029 shutdown_connections 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 -- 172.21.15.78:0/3274047029 wait complete. 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 -- start start 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.626+0000 7f38c5767740 1 Processor -- start 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf18550 0x56421d2ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56421d2ea370 con 0x56421cf20730 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56421d1e15d0 con 0x56421cf41410 2024-04-25T13:59:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56421d2ea4f0 con 0x56421cf18550 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 10 client.0 _ll_register_callbacks cb 0x56421d1cf080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2ec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33758/0 (socket says 172.21.15.78:33758) 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 -- 172.21.15.78:0/1445649649 learned_addr learned my addr 172.21.15.78:0/1445649649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf18550 0x56421d2ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38bce4c640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 -- 172.21.15.78:0/1445649649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf18550 msgr2=0x56421d2ec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf18550 0x56421d2ec2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 -- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 msgr2=0x56421d2f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 -- 172.21.15.78:0/1445649649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56421d081f70 con 0x56421cf20730 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38bce4c640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2ec810 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38ac031440 tx=0x7f38ac003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a158d7fd1aeeb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38977fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38977fe640 1 -- 172.21.15.78:0/1445649649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac049070 con 0x56421cf20730 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38977fe640 1 -- 172.21.15.78:0/1445649649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f38ac0c39a0 con 0x56421cf20730 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.627+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a80041c0 con 0x56421cf20730 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38977fe640 1 -- 172.21.15.78:0/1445649649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f38ac0c7060 con 0x56421cf20730 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 10 client.15411 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56421cf19170 con 0x56421cf20730 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38977fe640 1 -- 172.21.15.78:0/1445649649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a80041c0 con 0x56421cf20730 2024-04-25T13:59:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38977fe640 1 -- 172.21.15.78:0/1445649649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f38ac0c7970 con 0x56421cf20730 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 10 client.15411 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 20 client.15411 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421cf001f0 0x56421d2f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 4 client.15411 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56421d255710 con 0x56421cf001f0 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.628+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421cf001f0 0x56421d2f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.629+0000 7f38977fe640 10 client.15411 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.629+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421cf001f0 0x56421d2f9b60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb81a5db8b61742 server_cookie=34c665a6c75e90b5 in_seq=0 out_seq=0 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38977fe640 1 -- 172.21.15.78:0/1445649649 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56421d255710 con 0x56421cf001f0 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38977fe640 10 client.15411 handle_command_reply: tid=0 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 client.15411 shutdown 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 msgr2=0x56421d2ec810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2ec810 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38ac031440 tx=0x7f38ac003e80 comp rx=0 tx=0).stop 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 shutdown_connections 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2ec810 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421cf001f0 0x56421d2f9b60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 --2- 172.21.15.78:0/1445649649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf18550 0x56421d2ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38977fe640 0 client.15411 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 >> 172.21.15.78:0/1445649649 conn(0x56421cf35560 msgr2=0x56421d2dcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 shutdown_connections 2024-04-25T13:59:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 1 -- 172.21.15.78:0/1445649649 wait complete. 2024-04-25T13:59:17.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.630+0000 7f38c5767740 20 client.15411 trim_cache size 0 max 16384 2024-04-25T13:59:17.639 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:59:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 Processor -- start 2024-04-25T13:59:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 -- start start 2024-04-25T13:59:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf20730 0x56421d2fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81adb40 con 0x56421cf18550 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ab410 con 0x56421cf41410 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38c5767740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81a9260 con 0x56421cf20730 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2eafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33760/0 (socket says 172.21.15.78:33760) 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.648+0000 7f38b7fff640 1 -- 172.21.15.78:0/1217620395 learned_addr learned my addr 172.21.15.78:0/1217620395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf20730 0x56421d2fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38bce4c640 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b7fff640 1 -- 172.21.15.78:0/1217620395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf20730 msgr2=0x56421d2fe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf20730 0x56421d2fe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b7fff640 1 -- 172.21.15.78:0/1217620395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 msgr2=0x56421d2fe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2fe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b7fff640 1 -- 172.21.15.78:0/1217620395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56421d2e7180 con 0x56421cf18550 2024-04-25T13:59:17.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38bce4c640 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:17.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf20730 0x56421d2fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2eafa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f38a8011cb0 tx=0x7f38a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a891d495d9f2add0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f389dffb640 1 -- 172.21.15.78:0/1217620395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a80097a0 con 0x56421cf18550 2024-04-25T13:59:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f389dffb640 1 -- 172.21.15.78:0/1217620395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8029030 con 0x56421cf18550 2024-04-25T13:59:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.649+0000 7f389dffb640 1 -- 172.21.15.78:0/1217620395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8005290 con 0x56421cf18550 2024-04-25T13:59:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 -- 172.21.15.78:0/1217620395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 msgr2=0x56421d2eafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2eafa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f38a8011cb0 tx=0x7f38a8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 -- 172.21.15.78:0/1217620395 shutdown_connections 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf41410 0x56421d2fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf20730 0x56421d2fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 --2- 172.21.15.78:0/1217620395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf18550 0x56421d2eafa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 -- 172.21.15.78:0/1217620395 >> 172.21.15.78:0/1217620395 conn(0x56421cf35560 msgr2=0x56421d2ea890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 -- 172.21.15.78:0/1217620395 shutdown_connections 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 -- 172.21.15.78:0/1217620395 wait complete. 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.650+0000 7f38c5767740 1 -- start start 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 1 Processor -- start 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf18550 0x56421d2f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81a98d0 con 0x56421cf20730 2024-04-25T13:59:17.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81aad40 con 0x56421cf18550 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81a9fa0 con 0x56421cf41410 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38c5767740 10 client.0 _ll_register_callbacks cb 0x7f38ac102830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2f0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33768/0 (socket says 172.21.15.78:33768) 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 -- 172.21.15.78:0/1258385414 learned_addr learned my addr 172.21.15.78:0/1258385414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38bce4c640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf18550 0x56421d2f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 -- 172.21.15.78:0/1258385414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 msgr2=0x56421d2f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 -- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf18550 msgr2=0x56421d2f0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf18550 0x56421d2f0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b77fe640 1 -- 172.21.15.78:0/1258385414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56421d2f23b0 con 0x56421cf20730 2024-04-25T13:59:17.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.651+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf18550 0x56421d2f0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38b77fe640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2f0f80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38ac004010 tx=0x7f38ac005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581a42c467c33b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38977fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38977fe640 1 -- 172.21.15.78:0/1258385414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac049070 con 0x56421cf20730 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38977fe640 1 -- 172.21.15.78:0/1258385414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f38ac0d02f0 con 0x56421cf20730 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a8029030 con 0x56421cf20730 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38977fe640 1 -- 172.21.15.78:0/1258385414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f38ac0c6e80 con 0x56421cf20730 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 10 client.15417 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56421d362260 con 0x56421cf20730 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38977fe640 1 -- 172.21.15.78:0/1258385414 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a8029030 con 0x56421cf20730 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38977fe640 1 -- 172.21.15.78:0/1258385414 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f38ac0aebb0 con 0x56421cf20730 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 10 client.15417 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 20 client.15417 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:17.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421ced5630 0x56421d3648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 4 client.15417 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.652+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56421d24f580 con 0x56421ced5630 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.653+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421ced5630 0x56421d3648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.653+0000 7f38977fe640 10 client.15417 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.653+0000 7f38b7fff640 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421ced5630 0x56421d3648e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78e1bac273fbcc7 server_cookie=9b71928b0cd3e82b in_seq=0 out_seq=0 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38977fe640 1 -- 172.21.15.78:0/1258385414 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56421d24f580 con 0x56421ced5630 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38977fe640 10 client.15417 handle_command_reply: tid=0 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38c5767740 1 client.15417 shutdown 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 msgr2=0x56421d2f0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38c5767740 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2f0f80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38ac004010 tx=0x7f38ac005c90 comp rx=0 tx=0).stop 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 shutdown_connections 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38c5767740 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56421ced5630 0x56421d3648e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38c5767740 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56421cf41410 0x56421d2f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.654+0000 7f38977fe640 0 client.15417 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38c5767740 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421cf20730 0x56421d2f0f80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38c5767740 1 --2- 172.21.15.78:0/1258385414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56421cf18550 0x56421d2f0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 >> 172.21.15.78:0/1258385414 conn(0x56421cf35560 msgr2=0x56421d2e0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 shutdown_connections 2024-04-25T13:59:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38c5767740 1 -- 172.21.15.78:0/1258385414 wait complete. 2024-04-25T13:59:17.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38c5767740 20 client.15417 trim_cache size 0 max 16384 2024-04-25T13:59:17.661 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T13:59:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e265f791-e275-4565-b556-c760588aa3ba", 2024-04-25T13:59:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T13:59:17.662 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38a00afaf0 msgr2=0x7f38a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38a00afaf0 0x7f38a00b2000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x56421d2e3670 tx=0x7f38b00096a0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 msgr2=0x7f38b819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b819f020 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38ac006010 tx=0x7f38ac031a50 comp rx=0 tx=0).stop 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 shutdown_connections 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38b8108930 0x7f38b81a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38a00afaf0 0x7f38a00b2000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b8103c30 0x7f38b819f020 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 --2- 172.21.15.78:0/2320942023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38b81029f0 0x7f38b806bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 >> 172.21.15.78:0/2320942023 conn(0x7f38b80fdc00 msgr2=0x7f38b8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 shutdown_connections 2024-04-25T13:59:17.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.655+0000 7f38be8af640 1 -- 172.21.15.78:0/2320942023 wait complete. 2024-04-25T13:59:17.664 INFO:tasks.workunit.client.0.smithi078.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T13:59:17.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:59:17.667 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:59:17.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 -- 172.21.15.78:0/1024371135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564109a20 msgr2=0x7fb56410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 --2- 172.21.15.78:0/1024371135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564109a20 0x7fb56410be40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009ca0 tx=0x7fb55c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 -- 172.21.15.78:0/1024371135 shutdown_connections 2024-04-25T13:59:17.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 --2- 172.21.15.78:0/1024371135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564109a20 0x7fb56410be40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 --2- 172.21.15.78:0/1024371135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb564104d20 0x7fb564105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 --2- 172.21.15.78:0/1024371135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb564103ae0 0x7fb5641047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 -- 172.21.15.78:0/1024371135 >> 172.21.15.78:0/1024371135 conn(0x7fb5640fed10 msgr2=0x7fb564101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 -- 172.21.15.78:0/1024371135 shutdown_connections 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.819+0000 7fb569c34640 1 -- 172.21.15.78:0/1024371135 wait complete. 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 Processor -- start 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 -- start start 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564103ae0 0x7fb56419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb564104d20 0x7fb56419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb564109a20 0x7fb5641a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb56410f610 con 0x7fb564103ae0 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb56410f490 con 0x7fb564109a20 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb569c34640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb56410f790 con 0x7fb564104d20 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb5637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564103ae0 0x7fb56419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb5637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564103ae0 0x7fb56419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33782/0 (socket says 172.21.15.78:33782) 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb5637fe640 1 -- 172.21.15.78:0/1998217148 learned_addr learned my addr 172.21.15.78:0/1998217148 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb562ffd640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb564104d20 0x7fb56419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.820+0000 7fb563fff640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb564109a20 0x7fb5641a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb5637fe640 1 -- 172.21.15.78:0/1998217148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb564104d20 msgr2=0x7fb56419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb564104d20 0x7fb56419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb5637fe640 1 -- 172.21.15.78:0/1998217148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb564109a20 msgr2=0x7fb5641a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb564109a20 0x7fb5641a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb5637fe640 1 -- 172.21.15.78:0/1998217148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5641a3f20 con 0x7fb564103ae0 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564103ae0 0x7fb56419c3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb558005d00 tx=0x7fb55800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb560ff9640 1 -- 172.21.15.78:0/1998217148 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558017070 con 0x7fb564103ae0 2024-04-25T13:59:17.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb560ff9640 1 -- 172.21.15.78:0/1998217148 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb558009c30 con 0x7fb564103ae0 2024-04-25T13:59:17.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5641a4210 con 0x7fb564103ae0 2024-04-25T13:59:17.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb560ff9640 1 -- 172.21.15.78:0/1998217148 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55800f620 con 0x7fb564103ae0 2024-04-25T13:59:17.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.821+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5641a4710 con 0x7fb564103ae0 2024-04-25T13:59:17.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.823+0000 7fb560ff9640 1 -- 172.21.15.78:0/1998217148 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb55801a070 con 0x7fb564103ae0 2024-04-25T13:59:17.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 Processor -- start 2024-04-25T13:59:17.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 -- start start 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33e3390 0x5634d3784a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33ba4d0 0x5634d3784fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3785c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5634d36e9d80 con 0x5634d33ba4d0 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5634d36a38b0 con 0x5634d33e3390 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb570bc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5634d36e5d40 con 0x5634d33c26b0 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb560ff9640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb54c0afaf0 0x7fb54c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb5637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33ba4d0 0x5634d3784fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb5637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33ba4d0 0x5634d3784fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33790/0 (socket says 172.21.15.78:33790) 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb5637fe640 1 -- 172.21.15.78:0/1682255510 learned_addr learned my addr 172.21.15.78:0/1682255510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb563fff640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb54c0afaf0 0x7fb54c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb562ffd640 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3785c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb563fff640 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33e3390 0x5634d3784a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb560ff9640 1 -- 172.21.15.78:0/1998217148 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5580de6b0 con 0x7fb564103ae0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb5637fe640 1 -- 172.21.15.78:0/1682255510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 msgr2=0x5634d3785c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3785c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb5637fe640 1 -- 172.21.15.78:0/1682255510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33e3390 msgr2=0x5634d3784a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.824+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33e3390 0x5634d3784a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.825+0000 7fb5637fe640 1 -- 172.21.15.78:0/1682255510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d3674a40 con 0x5634d33ba4d0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.825+0000 7fb563fff640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb54c0afaf0 0x7fb54c0b2000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb55c004af0 tx=0x7fb55c0094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.825+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33ba4d0 0x5634d3784fe0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5580a2c00 tx=0x7fb558010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6979e803df20912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.825+0000 7fb549ffb640 1 -- 172.21.15.78:0/1682255510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558021070 con 0x5634d33ba4d0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.825+0000 7fb549ffb640 1 -- 172.21.15.78:0/1682255510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5580af330 con 0x5634d33ba4d0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.825+0000 7fb549ffb640 1 -- 172.21.15.78:0/1682255510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5580a6b80 con 0x5634d33ba4d0 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 -- 172.21.15.78:0/1682255510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33ba4d0 msgr2=0x5634d3784fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33ba4d0 0x5634d3784fe0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5580a2c00 tx=0x7fb558010f80 comp rx=0 tx=0).stop 2024-04-25T13:59:17.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 -- 172.21.15.78:0/1682255510 shutdown_connections 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3785c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33ba4d0 0x5634d3784fe0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1682255510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33e3390 0x5634d3784a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 -- 172.21.15.78:0/1682255510 >> 172.21.15.78:0/1682255510 conn(0x5634d33d74e0 msgr2=0x5634d3780270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 -- 172.21.15.78:0/1682255510 shutdown_connections 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 -- 172.21.15.78:0/1682255510 wait complete. 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 -- start start 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.826+0000 7fb570bc3740 1 Processor -- start 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d378e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d378eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d3795e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5634d37866c0 con 0x5634d33c26b0 2024-04-25T13:59:17.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5634d3670d10 con 0x5634d33ba4d0 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5634d3786840 con 0x5634d33e3390 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 10 client.0 _ll_register_callbacks cb 0x5634d369d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d378eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d378eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33798/0 (socket says 172.21.15.78:33798) 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 -- 172.21.15.78:0/2392915290 learned_addr learned my addr 172.21.15.78:0/2392915290 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb5637fe640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d378e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb563fff640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d3795e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 -- 172.21.15.78:0/2392915290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 msgr2=0x5634d3795e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d3795e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 -- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 msgr2=0x5634d378e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d378e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 -- 172.21.15.78:0/2392915290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d34c57d0 con 0x5634d33c26b0 2024-04-25T13:59:17.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb5637fe640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d378e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb562ffd640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d378eea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb554012720 tx=0x7fb554012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b243e7a649c243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb542ffd640 1 -- 172.21.15.78:0/2392915290 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554004030 con 0x5634d33c26b0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb542ffd640 1 -- 172.21.15.78:0/2392915290 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5540041d0 con 0x5634d33c26b0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5580af330 con 0x5634d33c26b0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.827+0000 7fb542ffd640 1 -- 172.21.15.78:0/2392915290 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb554004410 con 0x5634d33c26b0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 10 client.15429 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5634d33fb3f0 con 0x5634d33c26b0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb542ffd640 1 -- 172.21.15.78:0/2392915290 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5580af330 con 0x5634d33c26b0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb542ffd640 1 -- 172.21.15.78:0/2392915290 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb554003b20 con 0x5634d33c26b0 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 10 client.15429 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 20 client.15429 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:17.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33a1f60 0x5634d379c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 4 client.15429 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634d36f7990 con 0x5634d33a1f60 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.828+0000 7fb5637fe640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33a1f60 0x5634d379c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.829+0000 7fb542ffd640 10 client.15429 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.829+0000 7fb5637fe640 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33a1f60 0x5634d379c3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2342d281e309c566 server_cookie=7b0eb4fa18de4662 in_seq=0 out_seq=0 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.829+0000 7fb542ffd640 1 -- 172.21.15.78:0/2392915290 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634d36f7990 con 0x5634d33a1f60 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.829+0000 7fb542ffd640 10 client.15429 handle_command_reply: tid=0 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 client.15429 shutdown 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 msgr2=0x5634d378eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d378eea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb554012720 tx=0x7fb554012be0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 shutdown_connections 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33a1f60 0x5634d379c3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d3795e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d378eea0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 --2- 172.21.15.78:0/2392915290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d378e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 >> 172.21.15.78:0/2392915290 conn(0x5634d33d74e0 msgr2=0x5634d3783180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb542ffd640 0 client.15429 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 shutdown_connections 2024-04-25T13:59:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 1 -- 172.21.15.78:0/2392915290 wait complete. 2024-04-25T13:59:17.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.830+0000 7fb570bc3740 20 client.15429 trim_cache size 0 max 16384 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 Processor -- start 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 -- start start 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d37897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d3789ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d378cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641afda0 con 0x5634d33c26b0 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641ad670 con 0x5634d33ba4d0 2024-04-25T13:59:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb570bc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641ab4c0 con 0x5634d33e3390 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d3789ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d3789ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33814/0 (socket says 172.21.15.78:33814) 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.849+0000 7fb562ffd640 1 -- 172.21.15.78:0/908667335 learned_addr learned my addr 172.21.15.78:0/908667335 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb563fff640 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d378cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb5637fe640 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d37897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb562ffd640 1 -- 172.21.15.78:0/908667335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 msgr2=0x5634d378cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb562ffd640 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d378cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb562ffd640 1 -- 172.21.15.78:0/908667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 msgr2=0x5634d37897a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb562ffd640 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d37897a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb562ffd640 1 -- 172.21.15.78:0/908667335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d33bb0f0 con 0x5634d33c26b0 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb5637fe640 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d37897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb562ffd640 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d3789ce0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb554000c00 tx=0x7fb5540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c47d4e1eeb59b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb5497fa640 1 -- 172.21.15.78:0/908667335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554003fc0 con 0x5634d33c26b0 2024-04-25T13:59:17.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb5497fa640 1 -- 172.21.15.78:0/908667335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb55400fdd0 con 0x5634d33c26b0 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.850+0000 7fb5497fa640 1 -- 172.21.15.78:0/908667335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55401c750 con 0x5634d33c26b0 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 -- 172.21.15.78:0/908667335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 msgr2=0x5634d3789ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d3789ce0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb554000c00 tx=0x7fb5540057f0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 -- 172.21.15.78:0/908667335 shutdown_connections 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33e3390 0x5634d378cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33c26b0 0x5634d3789ce0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 --2- 172.21.15.78:0/908667335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d37897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 -- 172.21.15.78:0/908667335 >> 172.21.15.78:0/908667335 conn(0x5634d33d74e0 msgr2=0x5634d37849e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 -- 172.21.15.78:0/908667335 shutdown_connections 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 -- 172.21.15.78:0/908667335 wait complete. 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 -- start start 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.851+0000 7fb570bc3740 1 Processor -- start 2024-04-25T13:59:17.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d3792100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3792640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb570bc3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33e3390 0x5634d3802720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb570bc3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641abb30 con 0x5634d33e3390 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb570bc3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641acfa0 con 0x5634d33ba4d0 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb570bc3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641ac200 con 0x5634d33c26b0 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb570bc3740 10 client.0 _ll_register_callbacks cb 0x7fb554009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33e3390 0x5634d3802720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33e3390 0x5634d3802720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33822/0 (socket says 172.21.15.78:33822) 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 -- 172.21.15.78:0/1401771211 learned_addr learned my addr 172.21.15.78:0/1401771211 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb562ffd640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3792640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d3792100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 -- 172.21.15.78:0/1401771211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 msgr2=0x5634d3792640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3792640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 -- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 msgr2=0x5634d3792100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d3792100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 -- 172.21.15.78:0/1401771211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d36aa4b0 con 0x5634d33e3390 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb562ffd640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3792640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d3792100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb563fff640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33e3390 0x5634d3802720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb55c035000 tx=0x7fb55c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cb604c95135fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.852+0000 7fb542ffd640 1 -- 172.21.15.78:0/1401771211 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c004040 con 0x5634d33e3390 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb542ffd640 1 -- 172.21.15.78:0/1401771211 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb55c0041e0 con 0x5634d33e3390 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb55400fdd0 con 0x5634d33e3390 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb542ffd640 1 -- 172.21.15.78:0/1401771211 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb55c0043e0 con 0x5634d33e3390 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 10 client.15435 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5634d3803900 con 0x5634d33e3390 2024-04-25T13:59:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb542ffd640 1 -- 172.21.15.78:0/1401771211 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb55400fdd0 con 0x5634d33e3390 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb542ffd640 1 -- 172.21.15.78:0/1401771211 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb55c04a040 con 0x5634d33e3390 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 10 client.15435 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 20 client.15435 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33773a0 0x5634d38063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 4 client.15435 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5634d36f1800 con 0x5634d33773a0 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.853+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33773a0 0x5634d38063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb5637fe640 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33773a0 0x5634d38063a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42de1b70fc6a9e8 server_cookie=4ee8036408ce7e71 in_seq=0 out_seq=0 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb542ffd640 10 client.15435 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb542ffd640 1 -- 172.21.15.78:0/1401771211 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5634d36f1800 con 0x5634d33773a0 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb542ffd640 10 client.15435 handle_command_reply: tid=0 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 client.15435 shutdown 2024-04-25T13:59:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33e3390 msgr2=0x5634d3802720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33e3390 0x5634d3802720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb55c035000 tx=0x7fb55c048000 comp rx=0 tx=0).stop 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 shutdown_connections 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634d33e3390 0x5634d3802720 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5634d33773a0 0x5634d38063a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634d33c26b0 0x5634d3792640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 --2- 172.21.15.78:0/1401771211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5634d33ba4d0 0x5634d3792100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb542ffd640 0 client.15435 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 >> 172.21.15.78:0/1401771211 conn(0x5634d33d74e0 msgr2=0x5634d378e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 shutdown_connections 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 1 -- 172.21.15.78:0/1401771211 wait complete. 2024-04-25T13:59:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.854+0000 7fb570bc3740 20 client.15435 trim_cache size 0 max 16384 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: "e265f791-e275-4565-b556-c760588aa3ba": { 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "e265f791-e275-4565-b556-c760588aa3ba", 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T13:59:17.862 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb54c0afaf0 msgr2=0x7fb54c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb54c0afaf0 0x7fb54c0b2000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb55c004af0 tx=0x7fb55c0094a0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564103ae0 msgr2=0x7fb56419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564103ae0 0x7fb56419c3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb558005d00 tx=0x7fb55800dde0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 shutdown_connections 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb564109a20 0x7fb5641a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb564104d20 0x7fb56419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb54c0afaf0 0x7fb54c0b2000 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 --2- 172.21.15.78:0/1998217148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb564103ae0 0x7fb56419c3e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 >> 172.21.15.78:0/1998217148 conn(0x7fb5640fed10 msgr2=0x7fb5640ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 shutdown_connections 2024-04-25T13:59:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:17.855+0000 7fb569c34640 1 -- 172.21.15.78:0/1998217148 wait complete. 2024-04-25T13:59:17.865 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:17 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1858795716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:17.865 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:17 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/635374378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:17.865 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:17 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3063878098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:17.867 INFO:tasks.cephfs.filesystem:scrub status for tag:e265f791-e275-4565-b556-c760588aa3ba - {'path': '/', 'tag': 'e265f791-e275-4565-b556-c760588aa3ba', 'options': 'recursive,force'} 2024-04-25T13:59:17.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:17 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1858795716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:17.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:17 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/635374378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:17.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:17 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3063878098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:18.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:17 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1858795716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:59:18.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:17 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/635374378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:18.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:17 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3063878098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:59:18.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:18 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T13:59:18.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:18 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:18.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:18 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T13:59:18.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:18 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:18.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:18 smithi129 ceph-mon[32569]: pgmap v342: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T13:59:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:18 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T13:59:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:18 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:18 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T13:59:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:18 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:18 smithi078 ceph-mon[29163]: pgmap v342: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T13:59:19.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:18 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T13:59:19.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:18 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:19.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:18 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T13:59:19.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:18 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:19.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:18 smithi112 ceph-mon[32617]: pgmap v342: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T13:59:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:21 smithi078 ceph-mon[29163]: pgmap v343: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T13:59:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:21 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T13:59:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:21 smithi112 ceph-mon[32617]: pgmap v343: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T13:59:21.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:21 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T13:59:21.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:21 smithi129 ceph-mon[32569]: pgmap v343: 97 pgs: 97 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-25T13:59:21.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:21 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T13:59:22.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:22 smithi112 ceph-mon[32617]: pgmap v344: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T13:59:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:22 smithi129 ceph-mon[32569]: pgmap v344: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T13:59:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:22 smithi078 ceph-mon[29163]: pgmap v344: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T13:59:24.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:24 smithi112 ceph-mon[32617]: pgmap v345: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T13:59:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:24 smithi129 ceph-mon[32569]: pgmap v345: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T13:59:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:24 smithi078 ceph-mon[29163]: pgmap v345: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T13:59:26.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:26 smithi129 ceph-mon[32569]: pgmap v346: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T13:59:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:26 smithi078 ceph-mon[29163]: pgmap v346: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T13:59:26.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:26 smithi112 ceph-mon[32617]: pgmap v346: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T13:59:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:28 smithi129 ceph-mon[32569]: pgmap v347: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T13:59:28.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:28 smithi078 ceph-mon[29163]: pgmap v347: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T13:59:28.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:28 smithi112 ceph-mon[32617]: pgmap v347: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T13:59:30.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:30 smithi078 ceph-mon[29163]: pgmap v348: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T13:59:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 -- 172.21.15.78:0/566168547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0103c60 msgr2=0x7f29b0104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 --2- 172.21.15.78:0/566168547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0103c60 0x7f29b0104070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2998009900 tx=0x7f299802a410 comp rx=0 tx=0).stop 2024-04-25T13:59:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 -- 172.21.15.78:0/566168547 shutdown_connections 2024-04-25T13:59:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 --2- 172.21.15.78:0/566168547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29b0108960 0x7f29b010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 --2- 172.21.15.78:0/566168547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0103c60 0x7f29b0104070 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 --2- 172.21.15.78:0/566168547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b0102a20 0x7f29b0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 -- 172.21.15.78:0/566168547 >> 172.21.15.78:0/566168547 conn(0x7f29b00fdc10 msgr2=0x7f29b01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 -- 172.21.15.78:0/566168547 shutdown_connections 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.655+0000 7f29b5296640 1 -- 172.21.15.78:0/566168547 wait complete. 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 Processor -- start 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 -- start start 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0102a20 0x7f29b019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b0103c60 0x7f29b019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29b0108960 0x7f29b01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29b010e550 con 0x7f29b0102a20 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29b010e3d0 con 0x7f29b0103c60 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29b5296640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29b010e6d0 con 0x7f29b0108960 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0102a20 0x7f29b019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0102a20 0x7f29b019a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50272/0 (socket says 172.21.15.78:50272) 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 -- 172.21.15.78:0/66605719 learned_addr learned my addr 172.21.15.78:0/66605719 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29af7fe640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29b0108960 0x7f29b01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29a7fff640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b0103c60 0x7f29b019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 -- 172.21.15.78:0/66605719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29b0108960 msgr2=0x7f29b01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29b0108960 0x7f29b01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 -- 172.21.15.78:0/66605719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b0103c60 msgr2=0x7f29b019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b0103c60 0x7f29b019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29aeffd640 1 -- 172.21.15.78:0/66605719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b01a1d60 con 0x7f29b0102a20 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.656+0000 7f29a7fff640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b0103c60 0x7f29b019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.657+0000 7f29aeffd640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0102a20 0x7f29b019a220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f29a000d740 tx=0x7f29a000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.657+0000 7f29acff9640 1 -- 172.21.15.78:0/66605719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0017070 con 0x7f29b0102a20 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.657+0000 7f29acff9640 1 -- 172.21.15.78:0/66605719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a000de50 con 0x7f29b0102a20 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.657+0000 7f29acff9640 1 -- 172.21.15.78:0/66605719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0003c20 con 0x7f29b0102a20 2024-04-25T13:59:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.657+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b01a1ff0 con 0x7f29b0102a20 2024-04-25T13:59:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.657+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b01a24b0 con 0x7f29b0102a20 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.659+0000 7f29bc225740 1 Processor -- start 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.659+0000 7f29bc225740 1 -- start start 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29bc225740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x559600370970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29bc225740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x559600370ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29bc225740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600377df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29bc225740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5596002d6460 con 0x5595fffa72f0 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29bc225740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559600290590 con 0x5595fffaf4d0 2024-04-25T13:59:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29bc225740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596002d2420 con 0x5595fffd01b0 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29acff9640 1 -- 172.21.15.78:0/66605719 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f29a0004020 con 0x7f29b0102a20 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600377df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600377df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:35346/0 (socket says 172.21.15.78:35346) 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29a7fff640 1 -- 172.21.15.78:0/1993171298 learned_addr learned my addr 172.21.15.78:0/1993171298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x559600370970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29aeffd640 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x559600370ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29af7fe640 1 -- 172.21.15.78:0/1993171298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 msgr2=0x559600377df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600377df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29af7fe640 1 -- 172.21.15.78:0/1993171298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 msgr2=0x559600370ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x559600370ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29af7fe640 1 -- 172.21.15.78:0/1993171298 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960029d0d0 con 0x5595fffd01b0 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29aeffd640 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x559600370ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.660+0000 7f29a7fff640 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600377df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:59:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.661+0000 7f29acff9640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f299c0afae0 0x7f299c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.661+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x559600370970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009f30 tx=0x7f29a8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2656188f23b1760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.661+0000 7f29aeffd640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f299c0afae0 0x7f299c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.661+0000 7f29a57fa640 1 -- 172.21.15.78:0/1993171298 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8016070 con 0x5595fffd01b0 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.661+0000 7f29a57fa640 1 -- 172.21.15.78:0/1993171298 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a8004d20 con 0x5595fffd01b0 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.661+0000 7f29a57fa640 1 -- 172.21.15.78:0/1993171298 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a80127d0 con 0x5595fffd01b0 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.661+0000 7f29acff9640 1 -- 172.21.15.78:0/66605719 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f29a00de660 con 0x7f29b0102a20 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 -- 172.21.15.78:0/1993171298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 msgr2=0x559600370970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x559600370970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009f30 tx=0x7f29a8010800 comp rx=0 tx=0).stop 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29aeffd640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f299c0afae0 0x7f299c0b1ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x559600371d50 tx=0x7f29a00e4550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 -- 172.21.15.78:0/1993171298 shutdown_connections 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600377df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x559600370ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 --2- 172.21.15.78:0/1993171298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x559600370970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 -- 172.21.15.78:0/1993171298 >> 172.21.15.78:0/1993171298 conn(0x5595fffc4300 msgr2=0x55960036c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 -- 172.21.15.78:0/1993171298 shutdown_connections 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 -- 172.21.15.78:0/1993171298 wait complete. 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 -- start start 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 Processor -- start 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffa72f0 0x55960037eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffaf4d0 0x55960037eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.662+0000 7f29bc225740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x559600385f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29bc225740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559600378840 con 0x5595fffd01b0 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29bc225740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55960025db60 con 0x5595fffa72f0 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29bc225740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596003789c0 con 0x5595fffaf4d0 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29bc225740 10 client.0 _ll_register_callbacks cb 0x5596002822b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x559600385f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x559600385f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50298/0 (socket says 172.21.15.78:50298) 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 -- 172.21.15.78:0/1130404852 learned_addr learned my addr 172.21.15.78:0/1130404852 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29a7fff640 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffaf4d0 0x55960037eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffa72f0 0x55960037eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 -- 172.21.15.78:0/1130404852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffaf4d0 msgr2=0x55960037eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffaf4d0 0x55960037eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 -- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffa72f0 msgr2=0x55960037eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffa72f0 0x55960037eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 -- 172.21.15.78:0/1130404852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fffe8210 con 0x5595fffd01b0 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x559600385f50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005cc0 tx=0x7f29a8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5f6c93d2289258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29967fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29967fc640 1 -- 172.21.15.78:0/1130404852 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a801e020 con 0x5595fffd01b0 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29967fc640 1 -- 172.21.15.78:0/1130404852 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f29a8012c00 con 0x5595fffd01b0 2024-04-25T13:59:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.663+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a8004d20 con 0x5595fffd01b0 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29967fc640 1 -- 172.21.15.78:0/1130404852 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f29a8016070 con 0x5595fffd01b0 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 10 client.15447 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5596000b4620 con 0x5595fffd01b0 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29967fc640 1 -- 172.21.15.78:0/1130404852 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a8004d20 con 0x5595fffd01b0 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29967fc640 1 -- 172.21.15.78:0/1130404852 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f29a800f040 con 0x5595fffd01b0 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 10 client.15447 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 20 client.15447 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff8ed80 0x55960038c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 4 client.15447 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596002e4070 con 0x5595fff8ed80 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.664+0000 7f29a7fff640 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff8ed80 0x55960038c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.665+0000 7f29a7fff640 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff8ed80 0x55960038c5c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbea6d9847db8587 server_cookie=ae7ca640391d8eab in_seq=0 out_seq=0 2024-04-25T13:59:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.665+0000 7f29967fc640 10 client.15447 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29967fc640 1 -- 172.21.15.78:0/1130404852 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596002e4070 con 0x5595fff8ed80 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29967fc640 10 client.15447 handle_command_reply: tid=0 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 client.15447 shutdown 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 msgr2=0x559600385f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x559600385f50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005cc0 tx=0x7f29a8005c50 comp rx=0 tx=0).stop 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 shutdown_connections 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x559600385f50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff8ed80 0x55960038c5c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffaf4d0 0x55960037eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 --2- 172.21.15.78:0/1130404852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffa72f0 0x55960037eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29967fc640 0 client.15447 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 >> 172.21.15.78:0/1130404852 conn(0x5595fffc4300 msgr2=0x55960036b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 shutdown_connections 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 1 -- 172.21.15.78:0/1130404852 wait complete. 2024-04-25T13:59:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.666+0000 7f29bc225740 20 client.15447 trim_cache size 0 max 16384 2024-04-25T13:59:30.673 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:30 smithi129 ceph-mon[32569]: pgmap v348: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 Processor -- start 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 -- start start 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffa72f0 0x55960036bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600385e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x55960036c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01ab9c0 con 0x5595fffd01b0 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01a9290 con 0x5595fffaf4d0 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29bc225740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01a70e0 con 0x5595fffa72f0 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x55960036c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x55960036c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50306/0 (socket says 172.21.15.78:50306) 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.681+0000 7f29af7fe640 1 -- 172.21.15.78:0/1825867122 learned_addr learned my addr 172.21.15.78:0/1825867122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffa72f0 0x55960036bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29a7fff640 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600385e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29af7fe640 1 -- 172.21.15.78:0/1825867122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffa72f0 msgr2=0x55960036bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffa72f0 0x55960036bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29af7fe640 1 -- 172.21.15.78:0/1825867122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 msgr2=0x559600385e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600385e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29af7fe640 1 -- 172.21.15.78:0/1825867122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596003863b0 con 0x5595fffd01b0 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29af7fe640 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x55960036c120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009a80 tx=0x7f29a80021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13546eaf4b0c22b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29a4ff9640 1 -- 172.21.15.78:0/1825867122 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a801e070 con 0x5595fffd01b0 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29a4ff9640 1 -- 172.21.15.78:0/1825867122 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a8012760 con 0x5595fffd01b0 2024-04-25T13:59:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.682+0000 7f29a4ff9640 1 -- 172.21.15.78:0/1825867122 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a80051a0 con 0x5595fffd01b0 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 -- 172.21.15.78:0/1825867122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 msgr2=0x55960036c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x55960036c120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009a80 tx=0x7f29a80021f0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 -- 172.21.15.78:0/1825867122 shutdown_connections 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffd01b0 0x55960036c120 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x559600385e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 --2- 172.21.15.78:0/1825867122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffa72f0 0x55960036bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 -- 172.21.15.78:0/1825867122 >> 172.21.15.78:0/1825867122 conn(0x5595fffc4300 msgr2=0x559600374af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 -- 172.21.15.78:0/1825867122 shutdown_connections 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 -- 172.21.15.78:0/1825867122 wait complete. 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 -- start start 2024-04-25T13:59:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.683+0000 7f29bc225740 1 Processor -- start 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29bc225740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x5596003830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29bc225740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x5596003835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29bc225740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x5596003844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29bc225740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01a7750 con 0x5595fffa72f0 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29bc225740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01a8bc0 con 0x5595fffaf4d0 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29bc225740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01a7e20 con 0x5595fffd01b0 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29bc225740 10 client.0 _ll_register_callbacks cb 0x7f29a00e4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x5596003830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x5596003830b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50314/0 (socket says 172.21.15.78:50314) 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 -- 172.21.15.78:0/3061567654 learned_addr learned my addr 172.21.15.78:0/3061567654 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29a7fff640 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x5596003835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 -- 172.21.15.78:0/3061567654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 msgr2=0x5596003844b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x5596003844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 -- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 msgr2=0x5596003835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x5596003835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 -- 172.21.15.78:0/3061567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559600384a70 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29af7fe640 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x5596003830b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f29a8000f80 tx=0x7f29a8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c93dab0eba83651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29967fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.684+0000 7f29967fc640 1 -- 172.21.15.78:0/3061567654 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a801e040 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29967fc640 1 -- 172.21.15.78:0/3061567654 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f29a8023040 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a8012760 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29967fc640 1 -- 172.21.15.78:0/3061567654 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f29a801a4b0 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 10 client.15453 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559600385d00 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29967fc640 1 -- 172.21.15.78:0/3061567654 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a8012760 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29967fc640 1 -- 172.21.15.78:0/3061567654 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f29a803e610 con 0x5595fffa72f0 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 10 client.15453 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 20 client.15453 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff641c0 0x5596003f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 4 client.15453 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:59:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596002ddee0 con 0x5595fff641c0 2024-04-25T13:59:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.685+0000 7f29a7fff640 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff641c0 0x5596003f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.686+0000 7f29967fc640 10 client.15453 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.686+0000 7f29a7fff640 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff641c0 0x5596003f6c70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c94d3f6637c8e475 server_cookie=45d27e2ae5698600 in_seq=0 out_seq=0 2024-04-25T13:59:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29967fc640 1 -- 172.21.15.78:0/3061567654 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596002ddee0 con 0x5595fff641c0 2024-04-25T13:59:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29967fc640 10 client.15453 handle_command_reply: tid=0 2024-04-25T13:59:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 client.15453 shutdown 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 msgr2=0x5596003830b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x5596003830b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f29a8000f80 tx=0x7f29a8004b50 comp rx=0 tx=0).stop 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 shutdown_connections 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5595fff641c0 0x5596003f6c70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595fffd01b0 0x5596003844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595fffaf4d0 0x5596003835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29967fc640 0 client.15453 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 --2- 172.21.15.78:0/3061567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595fffa72f0 0x5596003830b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 >> 172.21.15.78:0/3061567654 conn(0x5595fffc4300 msgr2=0x55960036bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 shutdown_connections 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 1 -- 172.21.15.78:0/3061567654 wait complete. 2024-04-25T13:59:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.698+0000 7f29bc225740 20 client.15453 trim_cache size 0 max 16384 2024-04-25T13:59:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T13:59:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T13:59:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T13:59:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f299c0afae0 msgr2=0x7f299c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f299c0afae0 0x7f299c0b1ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x559600371d50 tx=0x7f29a00e4550 comp rx=0 tx=0).stop 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0102a20 msgr2=0x7f29b019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0102a20 0x7f29b019a220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f29a000d740 tx=0x7f29a000dc00 comp rx=0 tx=0).stop 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 shutdown_connections 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29b0108960 0x7f29b01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29b0103c60 0x7f29b019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f299c0afae0 0x7f299c0b1ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 --2- 172.21.15.78:0/66605719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29b0102a20 0x7f29b019a220 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 >> 172.21.15.78:0/66605719 conn(0x7f29b00fdc10 msgr2=0x7f29b0109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 shutdown_connections 2024-04-25T13:59:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:30.699+0000 7f29b5296640 1 -- 172.21.15.78:0/66605719 wait complete. 2024-04-25T13:59:30.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:30 smithi112 ceph-mon[32617]: pgmap v348: 97 pgs: 97 active+clean; 651 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T13:59:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:31 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:59:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:31 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:59:31.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:31 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:59:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:32 smithi078 ceph-mon[29163]: pgmap v349: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T13:59:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:32 smithi129 ceph-mon[32569]: pgmap v349: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T13:59:32.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:32 smithi112 ceph-mon[32617]: pgmap v349: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T13:59:34.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:59:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:59:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:59:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T13:59:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:35 smithi129 ceph-mon[32569]: pgmap v350: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T13:59:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:35 smithi078 ceph-mon[29163]: pgmap v350: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T13:59:35.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:35 smithi112 ceph-mon[32617]: pgmap v350: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T13:59:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:36 smithi129 ceph-mon[32569]: pgmap v351: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T13:59:36.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:36 smithi078 ceph-mon[29163]: pgmap v351: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T13:59:36.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:36 smithi112 ceph-mon[32617]: pgmap v351: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T13:59:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:38 smithi129 ceph-mon[32569]: pgmap v352: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T13:59:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:38 smithi078 ceph-mon[29163]: pgmap v352: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T13:59:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:38 smithi112 ceph-mon[32617]: pgmap v352: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-25T13:59:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:40 smithi129 ceph-mon[32569]: pgmap v353: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T13:59:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:40 smithi078 ceph-mon[29163]: pgmap v353: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T13:59:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:40 smithi112 ceph-mon[32617]: pgmap v353: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T13:59:42.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:42 smithi129 ceph-mon[32569]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-25T13:59:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:42 smithi078 ceph-mon[29163]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-25T13:59:42.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:42 smithi112 ceph-mon[32617]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-25T13:59:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.803+0000 7f9bc87ee640 1 -- 172.21.15.78:0/2539427048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 msgr2=0x7f9bc0104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.803+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/2539427048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc0104050 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009900 tx=0x7f9bb002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.803+0000 7f9bc87ee640 1 -- 172.21.15.78:0/2539427048 shutdown_connections 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.803+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/2539427048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bc0108940 0x7f9bc010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.803+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/2539427048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc0104050 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.803+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/2539427048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc0102a00 0x7f9bc0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.803+0000 7f9bc87ee640 1 -- 172.21.15.78:0/2539427048 >> 172.21.15.78:0/2539427048 conn(0x7f9bc00fdbd0 msgr2=0x7f9bc01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 -- 172.21.15.78:0/2539427048 shutdown_connections 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 -- 172.21.15.78:0/2539427048 wait complete. 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 Processor -- start 2024-04-25T13:59:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 -- start start 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bc0102a00 0x7f9bc0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc0108940 0x7f9bc01a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc010e530 con 0x7f9bc0103c40 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc010e3b0 con 0x7f9bc0108940 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc87ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc010e6b0 con 0x7f9bc0102a00 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.804+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34804/0 (socket says 172.21.15.78:34804) 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 -- 172.21.15.78:0/3281368208 learned_addr learned my addr 172.21.15.78:0/3281368208 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc658a640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bc0102a00 0x7f9bc0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc6d8b640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc0108940 0x7f9bc01a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 -- 172.21.15.78:0/3281368208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bc0102a00 msgr2=0x7f9bc0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bc0102a00 0x7f9bc0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 -- 172.21.15.78:0/3281368208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc0108940 msgr2=0x7f9bc01a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc0108940 0x7f9bc01a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 -- 172.21.15.78:0/3281368208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc01a1cc0 con 0x7f9bc0103c40 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc019efe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bb77fe640 1 -- 172.21.15.78:0/3281368208 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb003f030 con 0x7f9bc0103c40 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc01a1f50 con 0x7f9bc0103c40 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bb77fe640 1 -- 172.21.15.78:0/3281368208 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb0003aa0 con 0x7f9bc0103c40 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bb77fe640 1 -- 172.21.15.78:0/3281368208 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0035070 con 0x7f9bc0103c40 2024-04-25T13:59:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.805+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc01a2360 con 0x7f9bc0103c40 2024-04-25T13:59:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bb77fe640 1 -- 172.21.15.78:0/3281368208 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9bb0042030 con 0x7f9bc0103c40 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 Processor -- start 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 -- start start 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9451b0 0x5572aece5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9244d0 0x5572aececdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572aec4b460 con 0x5572ae9451b0 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572aec05590 con 0x5572ae91c2f0 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bcf776740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5572aec47420 con 0x5572ae9244d0 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bc6d8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9451b0 0x5572aece5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9244d0 0x5572aececdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bc6d8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9451b0 0x5572aece5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34806/0 (socket says 172.21.15.78:34806) 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bc6d8b640 1 -- 172.21.15.78:0/1661010361 learned_addr learned my addr 172.21.15.78:0/1661010361 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.808+0000 7f9bc658a640 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bb77fe640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9bac0afae0 0x7f9bac0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc5d89640 1 -- 172.21.15.78:0/1661010361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 msgr2=0x5572aece5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc5d89640 1 -- 172.21.15.78:0/1661010361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9451b0 msgr2=0x5572aece5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9451b0 0x5572aece5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc5d89640 1 -- 172.21.15.78:0/1661010361 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572aec120d0 con 0x5572ae9244d0 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc6d8b640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9bac0afae0 0x7f9bac0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bb77fe640 1 -- 172.21.15.78:0/3281368208 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9bb00ff1a0 con 0x7f9bc0103c40 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9244d0 0x5572aececdf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00fcc60 tx=0x7f9bb00c35e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9522296e9715e9f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bc6d8b640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9bac0afae0 0x7f9bac0b1ff0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009fd0 tx=0x7f9bb8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bb4ff9640 1 -- 172.21.15.78:0/1661010361 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0049020 con 0x5572ae9244d0 2024-04-25T13:59:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bb4ff9640 1 -- 172.21.15.78:0/1661010361 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb003b460 con 0x5572ae9244d0 2024-04-25T13:59:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.809+0000 7f9bb4ff9640 1 -- 172.21.15.78:0/1661010361 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00c74f0 con 0x5572ae9244d0 2024-04-25T13:59:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 -- 172.21.15.78:0/1661010361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9244d0 msgr2=0x5572aececdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9244d0 0x5572aececdf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00fcc60 tx=0x7f9bb00c35e0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 -- 172.21.15.78:0/1661010361 shutdown_connections 2024-04-25T13:59:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9244d0 0x5572aececdf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1661010361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9451b0 0x5572aece5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 -- 172.21.15.78:0/1661010361 >> 172.21.15.78:0/1661010361 conn(0x5572ae939300 msgr2=0x5572aece1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 -- 172.21.15.78:0/1661010361 shutdown_connections 2024-04-25T13:59:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 -- 172.21.15.78:0/1661010361 wait complete. 2024-04-25T13:59:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.810+0000 7f9bcf776740 1 -- start start 2024-04-25T13:59:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 1 Processor -- start 2024-04-25T13:59:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572aec47420 con 0x5572ae9244d0 2024-04-25T13:59:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5572aebd2b60 con 0x5572ae9451b0 2024-04-25T13:59:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5572aeced600 con 0x5572ae91c2f0 2024-04-25T13:59:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bcf776740 10 client.0 _ll_register_callbacks cb 0x5572aebf72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc658a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecef790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34814/0 (socket says 172.21.15.78:34814) 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 -- 172.21.15.78:0/889528414 learned_addr learned my addr 172.21.15.78:0/889528414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc6d8b640 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 -- 172.21.15.78:0/889528414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 msgr2=0x5572aecef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 -- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 msgr2=0x5572aecf66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 -- 172.21.15.78:0/889528414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ae95d210 con 0x5572ae9244d0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecef790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0004010 tx=0x7f9bb003b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a591a97765efc09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.811+0000 7f9ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/889528414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0049040 con 0x5572ae9244d0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/889528414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9bb00fc0e0 con 0x5572ae9244d0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572aec59070 con 0x5572ae9244d0 2024-04-25T13:59:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/889528414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9bb00c5af0 con 0x5572ae9244d0 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 10 client.15465 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5572aea29620 con 0x5572ae9244d0 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/889528414 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5572aec59070 con 0x5572ae9244d0 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/889528414 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9bb00fc460 con 0x5572ae9244d0 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 10 client.15465 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 20 client.15465 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae903d80 0x5572aecfcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 4 client.15465 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.812+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572aec52ee0 con 0x5572ae903d80 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.813+0000 7f9bc658a640 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae903d80 0x5572aecfcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.813+0000 7f9ba9ffb640 10 client.15465 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.813+0000 7f9bc658a640 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae903d80 0x5572aecfcdc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19def4fd91d0e5f7 server_cookie=f0be8a9318011092 in_seq=0 out_seq=0 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/889528414 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5572aec52ee0 con 0x5572ae903d80 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9ba9ffb640 10 client.15465 handle_command_reply: tid=0 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 client.15465 shutdown 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 msgr2=0x5572aecef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecef790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0004010 tx=0x7f9bb003b640 comp rx=0 tx=0).stop 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 shutdown_connections 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecef790 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae903d80 0x5572aecfcdc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 --2- 172.21.15.78:0/889528414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 >> 172.21.15.78:0/889528414 conn(0x5572ae939300 msgr2=0x5572aece4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9ba9ffb640 0 client.15465 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 shutdown_connections 2024-04-25T13:59:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 1 -- 172.21.15.78:0/889528414 wait complete. 2024-04-25T13:59:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.814+0000 7f9bcf776740 20 client.15465 trim_cache size 0 max 16384 2024-04-25T13:59:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 Processor -- start 2024-04-25T13:59:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 -- start start 2024-04-25T13:59:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9451b0 0x5572aece1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01adb60 con 0x5572ae9244d0 2024-04-25T13:59:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01ab430 con 0x5572ae91c2f0 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bcf776740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01a9280 con 0x5572ae9451b0 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34830/0 (socket says 172.21.15.78:34830) 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 -- 172.21.15.78:0/2279895355 learned_addr learned my addr 172.21.15.78:0/2279895355 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc6d8b640 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9451b0 0x5572aece1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc658a640 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 -- 172.21.15.78:0/2279895355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9451b0 msgr2=0x5572aece1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9451b0 0x5572aece1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 -- 172.21.15.78:0/2279895355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 msgr2=0x5572aece0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc5d89640 1 -- 172.21.15.78:0/2279895355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572aece65e0 con 0x5572ae9244d0 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc6d8b640 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9451b0 0x5572aece1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.829+0000 7f9bc658a640 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.830+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf6610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00ea140 tx=0x7f9bb00c3710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a9f76b7abed8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.830+0000 7f9babfff640 1 -- 172.21.15.78:0/2279895355 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0049070 con 0x5572ae9244d0 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.830+0000 7f9babfff640 1 -- 172.21.15.78:0/2279895355 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb00c7720 con 0x5572ae9244d0 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.830+0000 7f9babfff640 1 -- 172.21.15.78:0/2279895355 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00c5ce0 con 0x5572ae9244d0 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.830+0000 7f9bcf776740 1 -- 172.21.15.78:0/2279895355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 msgr2=0x5572aecf6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.830+0000 7f9bcf776740 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf6610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00ea140 tx=0x7f9bb00c3710 comp rx=0 tx=0).stop 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.830+0000 7f9bcf776740 1 -- 172.21.15.78:0/2279895355 shutdown_connections 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae9451b0 0x5572aece1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf6610 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 --2- 172.21.15.78:0/2279895355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae91c2f0 0x5572aece0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 -- 172.21.15.78:0/2279895355 >> 172.21.15.78:0/2279895355 conn(0x5572ae939300 msgr2=0x5572aecef480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 -- 172.21.15.78:0/2279895355 shutdown_connections 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 -- 172.21.15.78:0/2279895355 wait complete. 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 -- start start 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 Processor -- start 2024-04-25T13:59:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecf63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01a98f0 con 0x5572ae9244d0 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01aad60 con 0x5572ae9451b0 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01a9fc0 con 0x5572ae91c2f0 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bcf776740 10 client.0 _ll_register_callbacks cb 0x7f9bb8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bc5d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34836/0 (socket says 172.21.15.78:34836) 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bc5d89640 1 -- 172.21.15.78:0/1663277669 learned_addr learned my addr 172.21.15.78:0/1663277669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bc658a640 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecf63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.831+0000 7f9bc6d8b640 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bc5d89640 1 -- 172.21.15.78:0/1663277669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 msgr2=0x5572aecf63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecf63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bc5d89640 1 -- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 msgr2=0x5572aecf34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bc5d89640 1 -- 172.21.15.78:0/1663277669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572aecf6b50 con 0x5572ae9244d0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bc5d89640 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf2fa0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0032af0 tx=0x7f9bb0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219e102299f2f741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/1663277669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00e6060 con 0x5572ae9244d0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/1663277669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9bb00d0040 con 0x5572ae9244d0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb00c7720 con 0x5572ae9244d0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/1663277669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9bb0049070 con 0x5572ae9244d0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 10 client.15471 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5572aecf6b50 con 0x5572ae9244d0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/1663277669 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb00c7720 con 0x5572ae9244d0 2024-04-25T13:59:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/1663277669 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9bb00aebc0 con 0x5572ae9244d0 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 10 client.15471 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 20 client.15471 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae8d91c0 0x5572aed66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 4 client.15471 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.832+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572aebce5f0 con 0x5572ae8d91c0 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.833+0000 7f9bc658a640 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae8d91c0 0x5572aed66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.833+0000 7f9bc658a640 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae8d91c0 0x5572aed66600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6ddaac82aaf002 server_cookie=4f8dceb80fc74163 in_seq=0 out_seq=0 2024-04-25T13:59:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.833+0000 7f9ba9ffb640 10 client.15471 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9ba9ffb640 1 -- 172.21.15.78:0/1663277669 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572aebce5f0 con 0x5572ae8d91c0 2024-04-25T13:59:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9ba9ffb640 10 client.15471 handle_command_reply: tid=0 2024-04-25T13:59:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 client.15471 shutdown 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 msgr2=0x5572aecf2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf2fa0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0032af0 tx=0x7f9bb0005e50 comp rx=0 tx=0).stop 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 shutdown_connections 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5572ae9451b0 0x5572aecf34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572ae9244d0 0x5572aecf2fa0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5572ae8d91c0 0x5572aed66600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 --2- 172.21.15.78:0/1663277669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572ae91c2f0 0x5572aecf63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9ba9ffb640 0 client.15471 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 >> 172.21.15.78:0/1663277669 conn(0x5572ae939300 msgr2=0x5572aece4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 shutdown_connections 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 1 -- 172.21.15.78:0/1663277669 wait complete. 2024-04-25T13:59:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.837+0000 7f9bcf776740 20 client.15471 trim_cache size 0 max 16384 2024-04-25T13:59:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T13:59:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T13:59:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T13:59:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9bac0afae0 msgr2=0x7f9bac0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9bac0afae0 0x7f9bac0b1ff0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009fd0 tx=0x7f9bb8004a40 comp rx=0 tx=0).stop 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 msgr2=0x7f9bc019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc019efe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb0031a50 comp rx=0 tx=0).stop 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 shutdown_connections 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bc0108940 0x7f9bc01a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9bac0afae0 0x7f9bac0b1ff0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc0103c40 0x7f9bc019efe0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 --2- 172.21.15.78:0/3281368208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bc0102a00 0x7f9bc0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 >> 172.21.15.78:0/3281368208 conn(0x7f9bc00fdbd0 msgr2=0x7f9bc0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 shutdown_connections 2024-04-25T13:59:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:43.838+0000 7f9bc87ee640 1 -- 172.21.15.78:0/3281368208 wait complete. 2024-04-25T13:59:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:44 smithi129 ceph-mon[32569]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T13:59:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:44 smithi078 ceph-mon[29163]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T13:59:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:44 smithi112 ceph-mon[32617]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-25T13:59:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:46 smithi129 ceph-mon[32569]: pgmap v356: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:59:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:46 smithi078 ceph-mon[29163]: pgmap v356: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:59:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:46 smithi112 ceph-mon[32617]: pgmap v356: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T13:59:47.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:59:47.868 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:59:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 -- 172.21.15.78:0/1256368693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea0103c20 msgr2=0x7f4ea0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1256368693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea0103c20 0x7f4ea0104030 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009900 tx=0x7f4e9002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 -- 172.21.15.78:0/1256368693 shutdown_connections 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1256368693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea0108920 0x7f4ea010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1256368693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea0103c20 0x7f4ea0104030 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1256368693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ea01029e0 0x7f4ea01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 -- 172.21.15.78:0/1256368693 >> 172.21.15.78:0/1256368693 conn(0x7f4ea00fdbd0 msgr2=0x7f4ea0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 -- 172.21.15.78:0/1256368693 shutdown_connections 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.993+0000 7f4ea5985640 1 -- 172.21.15.78:0/1256368693 wait complete. 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 Processor -- start 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 -- start start 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea01029e0 0x7f4ea0199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea0103c20 0x7f4ea0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ea0108920 0x7f4ea01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea010e500 con 0x7f4ea01029e0 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea010e380 con 0x7f4ea0103c20 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4ea5985640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea010e680 con 0x7f4ea0108920 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4e9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea01029e0 0x7f4ea0199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4e9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea01029e0 0x7f4ea0199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42380/0 (socket says 172.21.15.78:42380) 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4e9effd640 1 -- 172.21.15.78:0/1151879859 learned_addr learned my addr 172.21.15.78:0/1151879859 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea0103c20 0x7f4ea0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.994+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ea0108920 0x7f4ea01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4e9effd640 1 -- 172.21.15.78:0/1151879859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ea0108920 msgr2=0x7f4ea01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4e9effd640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ea0108920 0x7f4ea01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4e9effd640 1 -- 172.21.15.78:0/1151879859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea0103c20 msgr2=0x7f4ea0199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4e9effd640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea0103c20 0x7f4ea0199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4e9effd640 1 -- 172.21.15.78:0/1151879859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea01a0c90 con 0x7f4ea01029e0 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4e9effd640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea01029e0 0x7f4ea0199150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800e9f0 tx=0x7f4e8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4ea4983640 1 -- 172.21.15.78:0/1151879859 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8800ccc0 con 0x7f4ea01029e0 2024-04-25T13:59:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4ea4983640 1 -- 172.21.15.78:0/1151879859 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e880035f0 con 0x7f4ea01029e0 2024-04-25T13:59:47.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea01a0f80 con 0x7f4ea01029e0 2024-04-25T13:59:47.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4ea4983640 1 -- 172.21.15.78:0/1151879859 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8800ce60 con 0x7f4ea01029e0 2024-04-25T13:59:47.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.995+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01a1390 con 0x7f4ea01029e0 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 Processor -- start 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 -- start start 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ed15390 0x563d4f0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecf46b0 0x563d4f0b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d4f01bd80 con 0x563d4ecf46b0 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563d4efd58b0 con 0x563d4ed15390 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4eac831740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563d4f017d40 con 0x563d4ecec4d0 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4ea4983640 1 -- 172.21.15.78:0/1151879859 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4e88004030 con 0x7f4ea01029e0 2024-04-25T13:59:48.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecf46b0 0x563d4f0b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecf46b0 0x563d4f0b9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42388/0 (socket says 172.21.15.78:42388) 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 -- 172.21.15.78:0/173357400 learned_addr learned my addr 172.21.15.78:0/173357400 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ed15390 0x563d4f0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9effd640 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 -- 172.21.15.78:0/173357400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 msgr2=0x563d4f0b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 -- 172.21.15.78:0/173357400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ed15390 msgr2=0x563d4f0b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ed15390 0x563d4f0b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.998+0000 7f4e9e7fc640 1 -- 172.21.15.78:0/173357400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4efa6a40 con 0x563d4ecf46b0 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.999+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecf46b0 0x563d4f0b9e80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4e90031500 tx=0x7f4e90031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf61355f70a6c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.999+0000 7f4ea4983640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4e8c0afaf0 0x7f4e8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.999+0000 7f4e95ffb640 1 -- 172.21.15.78:0/173357400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90035030 con 0x563d4ecf46b0 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.999+0000 7f4e95ffb640 1 -- 172.21.15.78:0/173357400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e90040070 con 0x563d4ecf46b0 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.999+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4e8c0afaf0 0x7f4e8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.999+0000 7f4e95ffb640 1 -- 172.21.15.78:0/173357400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90003910 con 0x563d4ecf46b0 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:47.999+0000 7f4ea4983640 1 -- 172.21.15.78:0/1151879859 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4e880defb0 con 0x7f4ea01029e0 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 -- 172.21.15.78:0/173357400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecf46b0 msgr2=0x563d4f0b9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4e8c0afaf0 0x7f4e8c0b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x563d4f0b9cc0 tx=0x7f4e98009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecf46b0 0x563d4f0b9e80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4e90031500 tx=0x7f4e90031b10 comp rx=0 tx=0).stop 2024-04-25T13:59:48.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 -- 172.21.15.78:0/173357400 shutdown_connections 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecf46b0 0x563d4f0b9e80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 --2- 172.21.15.78:0/173357400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ed15390 0x563d4f0b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 -- 172.21.15.78:0/173357400 >> 172.21.15.78:0/173357400 conn(0x563d4ed094e0 msgr2=0x563d4f0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 -- 172.21.15.78:0/173357400 shutdown_connections 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 -- 172.21.15.78:0/173357400 wait complete. 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 -- start start 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.000+0000 7f4eac831740 1 Processor -- start 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecec4d0 0x563d4f0c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ed15390 0x563d4f0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d4f0ba930 con 0x563d4ecec4d0 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563d4efa2d10 con 0x563d4ecf46b0 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563d4f0baab0 con 0x563d4ed15390 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 10 client.0 _ll_register_callbacks cb 0x563d4efcf210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecec4d0 0x563d4f0c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecec4d0 0x563d4f0c0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42396/0 (socket says 172.21.15.78:42396) 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 -- 172.21.15.78:0/2703733592 learned_addr learned my addr 172.21.15.78:0/2703733592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ed15390 0x563d4f0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 -- 172.21.15.78:0/2703733592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ed15390 msgr2=0x563d4f0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ed15390 0x563d4f0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 -- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 msgr2=0x563d4f0c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 -- 172.21.15.78:0/2703733592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4edf77d0 con 0x563d4ecec4d0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e9effd640 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecec4d0 0x563d4f0c0920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4e880a7410 tx=0x7f4e880a7250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5710ea6f52dc32a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e7effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e7effd640 1 -- 172.21.15.78:0/2703733592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88004320 con 0x563d4ecec4d0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4e7effd640 1 -- 172.21.15.78:0/2703733592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4e880e10f0 con 0x563d4ecec4d0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.001+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e90040070 con 0x563d4ecec4d0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4e7effd640 1 -- 172.21.15.78:0/2703733592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4e880a6f90 con 0x563d4ecec4d0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 10 client.15483 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563d4ed2d3f0 con 0x563d4ecec4d0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4e7effd640 1 -- 172.21.15.78:0/2703733592 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e90040070 con 0x563d4ecec4d0 2024-04-25T13:59:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4e7effd640 1 -- 172.21.15.78:0/2703733592 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4e88003aa0 con 0x563d4ecec4d0 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 10 client.15483 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 20 client.15483 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4ecd3f60 0x563d4f0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 4 client.15483 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d4f029990 con 0x563d4ecd3f60 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.002+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4ecd3f60 0x563d4f0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.003+0000 7f4e7effd640 10 client.15483 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.003+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4ecd3f60 0x563d4f0ce2f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a10fc7d7a2a012 server_cookie=a8c62c98b21da7e4 in_seq=0 out_seq=0 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4e7effd640 1 -- 172.21.15.78:0/2703733592 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d4f029990 con 0x563d4ecd3f60 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4e7effd640 10 client.15483 handle_command_reply: tid=0 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 client.15483 shutdown 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecec4d0 msgr2=0x563d4f0c0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecec4d0 0x563d4f0c0920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4e880a7410 tx=0x7f4e880a7250 comp rx=0 tx=0).stop 2024-04-25T13:59:48.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 shutdown_connections 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4ecd3f60 0x563d4f0ce2f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ed15390 0x563d4f0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4e7effd640 0 client.15483 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 --2- 172.21.15.78:0/2703733592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ecec4d0 0x563d4f0c0920 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 >> 172.21.15.78:0/2703733592 conn(0x563d4ed094e0 msgr2=0x563d4f0b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 shutdown_connections 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 1 -- 172.21.15.78:0/2703733592 wait complete. 2024-04-25T13:59:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.004+0000 7f4eac831740 20 client.15483 trim_cache size 0 max 16384 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 Processor -- start 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 -- start start 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0bff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0b5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f0d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01ab940 con 0x563d4ed15390 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01a9210 con 0x563d4ecf46b0 2024-04-25T13:59:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4eac831740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01a7060 con 0x563d4ecec4d0 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f0d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f0d2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42410/0 (socket says 172.21.15.78:42410) 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/3009663695 learned_addr learned my addr 172.21.15.78:0/3009663695 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9effd640 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/3009663695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 msgr2=0x563d4f0bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0bff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/3009663695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 msgr2=0x563d4f0b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/3009663695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4eced0f0 con 0x563d4ed15390 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.022+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f0d2aa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800ec40 tx=0x7f4e98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619de5ca843cb869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4e957fa640 1 -- 172.21.15.78:0/3009663695 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800ed30 con 0x563d4ed15390 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4e957fa640 1 -- 172.21.15.78:0/3009663695 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9801f910 con 0x563d4ed15390 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4e957fa640 1 -- 172.21.15.78:0/3009663695 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98005360 con 0x563d4ed15390 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 -- 172.21.15.78:0/3009663695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 msgr2=0x563d4f0d2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f0d2aa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800ec40 tx=0x7f4e98019000 comp rx=0 tx=0).stop 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 -- 172.21.15.78:0/3009663695 shutdown_connections 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f0d2aa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecf46b0 0x563d4f0b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 --2- 172.21.15.78:0/3009663695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecec4d0 0x563d4f0bff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 -- 172.21.15.78:0/3009663695 >> 172.21.15.78:0/3009663695 conn(0x563d4ed094e0 msgr2=0x563d4f0b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 -- 172.21.15.78:0/3009663695 shutdown_connections 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.023+0000 7f4eac831740 1 -- 172.21.15.78:0/3009663695 wait complete. 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 -- start start 2024-04-25T13:59:48.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 Processor -- start 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecec4d0 0x563d4f0c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecf46b0 0x563d4f0c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01a76d0 con 0x563d4ed15390 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01a8b40 con 0x563d4ecec4d0 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01a7da0 con 0x563d4ecf46b0 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4eac831740 10 client.0 _ll_register_callbacks cb 0x7f4e880045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4e9f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f134bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42424/0 (socket says 172.21.15.78:42424) 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/1963501505 learned_addr learned my addr 172.21.15.78:0/1963501505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4e9effd640 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecec4d0 0x563d4f0c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.024+0000 7f4e9e7fc640 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecf46b0 0x563d4f0c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/1963501505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecf46b0 msgr2=0x563d4f0c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecf46b0 0x563d4f0c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecec4d0 msgr2=0x563d4f0c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecec4d0 0x563d4f0c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e9f7fe640 1 -- 172.21.15.78:0/1963501505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4efdc4b0 con 0x563d4ed15390 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e9f7fe640 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f134bb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800ef10 tx=0x7f4e9800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d39deaa0fad0303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e7effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e7effd640 1 -- 172.21.15.78:0/1963501505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98004c50 con 0x563d4ed15390 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e7effd640 1 -- 172.21.15.78:0/1963501505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4e9802f040 con 0x563d4ed15390 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9801f910 con 0x563d4ed15390 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e7effd640 1 -- 172.21.15.78:0/1963501505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4e98004df0 con 0x563d4ed15390 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 10 client.15489 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563d4f136230 con 0x563d4ed15390 2024-04-25T13:59:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e7effd640 1 -- 172.21.15.78:0/1963501505 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9801f910 con 0x563d4ed15390 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4e7effd640 1 -- 172.21.15.78:0/1963501505 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4e9800f040 con 0x563d4ed15390 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 10 client.15489 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 20 client.15489 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4eca93a0 0x563d4f138c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 4 client.15489 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.025+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d4f023800 con 0x563d4eca93a0 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4e9effd640 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4eca93a0 0x563d4f138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4e7effd640 10 client.15489 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4e9effd640 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4eca93a0 0x563d4f138c40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5979f9c749417b server_cookie=c95b475469e63af5 in_seq=0 out_seq=0 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4e7effd640 1 -- 172.21.15.78:0/1963501505 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d4f023800 con 0x563d4eca93a0 2024-04-25T13:59:48.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4e7effd640 10 client.15489 handle_command_reply: tid=0 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 client.15489 shutdown 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 msgr2=0x563d4f134bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f134bb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800ef10 tx=0x7f4e9800ef40 comp rx=0 tx=0).stop 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 shutdown_connections 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d4ed15390 0x563d4f134bb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563d4eca93a0 0x563d4f138c40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d4ecf46b0 0x563d4f0c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 --2- 172.21.15.78:0/1963501505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563d4ecec4d0 0x563d4f0c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4e7effd640 0 client.15489 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.026+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 >> 172.21.15.78:0/1963501505 conn(0x563d4ed094e0 msgr2=0x563d4f0bef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 shutdown_connections 2024-04-25T13:59:48.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4eac831740 1 -- 172.21.15.78:0/1963501505 wait complete. 2024-04-25T13:59:48.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4eac831740 20 client.15489 trim_cache size 0 max 16384 2024-04-25T13:59:48.035 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:48.036 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T13:59:48.036 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T13:59:48.036 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4e8c0afaf0 msgr2=0x7f4e8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4e8c0afaf0 0x7f4e8c0b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x563d4f0b9cc0 tx=0x7f4e98009600 comp rx=0 tx=0).stop 2024-04-25T13:59:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea01029e0 msgr2=0x7f4ea0199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea01029e0 0x7f4ea0199150 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800e9f0 tx=0x7f4e8800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 shutdown_connections 2024-04-25T13:59:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ea0108920 0x7f4ea01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ea0103c20 0x7f4ea0199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4e8c0afaf0 0x7f4e8c0b2000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 --2- 172.21.15.78:0/1151879859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea01029e0 0x7f4ea0199150 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 >> 172.21.15.78:0/1151879859 conn(0x7f4ea00fdbd0 msgr2=0x7f4ea0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 shutdown_connections 2024-04-25T13:59:48.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.027+0000 7f4ea5985640 1 -- 172.21.15.78:0/1151879859 wait complete. 2024-04-25T13:59:48.040 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T13:59:48.040 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:59:48.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.148+0000 7f7a1aed1640 1 -- 172.21.15.78:0/2666195614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 msgr2=0x7f7a1410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.148+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/2666195614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1410ae20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009ca0 tx=0x7f7a0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.148+0000 7f7a1aed1640 1 -- 172.21.15.78:0/2666195614 shutdown_connections 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.148+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/2666195614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1410ae20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/2666195614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a141043b0 0x7f7a14102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/2666195614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a14103ed0 0x7f7a141027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 -- 172.21.15.78:0/2666195614 >> 172.21.15.78:0/2666195614 conn(0x7f7a140fd330 msgr2=0x7f7a140ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 -- 172.21.15.78:0/2666195614 shutdown_connections 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 -- 172.21.15.78:0/2666195614 wait complete. 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 Processor -- start 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 -- start start 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a14103ed0 0x7f7a14199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a1419f270 0x7f7a141a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1410e510 con 0x7f7a14108a00 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1410e390 con 0x7f7a14103ed0 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a1aed1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1410e690 con 0x7f7a1419f270 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.149+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42448/0 (socket says 172.21.15.78:42448) 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 -- 172.21.15.78:0/1470249937 learned_addr learned my addr 172.21.15.78:0/1470249937 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a18c6d640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a14103ed0 0x7f7a14199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a1946e640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a1419f270 0x7f7a141a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 -- 172.21.15.78:0/1470249937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a1419f270 msgr2=0x7f7a141a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a1419f270 0x7f7a141a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 -- 172.21.15.78:0/1470249937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a14103ed0 msgr2=0x7f7a14199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a14103ed0 0x7f7a14199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 -- 172.21.15.78:0/1470249937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a141a1d00 con 0x7f7a14108a00 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a13fff640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1419a2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800d950 tx=0x7f7a0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a11ffb640 1 -- 172.21.15.78:0/1470249937 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08017030 con 0x7f7a14108a00 2024-04-25T13:59:48.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a11ffb640 1 -- 172.21.15.78:0/1470249937 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a08004020 con 0x7f7a14108a00 2024-04-25T13:59:48.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a11ffb640 1 -- 172.21.15.78:0/1470249937 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a080047a0 con 0x7f7a14108a00 2024-04-25T13:59:48.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a141a1ff0 con 0x7f7a14108a00 2024-04-25T13:59:48.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.150+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a141a24b0 con 0x7f7a14108a00 2024-04-25T13:59:48.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.151+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a141a2770 con 0x7f7a14108a00 2024-04-25T13:59:48.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.153+0000 7f7a11ffb640 1 -- 172.21.15.78:0/1470249937 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7a08009a30 con 0x7f7a14108a00 2024-04-25T13:59:48.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.153+0000 7f7a11ffb640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f79fc0afb70 0x7f79fc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.154+0000 7f7a18c6d640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f79fc0afb70 0x7f79fc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.154+0000 7f7a11ffb640 1 -- 172.21.15.78:0/1470249937 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7a080de540 con 0x7f7a14108a00 2024-04-25T13:59:48.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.154+0000 7f7a18c6d640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f79fc0afb70 0x7f79fc0b2080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a14069cb0 tx=0x7f7a04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.158+0000 7f7a11ffb640 1 -- 172.21.15.78:0/1470249937 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a080a7110 con 0x7f7a14108a00 2024-04-25T13:59:48.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.376+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a141a2940 con 0x7f7a14108a00 2024-04-25T13:59:48.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.377+0000 7f7a11ffb640 1 -- 172.21.15.78:0/1470249937 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f7a080a6a10 con 0x7f7a14108a00 2024-04-25T13:59:48.377 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T13:59:48.377 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T13:59:48.377 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T13:59:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f79fc0afb70 msgr2=0x7f79fc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f79fc0afb70 0x7f79fc0b2080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7a14069cb0 tx=0x7f7a04005f90 comp rx=0 tx=0).stop 2024-04-25T13:59:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 msgr2=0x7f7a1419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1419a2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800d950 tx=0x7f7a0800de10 comp rx=0 tx=0).stop 2024-04-25T13:59:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 shutdown_connections 2024-04-25T13:59:48.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a1419f270 0x7f7a141a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f79fc0afb70 0x7f79fc0b2080 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a14108a00 0x7f7a1419a2f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 --2- 172.21.15.78:0/1470249937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a14103ed0 0x7f7a14199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 >> 172.21.15.78:0/1470249937 conn(0x7f7a140fd330 msgr2=0x7f7a141016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 shutdown_connections 2024-04-25T13:59:48.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.379+0000 7f7a1aed1640 1 -- 172.21.15.78:0/1470249937 wait complete. 2024-04-25T13:59:48.391 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.501+0000 7f7c492e0640 1 -- 172.21.15.78:0/309164846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 msgr2=0x7f7c4410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.501+0000 7f7c492e0640 1 --2- 172.21.15.78:0/309164846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c4410bcb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009ca0 tx=0x7f7c3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 -- 172.21.15.78:0/309164846 shutdown_connections 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 --2- 172.21.15.78:0/309164846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c4410bcb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 --2- 172.21.15.78:0/309164846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c44106f30 0x7f7c44109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 --2- 172.21.15.78:0/309164846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c441045d0 0x7f7c441069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 -- 172.21.15.78:0/309164846 >> 172.21.15.78:0/309164846 conn(0x7f7c440fdbd0 msgr2=0x7f7c44100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 -- 172.21.15.78:0/309164846 shutdown_connections 2024-04-25T13:59:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 -- 172.21.15.78:0/309164846 wait complete. 2024-04-25T13:59:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 Processor -- start 2024-04-25T13:59:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.502+0000 7f7c492e0640 1 -- start start 2024-04-25T13:59:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c441045d0 0x7f7c4419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c44106f30 0x7f7c4419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c441a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410f610 con 0x7f7c44109890 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410f490 con 0x7f7c44106f30 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410f790 con 0x7f7c441045d0 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c441a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c441a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42460/0 (socket says 172.21.15.78:42460) 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 -- 172.21.15.78:0/3209915374 learned_addr learned my addr 172.21.15.78:0/3209915374 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c441045d0 0x7f7c4419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 -- 172.21.15.78:0/3209915374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c441045d0 msgr2=0x7f7c4419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c441045d0 0x7f7c4419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c427fc640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c44106f30 0x7f7c4419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 -- 172.21.15.78:0/3209915374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c44106f30 msgr2=0x7f7c4419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c44106f30 0x7f7c4419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 -- 172.21.15.78:0/3209915374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c441a2080 con 0x7f7c44109890 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c441045d0 0x7f7c4419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c437fe640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c441a1a10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009430 tx=0x7f7c3c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c2bfff640 1 -- 172.21.15.78:0/3209915374 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c02f380 con 0x7f7c44109890 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c441a2310 con 0x7f7c44109890 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c2bfff640 1 -- 172.21.15.78:0/3209915374 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c3c0317c0 con 0x7f7c44109890 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c2bfff640 1 -- 172.21.15.78:0/3209915374 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c02fcd0 con 0x7f7c44109890 2024-04-25T13:59:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.503+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441a2860 con 0x7f7c44109890 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c2bfff640 1 -- 172.21.15.78:0/3209915374 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7c3c004040 con 0x7f7c44109890 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 Processor -- start 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 -- start start 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cda2390 0x558b3d144810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd794d0 0x558b3d144d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 0x558b3d145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558b3d0a8d80 con 0x558b3cd794d0 2024-04-25T13:59:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558b3d062a20 con 0x558b3cda2390 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c50194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558b3d0a4d40 con 0x558b3cd816b0 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd794d0 0x558b3d144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd794d0 0x558b3d144d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42468/0 (socket says 172.21.15.78:42468) 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 -- 172.21.15.78:0/2437705964 learned_addr learned my addr 172.21.15.78:0/2437705964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c427fc640 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 0x558b3d145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c2bfff640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7c2c0afc90 0x7f7c2c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c427fc640 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cda2390 0x558b3d144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 -- 172.21.15.78:0/2437705964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 msgr2=0x558b3d145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 0x558b3d145a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 -- 172.21.15.78:0/2437705964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cda2390 msgr2=0x558b3d144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cda2390 0x558b3d144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c437fe640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7c2c0afc90 0x7f7c2c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c42ffd640 1 -- 172.21.15.78:0/2437705964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3d06dc30 con 0x558b3cd794d0 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.506+0000 7f7c427fc640 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cda2390 0x558b3d144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.507+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd794d0 0x558b3d144d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009c20 tx=0x7f7c3400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932f5f034952a408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.507+0000 7f7c2bfff640 1 -- 172.21.15.78:0/3209915374 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7c3c100370 con 0x7f7c44109890 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.507+0000 7f7c297fa640 1 -- 172.21.15.78:0/2437705964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34016070 con 0x558b3cd794d0 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.507+0000 7f7c297fa640 1 -- 172.21.15.78:0/2437705964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c34011920 con 0x558b3cd794d0 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.507+0000 7f7c297fa640 1 -- 172.21.15.78:0/2437705964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340107d0 con 0x558b3cd794d0 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- 172.21.15.78:0/2437705964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd794d0 msgr2=0x558b3d144d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c437fe640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7c2c0afc90 0x7f7c2c0b21a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c092920 tx=0x7f7c3c0c2e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd794d0 0x558b3d144d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009c20 tx=0x7f7c3400e6c0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- 172.21.15.78:0/2437705964 shutdown_connections 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 0x558b3d145a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd794d0 0x558b3d144d90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 --2- 172.21.15.78:0/2437705964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cda2390 0x558b3d144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- 172.21.15.78:0/2437705964 >> 172.21.15.78:0/2437705964 conn(0x558b3cd964e0 msgr2=0x558b3d140020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- 172.21.15.78:0/2437705964 shutdown_connections 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- 172.21.15.78:0/2437705964 wait complete. 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- start start 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 Processor -- start 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d14e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cda2390 0x558b3d1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558b3d146530 con 0x558b3cd816b0 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558b3d02fdc0 con 0x558b3cd794d0 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558b3d1466b0 con 0x558b3cda2390 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.508+0000 7f7c50194740 10 client.0 _ll_register_callbacks cb 0x558b3d04b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d14ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42474/0 (socket says 172.21.15.78:42474) 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 -- 172.21.15.78:0/3937976971 learned_addr learned my addr 172.21.15.78:0/3937976971 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c427fc640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c427fc640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cda2390 0x558b3d1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 -- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cda2390 msgr2=0x558b3d1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cda2390 0x558b3d1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 -- 172.21.15.78:0/3937976971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 msgr2=0x558b3d14e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d14e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 -- 172.21.15.78:0/3937976971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3cdba3f0 con 0x558b3cd816b0 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c427fc640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d14e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d14ea80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c340097e0 tx=0x7f7c34009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3285ba0788da53ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c26ff5640 1 -- 172.21.15.78:0/3937976971 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34016050 con 0x558b3cd816b0 2024-04-25T13:59:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c26ff5640 1 -- 172.21.15.78:0/3937976971 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7c34005040 con 0x558b3cd816b0 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c34011920 con 0x558b3cd816b0 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c26ff5640 1 -- 172.21.15.78:0/3937976971 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7c34005240 con 0x558b3cd816b0 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c50194740 10 client.15507 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558b3ce847d0 con 0x558b3cd816b0 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.509+0000 7f7c26ff5640 1 -- 172.21.15.78:0/3937976971 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c34011920 con 0x558b3cd816b0 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c26ff5640 1 -- 172.21.15.78:0/3937976971 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7c3402c030 con 0x558b3cd816b0 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c50194740 10 client.15507 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c50194740 10 client.15507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c50194740 20 client.15507 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c50194740 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cd60f60 0x558b3d15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c50194740 4 client.15507 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b3d0b6990 con 0x558b3cd60f60 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c427fc640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cd60f60 0x558b3d15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c26ff5640 10 client.15507 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.510+0000 7f7c427fc640 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cd60f60 0x558b3d15bf20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb508286d8139d5a server_cookie=cf7c738029174851 in_seq=0 out_seq=0 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c26ff5640 1 -- 172.21.15.78:0/3937976971 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b3d0b6990 con 0x558b3cd60f60 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c26ff5640 10 client.15507 handle_command_reply: tid=0 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 client.15507 shutdown 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 msgr2=0x558b3d14ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d14ea80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7c340097e0 tx=0x7f7c34009810 comp rx=0 tx=0).stop 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 shutdown_connections 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cd60f60 0x558b3d15bf20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cda2390 0x558b3d1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c26ff5640 0 client.15507 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d14ea80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 --2- 172.21.15.78:0/3937976971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d14e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 >> 172.21.15.78:0/3937976971 conn(0x558b3cd964e0 msgr2=0x558b3d142f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 shutdown_connections 2024-04-25T13:59:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.511+0000 7f7c50194740 1 -- 172.21.15.78:0/3937976971 wait complete. 2024-04-25T13:59:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.512+0000 7f7c50194740 20 client.15507 trim_cache size 0 max 16384 2024-04-25T13:59:48.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 Processor -- start 2024-04-25T13:59:48.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 -- start start 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd60f60 0x558b3d13b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d1616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 0x558b3d161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441adb60 con 0x558b3cd60f60 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441ab430 con 0x558b3cd794d0 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c50194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441a9280 con 0x558b3cd816b0 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd60f60 0x558b3d13b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd60f60 0x558b3d13b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42478/0 (socket says 172.21.15.78:42478) 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c427fc640 1 -- 172.21.15.78:0/4180409207 learned_addr learned my addr 172.21.15.78:0/4180409207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c427fc640 1 -- 172.21.15.78:0/4180409207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 msgr2=0x558b3d161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c427fc640 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 0x558b3d161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d1616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c427fc640 1 -- 172.21.15.78:0/4180409207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 msgr2=0x558b3d1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.525+0000 7f7c427fc640 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d1616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.526+0000 7f7c427fc640 1 -- 172.21.15.78:0/4180409207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3d1443d0 con 0x558b3cd60f60 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.526+0000 7f7c427fc640 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd60f60 0x558b3d13b190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c380058c0 tx=0x7f7c3800e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f8ad483a8e990f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.526+0000 7f7c28ff9640 1 -- 172.21.15.78:0/4180409207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38011a90 con 0x558b3cd60f60 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.526+0000 7f7c28ff9640 1 -- 172.21.15.78:0/4180409207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38023070 con 0x558b3cd60f60 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.526+0000 7f7c28ff9640 1 -- 172.21.15.78:0/4180409207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38004a00 con 0x558b3cd60f60 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.526+0000 7f7c50194740 1 -- 172.21.15.78:0/4180409207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd60f60 msgr2=0x558b3d13b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.526+0000 7f7c50194740 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd60f60 0x558b3d13b190 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c380058c0 tx=0x7f7c3800e740 comp rx=0 tx=0).stop 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- 172.21.15.78:0/4180409207 shutdown_connections 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd816b0 0x558b3d161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd794d0 0x558b3d1616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 --2- 172.21.15.78:0/4180409207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd60f60 0x558b3d13b190 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- 172.21.15.78:0/4180409207 >> 172.21.15.78:0/4180409207 conn(0x558b3cd964e0 msgr2=0x558b3d149270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- 172.21.15.78:0/4180409207 shutdown_connections 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- 172.21.15.78:0/4180409207 wait complete. 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- start start 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 Processor -- start 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd60f60 0x558b3d154340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd794d0 0x558b3d154880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d1c2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441a98f0 con 0x558b3cd816b0 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441aad60 con 0x558b3cd60f60 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441a9fc0 con 0x558b3cd794d0 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c50194740 10 client.0 _ll_register_callbacks cb 0x7f7c3800e5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d1c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d1c2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42486/0 (socket says 172.21.15.78:42486) 2024-04-25T13:59:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c427fc640 1 -- 172.21.15.78:0/1015933926 learned_addr learned my addr 172.21.15.78:0/1015933926 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c427fc640 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd60f60 0x558b3d154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.527+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd794d0 0x558b3d154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c427fc640 1 -- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd794d0 msgr2=0x558b3d154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c427fc640 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd794d0 0x558b3d154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c427fc640 1 -- 172.21.15.78:0/1015933926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd60f60 msgr2=0x558b3d154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c427fc640 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd60f60 0x558b3d154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c427fc640 1 -- 172.21.15.78:0/1015933926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3d1c2a10 con 0x558b3cd816b0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c427fc640 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d1c2420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x558b3d1bc380 tx=0x7f7c38021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fbf9a2f7965155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c26ff5640 1 -- 172.21.15.78:0/1015933926 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3801f400 con 0x558b3cd816b0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c26ff5640 1 -- 172.21.15.78:0/1015933926 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7c3801f5a0 con 0x558b3cd816b0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c38023070 con 0x558b3cd816b0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c26ff5640 1 -- 172.21.15.78:0/1015933926 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7c3803d050 con 0x558b3cd816b0 2024-04-25T13:59:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c50194740 10 client.15513 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558b3d1c3cc0 con 0x558b3cd816b0 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c26ff5640 1 -- 172.21.15.78:0/1015933926 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38023070 con 0x558b3cd816b0 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c26ff5640 1 -- 172.21.15.78:0/1015933926 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7c38035020 con 0x558b3cd816b0 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c50194740 10 client.15513 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c50194740 10 client.15513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c50194740 20 client.15513 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.528+0000 7f7c50194740 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cda2390 0x558b3d1c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c50194740 4 client.15513 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b3d0b0800 con 0x558b3cda2390 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cda2390 0x558b3d1c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c26ff5640 10 client.15513 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cda2390 0x558b3d1c6690 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb1fa117c696a612 server_cookie=ef55a7d603ab1c6 in_seq=0 out_seq=0 2024-04-25T13:59:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c26ff5640 1 -- 172.21.15.78:0/1015933926 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b3d0b0800 con 0x558b3cda2390 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c26ff5640 10 client.15513 handle_command_reply: tid=0 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.529+0000 7f7c50194740 1 client.15513 shutdown 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 msgr2=0x558b3d1c2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d1c2420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x558b3d1bc380 tx=0x7f7c38021040 comp rx=0 tx=0).stop 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 shutdown_connections 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x558b3cda2390 0x558b3d1c6690 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b3cd816b0 0x558b3d1c2420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3cd794d0 0x558b3d154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 --2- 172.21.15.78:0/1015933926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b3cd60f60 0x558b3d154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 >> 172.21.15.78:0/1015933926 conn(0x558b3cd964e0 msgr2=0x558b3d14cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c26ff5640 0 client.15513 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 shutdown_connections 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 1 -- 172.21.15.78:0/1015933926 wait complete. 2024-04-25T13:59:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c50194740 20 client.15513 trim_cache size 0 max 16384 2024-04-25T13:59:48.540 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7c2c0afc90 msgr2=0x7f7c2c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7c2c0afc90 0x7f7c2c0b21a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c092920 tx=0x7f7c3c0c2e30 comp rx=0 tx=0).stop 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 msgr2=0x7f7c441a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c441a1a10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009430 tx=0x7f7c3c003ba0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 shutdown_connections 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7c2c0afc90 0x7f7c2c0b21a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c44109890 0x7f7c441a1a10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c44106f30 0x7f7c4419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 --2- 172.21.15.78:0/3209915374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c441045d0 0x7f7c4419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 >> 172.21.15.78:0/3209915374 conn(0x7f7c440fdbd0 msgr2=0x7f7c44105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 shutdown_connections 2024-04-25T13:59:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.530+0000 7f7c492e0640 1 -- 172.21.15.78:0/3209915374 wait complete. 2024-04-25T13:59:48.542 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:48 smithi078 ceph-mon[29163]: pgmap v357: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T13:59:48.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T13:59:48.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/200119794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0102250 msgr2=0x7f7ac010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/200119794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0102250 0x7f7ac010bcb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009ca0 tx=0x7f7ab802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/200119794 shutdown_connections 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/200119794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0102250 0x7f7ac010bcb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/200119794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac0101750 0x7f7ac0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/200119794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ac01050a0 0x7f7ac01054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/200119794 >> 172.21.15.78:0/200119794 conn(0x7f7ac00fd350 msgr2=0x7f7ac00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/200119794 shutdown_connections 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.653+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/200119794 wait complete. 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 Processor -- start 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 -- start start 2024-04-25T13:59:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0101750 0x7f7ac019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ac0102250 0x7f7ac019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac01050a0 0x7f7ac01a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac010f610 con 0x7f7ac0101750 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac010f490 con 0x7f7ac01050a0 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac7ce0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac010f790 con 0x7f7ac0102250 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac5a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0101750 0x7f7ac019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac5a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0101750 0x7f7ac019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42496/0 (socket says 172.21.15.78:42496) 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/4057330510 learned_addr learned my addr 172.21.15.78:0/4057330510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac527b640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ac0102250 0x7f7ac019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.654+0000 7f7ac627d640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac01050a0 0x7f7ac01a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/4057330510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ac0102250 msgr2=0x7f7ac019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ac0102250 0x7f7ac019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/4057330510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac01050a0 msgr2=0x7f7ac01a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac01050a0 0x7f7ac01a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/4057330510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac01a1c90 con 0x7f7ac0101750 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac627d640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac01050a0 0x7f7ac01a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0101750 0x7f7ac019a150 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e9f0 tx=0x7f7abc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/4057330510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc00ccc0 con 0x7f7ac0101750 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/4057330510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7abc0035f0 con 0x7f7ac0101750 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac01a1f80 con 0x7f7ac0101750 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/4057330510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc00ce60 con 0x7f7ac0101750 2024-04-25T13:59:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.655+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac01a23e0 con 0x7f7ac0101750 2024-04-25T13:59:48.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:48 smithi129 ceph-mon[32569]: pgmap v357: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.657+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/4057330510 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7abc003aa0 con 0x7f7ac0101750 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 Processor -- start 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 -- start start 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e32390 0x5622f61d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622f6138d80 con 0x5622f5e116b0 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622f60f2a20 con 0x5622f5e32390 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7aceb94740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5622f6134d40 con 0x5622f5e094d0 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7ac527b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7ab6ffd640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7aac0afaf0 0x7f7aac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7ac527b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61d5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42508/0 (socket says 172.21.15.78:42508) 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7ac527b640 1 -- 172.21.15.78:0/1518210623 learned_addr learned my addr 172.21.15.78:0/1518210623 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.658+0000 7f7ac627d640 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e32390 0x5622f61d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac627d640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7aac0afaf0 0x7f7aac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/4057330510 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7abc0defd0 con 0x7f7ac0101750 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac527b640 1 -- 172.21.15.78:0/1518210623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 msgr2=0x5622f61d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac527b640 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61d4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac527b640 1 -- 172.21.15.78:0/1518210623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e32390 msgr2=0x5622f61d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac527b640 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e32390 0x5622f61d4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac527b640 1 -- 172.21.15.78:0/1518210623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f60fdc30 con 0x5622f5e116b0 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac527b640 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61d5a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000bcb0 tx=0x7f7ab00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2559a2a2aa22d415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7ac627d640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7aac0afaf0 0x7f7aac0b2000 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80094e0 tx=0x7f7ab80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7aabfff640 1 -- 172.21.15.78:0/1518210623 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab001c480 con 0x5622f5e116b0 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7aabfff640 1 -- 172.21.15.78:0/1518210623 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab001cad0 con 0x5622f5e116b0 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.659+0000 7f7aabfff640 1 -- 172.21.15.78:0/1518210623 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0011b80 con 0x5622f5e116b0 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 -- 172.21.15.78:0/1518210623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 msgr2=0x5622f61d5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61d5a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000bcb0 tx=0x7f7ab00107a0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 -- 172.21.15.78:0/1518210623 shutdown_connections 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61d5a20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 --2- 172.21.15.78:0/1518210623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e32390 0x5622f61d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 -- 172.21.15.78:0/1518210623 >> 172.21.15.78:0/1518210623 conn(0x5622f5e264e0 msgr2=0x5622f61d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 -- 172.21.15.78:0/1518210623 shutdown_connections 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 -- 172.21.15.78:0/1518210623 wait complete. 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.660+0000 7f7aceb94740 1 -- start start 2024-04-25T13:59:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 1 Processor -- start 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e094d0 0x5622f61de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f61deb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e32390 0x5622f61e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622f61d6530 con 0x5622f5e094d0 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5622f60bfdc0 con 0x5622f5e116b0 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5622f61d66b0 con 0x5622f5e32390 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7aceb94740 10 client.0 _ll_register_callbacks cb 0x5622f60db600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e094d0 0x5622f61de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e094d0 0x5622f61de630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42514/0 (socket says 172.21.15.78:42514) 2024-04-25T13:59:48.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/2949273763 learned_addr learned my addr 172.21.15.78:0/2949273763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac627d640 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e32390 0x5622f61e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac527b640 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f61deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e32390 msgr2=0x5622f61e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e32390 0x5622f61e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/2949273763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 msgr2=0x5622f61deb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f61deb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.661+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/2949273763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f5e4a3f0 con 0x5622f5e094d0 2024-04-25T13:59:48.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e094d0 0x5622f61de630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e9c0 tx=0x7f7abc071cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd20f6d609dab1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aa97fa640 1 -- 172.21.15.78:0/2949273763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc06de20 con 0x5622f5e094d0 2024-04-25T13:59:48.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aa97fa640 1 -- 172.21.15.78:0/2949273763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7abc0e10b0 con 0x5622f5e094d0 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab001cad0 con 0x5622f5e094d0 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aa97fa640 1 -- 172.21.15.78:0/2949273763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7abc0e12d0 con 0x5622f5e094d0 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 10 client.15525 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5622f5f147d0 con 0x5622f5e094d0 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aa97fa640 1 -- 172.21.15.78:0/2949273763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab001cad0 con 0x5622f5e094d0 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aa97fa640 1 -- 172.21.15.78:0/2949273763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7abc0a7700 con 0x5622f5e094d0 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 10 client.15525 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 10 client.15525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 20 client.15525 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5df0f60 0x5622f61ec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 4 client.15525 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.662+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622f6146990 con 0x5622f5df0f60 2024-04-25T13:59:48.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.663+0000 7f7ac527b640 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5df0f60 0x5622f61ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.663+0000 7f7aa97fa640 10 client.15525 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.663+0000 7f7ac527b640 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5df0f60 0x5622f61ec370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d15a092216b6710f server_cookie=2c288f328500ecb1 in_seq=0 out_seq=0 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aa97fa640 1 -- 172.21.15.78:0/2949273763 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622f6146990 con 0x5622f5df0f60 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aa97fa640 10 client.15525 handle_command_reply: tid=0 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 client.15525 shutdown 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e094d0 msgr2=0x5622f61de630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e094d0 0x5622f61de630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e9c0 tx=0x7f7abc071cc0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 shutdown_connections 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5df0f60 0x5622f61ec370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e32390 0x5622f61e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f61deb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aa97fa640 0 client.15525 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:48.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2949273763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e094d0 0x5622f61de630 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 >> 172.21.15.78:0/2949273763 conn(0x5622f5e264e0 msgr2=0x5622f61d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 shutdown_connections 2024-04-25T13:59:48.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 1 -- 172.21.15.78:0/2949273763 wait complete. 2024-04-25T13:59:48.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.664+0000 7f7aceb94740 20 client.15525 trim_cache size 0 max 16384 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 Processor -- start 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 -- start start 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5df0f60 0x5622f61d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61de200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61de740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01adb70 con 0x5622f5e116b0 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01ab440 con 0x5622f5df0f60 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7aceb94740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01a9290 con 0x5622f5e094d0 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61de740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61de740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42520/0 (socket says 172.21.15.78:42520) 2024-04-25T13:59:48.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 -- 172.21.15.78:0/2631232447 learned_addr learned my addr 172.21.15.78:0/2631232447 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac527b640 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61de200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5df0f60 0x5622f61d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 -- 172.21.15.78:0/2631232447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 msgr2=0x5622f61de200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61de200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 -- 172.21.15.78:0/2631232447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5df0f60 msgr2=0x5622f61d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5df0f60 0x5622f61d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.678+0000 7f7ac627d640 1 -- 172.21.15.78:0/2631232447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f61df6d0 con 0x5622f5e116b0 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7ac627d640 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61de740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8035000 tx=0x7f7ab8043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f09a0a78602b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aab7fe640 1 -- 172.21.15.78:0/2631232447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8004040 con 0x5622f5e116b0 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aab7fe640 1 -- 172.21.15.78:0/2631232447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab8036040 con 0x5622f5e116b0 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aab7fe640 1 -- 172.21.15.78:0/2631232447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab802fb60 con 0x5622f5e116b0 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 -- 172.21.15.78:0/2631232447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 msgr2=0x5622f61de740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61de740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8035000 tx=0x7f7ab8043000 comp rx=0 tx=0).stop 2024-04-25T13:59:48.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 -- 172.21.15.78:0/2631232447 shutdown_connections 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5e116b0 0x5622f61de740 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61de200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 --2- 172.21.15.78:0/2631232447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5df0f60 0x5622f61d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 -- 172.21.15.78:0/2631232447 >> 172.21.15.78:0/2631232447 conn(0x5622f5e264e0 msgr2=0x5622f61cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.679+0000 7f7aceb94740 1 -- 172.21.15.78:0/2631232447 shutdown_connections 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 -- 172.21.15.78:0/2631232447 wait complete. 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 -- start start 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 Processor -- start 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5df0f60 0x5622f61e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61e2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f62526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01a9900 con 0x5622f5df0f60 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01aad70 con 0x5622f5e116b0 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac01a9fd0 con 0x5622f5e094d0 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7aceb94740 10 client.0 _ll_register_callbacks cb 0x7f7abc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7ac5a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5df0f60 0x5622f61e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7ac5a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5df0f60 0x5622f61e1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42526/0 (socket says 172.21.15.78:42526) 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/3476013088 learned_addr learned my addr 172.21.15.78:0/3476013088 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.680+0000 7f7ac527b640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac627d640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f62526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 msgr2=0x5622f61e2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61e2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/3476013088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 msgr2=0x5622f62526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f62526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac5a7c640 1 -- 172.21.15.78:0/3476013088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f61e5340 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac627d640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f62526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7ac5a7c640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5df0f60 0x5622f61e1e40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0a3a40 tx=0x7f7abc0b00a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a933bdd66340037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aa97fa640 1 -- 172.21.15.78:0/3476013088 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0a3360 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aa97fa640 1 -- 172.21.15.78:0/3476013088 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7abc0e1070 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab8036040 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aa97fa640 1 -- 172.21.15.78:0/3476013088 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7abc0a6c10 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aceb94740 10 client.15531 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5622f6253be0 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aa97fa640 1 -- 172.21.15.78:0/3476013088 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab8036040 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aa97fa640 1 -- 172.21.15.78:0/3476013088 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7abc003fb0 con 0x5622f5df0f60 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aceb94740 10 client.15531 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aceb94740 10 client.15531 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.681+0000 7f7aceb94740 20 client.15531 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.682+0000 7f7aceb94740 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5e32390 0x5622f62564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.682+0000 7f7aceb94740 4 client.15531 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.682+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622f6140800 con 0x5622f5e32390 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.682+0000 7f7ac527b640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5e32390 0x5622f62564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.682+0000 7f7ac527b640 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5e32390 0x5622f62564f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1e3539c1f82b8ef server_cookie=9958bb8b3ba101aa in_seq=0 out_seq=0 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.682+0000 7f7aa97fa640 10 client.15531 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aa97fa640 1 -- 172.21.15.78:0/3476013088 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622f6140800 con 0x5622f5e32390 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aa97fa640 10 client.15531 handle_command_reply: tid=0 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 client.15531 shutdown 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5df0f60 msgr2=0x5622f61e1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5df0f60 0x5622f61e1e40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0a3a40 tx=0x7f7abc0b00a0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 shutdown_connections 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5622f5e116b0 0x5622f62526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5622f5e32390 0x5622f62564f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f5e094d0 0x5622f61e2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 --2- 172.21.15.78:0/3476013088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622f5df0f60 0x5622f61e1e40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aa97fa640 0 client.15531 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 >> 172.21.15.78:0/3476013088 conn(0x5622f5e264e0 msgr2=0x5622f61de200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 shutdown_connections 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 1 -- 172.21.15.78:0/3476013088 wait complete. 2024-04-25T13:59:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7aceb94740 20 client.15531 trim_cache size 0 max 16384 2024-04-25T13:59:48.691 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T13:59:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7aac0afaf0 msgr2=0x7f7aac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7aac0afaf0 0x7f7aac0b2000 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80094e0 tx=0x7f7ab80095c0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0101750 msgr2=0x7f7ac019a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0101750 0x7f7ac019a150 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e9f0 tx=0x7f7abc00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 shutdown_connections 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ac01050a0 0x7f7ac01a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.683+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ac0102250 0x7f7ac019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.684+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7aac0afaf0 0x7f7aac0b2000 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.684+0000 7f7ac7ce0640 1 --2- 172.21.15.78:0/4057330510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ac0101750 0x7f7ac019a150 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.684+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 >> 172.21.15.78:0/4057330510 conn(0x7f7ac00fd350 msgr2=0x7f7ac0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.684+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 shutdown_connections 2024-04-25T13:59:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.684+0000 7f7ac7ce0640 1 -- 172.21.15.78:0/4057330510 wait complete. 2024-04-25T13:59:48.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T13:59:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:48 smithi112 ceph-mon[32617]: pgmap v357: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 -- 172.21.15.78:0/890968153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 msgr2=0x7efc0c10bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 --2- 172.21.15.78:0/890968153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c10bc90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efc04009ca0 tx=0x7efc0402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 -- 172.21.15.78:0/890968153 shutdown_connections 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 --2- 172.21.15.78:0/890968153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c10bc90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 --2- 172.21.15.78:0/890968153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc0c102360 0x7efc0c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 --2- 172.21.15.78:0/890968153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc0c101e60 0x7efc0c106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 -- 172.21.15.78:0/890968153 >> 172.21.15.78:0/890968153 conn(0x7efc0c0fd350 msgr2=0x7efc0c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 -- 172.21.15.78:0/890968153 shutdown_connections 2024-04-25T13:59:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.804+0000 7efc10f6c640 1 -- 172.21.15.78:0/890968153 wait complete. 2024-04-25T13:59:48.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 Processor -- start 2024-04-25T13:59:48.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 -- start start 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc0c101e60 0x7efc0c104330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc0c102360 0x7efc0c1029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c10f650 con 0x7efc0c109780 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c10f4d0 con 0x7efc0c102360 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc10f6c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c10f7d0 con 0x7efc0c101e60 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42536/0 (socket says 172.21.15.78:42536) 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 -- 172.21.15.78:0/3773183398 learned_addr learned my addr 172.21.15.78:0/3773183398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0a59c640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc0c101e60 0x7efc0c104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc0c102360 0x7efc0c1029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 -- 172.21.15.78:0/3773183398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc0c101e60 msgr2=0x7efc0c104330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc0c101e60 0x7efc0c104330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 -- 172.21.15.78:0/3773183398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc0c102360 msgr2=0x7efc0c1029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc0c102360 0x7efc0c1029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.805+0000 7efc0ad9d640 1 -- 172.21.15.78:0/3773183398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c1035b0 con 0x7efc0c109780 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.806+0000 7efc0ad9d640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c102f10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efc040097b0 tx=0x7efc04003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.806+0000 7efc037fe640 1 -- 172.21.15.78:0/3773183398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04004240 con 0x7efc0c109780 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.806+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c1a4280 con 0x7efc0c109780 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.806+0000 7efc037fe640 1 -- 172.21.15.78:0/3773183398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc040317c0 con 0x7efc0c109780 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.806+0000 7efc037fe640 1 -- 172.21.15.78:0/3773183398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0402fde0 con 0x7efc0c109780 2024-04-25T13:59:48.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.806+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c1a46b0 con 0x7efc0c109780 2024-04-25T13:59:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.808+0000 7efc037fe640 1 -- 172.21.15.78:0/3773183398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efc040043e0 con 0x7efc0c109780 2024-04-25T13:59:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.808+0000 7efc17e16740 1 Processor -- start 2024-04-25T13:59:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc17e16740 1 -- start start 2024-04-25T13:59:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc17e16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 0x55698aab0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc17e16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6e54d0 0x55698aab0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc17e16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6ed6b0 0x55698aab1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc17e16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55698aa14d80 con 0x55698a6ed6b0 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc17e16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55698a9cea20 con 0x55698a6e54d0 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc17e16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55698aa10d40 con 0x55698a70e390 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6ed6b0 0x55698aab1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6ed6b0 0x55698aab1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42538/0 (socket says 172.21.15.78:42538) 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 -- 172.21.15.78:0/1914755008 learned_addr learned my addr 172.21.15.78:0/1914755008 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 0x55698aab0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc0a59c640 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6e54d0 0x55698aab0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc037fe640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efbf80afaf0 0x7efbf80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc0ad9d640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efbf80afaf0 0x7efbf80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 -- 172.21.15.78:0/1914755008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 msgr2=0x55698aab0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 0x55698aab0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 -- 172.21.15.78:0/1914755008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6e54d0 msgr2=0x55698aab0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6e54d0 0x55698aab0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc09d9b640 1 -- 172.21.15.78:0/1914755008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55698a9d9c30 con 0x55698a6ed6b0 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.809+0000 7efc037fe640 1 -- 172.21.15.78:0/3773183398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efc04100770 con 0x7efc0c109780 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.810+0000 7efc09d9b640 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6ed6b0 0x55698aab1a20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efbfc016500 tx=0x7efbfc00a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152040589c403a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.810+0000 7efc00ff9640 1 -- 172.21.15.78:0/1914755008 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc018430 con 0x55698a6ed6b0 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.810+0000 7efc00ff9640 1 -- 172.21.15.78:0/1914755008 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc018a80 con 0x55698a6ed6b0 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.810+0000 7efc00ff9640 1 -- 172.21.15.78:0/1914755008 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc017910 con 0x55698a6ed6b0 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- 172.21.15.78:0/1914755008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6ed6b0 msgr2=0x55698aab1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc0ad9d640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efbf80afaf0 0x7efbf80b2000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc040c8940 tx=0x7efc04096460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6ed6b0 0x55698aab1a20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efbfc016500 tx=0x7efbfc00a2b0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- 172.21.15.78:0/1914755008 shutdown_connections 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6ed6b0 0x55698aab1a20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6e54d0 0x55698aab0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 --2- 172.21.15.78:0/1914755008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 0x55698aab0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- 172.21.15.78:0/1914755008 >> 172.21.15.78:0/1914755008 conn(0x55698a7024e0 msgr2=0x55698aaac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- 172.21.15.78:0/1914755008 shutdown_connections 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- 172.21.15.78:0/1914755008 wait complete. 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- start start 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 Processor -- start 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aaba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 0x55698aac1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55698aab2470 con 0x55698a6e54d0 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55698a99bdc0 con 0x55698a6ed6b0 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55698aab25f0 con 0x55698a70e390 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc17e16740 10 client.0 _ll_register_callbacks cb 0x55698a9b7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.811+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaba480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42540/0 (socket says 172.21.15.78:42540) 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc09d9b640 1 -- 172.21.15.78:0/356308156 learned_addr learned my addr 172.21.15.78:0/356308156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc0a59c640 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aaba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc09d9b640 1 -- 172.21.15.78:0/356308156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 msgr2=0x55698aac1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc09d9b640 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 0x55698aac1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc09d9b640 1 -- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 msgr2=0x55698aaba9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc09d9b640 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aaba9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc09d9b640 1 -- 172.21.15.78:0/356308156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55698a7263f0 con 0x55698a6e54d0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc09d9b640 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaba480 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efbfc01a040 tx=0x7efbfc016e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab43f38b47530fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efbf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efbf1ffb640 1 -- 172.21.15.78:0/356308156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc025020 con 0x55698a6e54d0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efbf1ffb640 1 -- 172.21.15.78:0/356308156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efbfc018840 con 0x55698a6e54d0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc018a80 con 0x55698a6e54d0 2024-04-25T13:59:48.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efbf1ffb640 1 -- 172.21.15.78:0/356308156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efbfc030050 con 0x55698a6e54d0 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc17e16740 10 client.15543 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.812+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55698a7f07d0 con 0x55698a6e54d0 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efbf1ffb640 1 -- 172.21.15.78:0/356308156 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc018a80 con 0x55698a6e54d0 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efbf1ffb640 1 -- 172.21.15.78:0/356308156 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7efbfc012050 con 0x55698a6e54d0 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efc17e16740 10 client.15543 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efc17e16740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efc17e16740 20 client.15543 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efc17e16740 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6ccf60 0x55698aac7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efc17e16740 4 client.15543 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55698aa22990 con 0x55698a6ccf60 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.813+0000 7efc0a59c640 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6ccf60 0x55698aac7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efbf1ffb640 10 client.15543 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc0a59c640 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6ccf60 0x55698aac7e80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6fc5b6547bac8da server_cookie=6b6f82c69e291753 in_seq=0 out_seq=0 2024-04-25T13:59:48.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efbf1ffb640 1 -- 172.21.15.78:0/356308156 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55698aa22990 con 0x55698a6ccf60 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efbf1ffb640 10 client.15543 handle_command_reply: tid=0 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 client.15543 shutdown 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 msgr2=0x55698aaba480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaba480 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efbfc01a040 tx=0x7efbfc016e80 comp rx=0 tx=0).stop 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 shutdown_connections 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6ccf60 0x55698aac7e80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a70e390 0x55698aac1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aaba9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efbf1ffb640 0 client.15543 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 --2- 172.21.15.78:0/356308156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaba480 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.814+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 >> 172.21.15.78:0/356308156 conn(0x55698a7024e0 msgr2=0x55698aaaef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.815+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 shutdown_connections 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.815+0000 7efc17e16740 1 -- 172.21.15.78:0/356308156 wait complete. 2024-04-25T13:59:48.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.815+0000 7efc17e16740 20 client.15543 trim_cache size 0 max 16384 2024-04-25T13:59:48.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 Processor -- start 2024-04-25T13:59:48.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 -- start start 2024-04-25T13:59:48.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaaba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6ed6b0 0x55698aaabf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a70e390 0x55698aaba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1afca0 con 0x55698a6e54d0 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1ad570 con 0x55698a70e390 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc17e16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1ab3c0 con 0x55698a6ed6b0 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaaba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaaba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42556/0 (socket says 172.21.15.78:42556) 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.828+0000 7efc09d9b640 1 -- 172.21.15.78:0/3229573861 learned_addr learned my addr 172.21.15.78:0/3229573861 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc0a59c640 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6ed6b0 0x55698aaabf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc09d9b640 1 -- 172.21.15.78:0/3229573861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6ed6b0 msgr2=0x55698aaabf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6ed6b0 0x55698aaabf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc09d9b640 1 -- 172.21.15.78:0/3229573861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a70e390 msgr2=0x55698aaba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a70e390 0x55698aaba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc09d9b640 1 -- 172.21.15.78:0/3229573861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55698aab89d0 con 0x55698a6e54d0 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc0a59c640 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6ed6b0 0x55698aaabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaaba50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00a980 tx=0x7efbfc00a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adf5becd09b9932 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efbf3fff640 1 -- 172.21.15.78:0/3229573861 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc025070 con 0x55698a6e54d0 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efbf3fff640 1 -- 172.21.15.78:0/3229573861 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc021660 con 0x55698a6e54d0 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.829+0000 7efbf3fff640 1 -- 172.21.15.78:0/3229573861 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc017810 con 0x55698a6e54d0 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- 172.21.15.78:0/3229573861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 msgr2=0x55698aaaba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaaba50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00a980 tx=0x7efbfc00a290 comp rx=0 tx=0).stop 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- 172.21.15.78:0/3229573861 shutdown_connections 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a70e390 0x55698aaba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6ed6b0 0x55698aaabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 --2- 172.21.15.78:0/3229573861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a6e54d0 0x55698aaaba50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- 172.21.15.78:0/3229573861 >> 172.21.15.78:0/3229573861 conn(0x55698a7024e0 msgr2=0x55698aab5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- 172.21.15.78:0/3229573861 shutdown_connections 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- 172.21.15.78:0/3229573861 wait complete. 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- start start 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 Processor -- start 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6e54d0 0x55698aabdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aabe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a70e390 0x55698ab2e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1aba30 con 0x55698a70e390 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1acea0 con 0x55698a6ed6b0 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1ac100 con 0x55698a6e54d0 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc17e16740 10 client.0 _ll_register_callbacks cb 0x7efbf400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:48.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.830+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a70e390 0x55698ab2e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a70e390 0x55698ab2e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42570/0 (socket says 172.21.15.78:42570) 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 -- 172.21.15.78:0/3182364421 learned_addr learned my addr 172.21.15.78:0/3182364421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6e54d0 0x55698aabdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc0a59c640 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aabe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 -- 172.21.15.78:0/3182364421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6e54d0 msgr2=0x55698aabdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6e54d0 0x55698aabdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 -- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 msgr2=0x55698aabe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aabe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 -- 172.21.15.78:0/3182364421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55698ab2e600 con 0x55698a70e390 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc09d9b640 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a70e390 0x55698ab2e0c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0215f0 tx=0x7efbfc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893daccf3b443927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:48.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efbf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efbf1ffb640 1 -- 172.21.15.78:0/3182364421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc025030 con 0x55698a70e390 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efbf1ffb640 1 -- 172.21.15.78:0/3182364421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efbfc017290 con 0x55698a70e390 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc021660 con 0x55698a70e390 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efbf1ffb640 1 -- 172.21.15.78:0/3182364421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efbfc018940 con 0x55698a70e390 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc17e16740 10 client.15549 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55698ab2f6f0 con 0x55698a70e390 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.831+0000 7efbf1ffb640 1 -- 172.21.15.78:0/3182364421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc021660 con 0x55698a70e390 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efbf1ffb640 1 -- 172.21.15.78:0/3182364421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7efbfc049030 con 0x55698a70e390 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc17e16740 10 client.15549 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc17e16740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc17e16740 20 client.15549 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:48.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc17e16740 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6a23a0 0x55698ab31fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc17e16740 4 client.15549 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55698aa1c800 con 0x55698a6a23a0 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc0a59c640 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6a23a0 0x55698ab31fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efbf1ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc0a59c640 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6a23a0 0x55698ab31fc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71d3dee7d250e411 server_cookie=ac41b4f3ea99e86a in_seq=0 out_seq=0 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efbf1ffb640 1 -- 172.21.15.78:0/3182364421 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55698aa1c800 con 0x55698a6a23a0 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efbf1ffb640 10 client.15549 handle_command_reply: tid=0 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.832+0000 7efc17e16740 1 client.15549 shutdown 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a70e390 msgr2=0x55698ab2e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a70e390 0x55698ab2e0c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0215f0 tx=0x7efbfc01a040 comp rx=0 tx=0).stop 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 shutdown_connections 2024-04-25T13:59:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55698a70e390 0x55698ab2e0c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55698a6ed6b0 0x55698aabe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55698a6a23a0 0x55698ab31fc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 --2- 172.21.15.78:0/3182364421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55698a6e54d0 0x55698aabdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efbf1ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 >> 172.21.15.78:0/3182364421 conn(0x55698a7024e0 msgr2=0x55698aaba050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 shutdown_connections 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 1 -- 172.21.15.78:0/3182364421 wait complete. 2024-04-25T13:59:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc17e16740 20 client.15549 trim_cache size 0 max 16384 2024-04-25T13:59:48.842 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T13:59:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efbf80afaf0 msgr2=0x7efbf80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efbf80afaf0 0x7efbf80b2000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc040c8940 tx=0x7efc04096460 comp rx=0 tx=0).stop 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 msgr2=0x7efc0c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c102f10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efc040097b0 tx=0x7efc04003c20 comp rx=0 tx=0).stop 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 shutdown_connections 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efbf80afaf0 0x7efbf80b2000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc0c109780 0x7efc0c102f10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc0c102360 0x7efc0c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 --2- 172.21.15.78:0/3773183398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc0c101e60 0x7efc0c104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 >> 172.21.15.78:0/3773183398 conn(0x7efc0c0fd350 msgr2=0x7efc0c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 shutdown_connections 2024-04-25T13:59:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:48.833+0000 7efc10f6c640 1 -- 172.21.15.78:0/3773183398 wait complete. 2024-04-25T13:59:49.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:49 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1470249937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:49 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1470249937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:49.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:49 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1470249937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:59:49.845 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T13:59:49.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:59:49.846 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T13:59:49.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 -- 172.21.15.78:0/1671293056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb01045d0 msgr2=0x7f8cb01069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1671293056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb01045d0 0x7f8cb01069f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009900 tx=0x7f8ca402a410 comp rx=0 tx=0).stop 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 -- 172.21.15.78:0/1671293056 shutdown_connections 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1671293056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb0109890 0x7f8cb010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1671293056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb0106f30 0x7f8cb0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1671293056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb01045d0 0x7f8cb01069f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 -- 172.21.15.78:0/1671293056 >> 172.21.15.78:0/1671293056 conn(0x7f8cb00fdbd0 msgr2=0x7f8cb0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 -- 172.21.15.78:0/1671293056 shutdown_connections 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.970+0000 7f8cb7092640 1 -- 172.21.15.78:0/1671293056 wait complete. 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 Processor -- start 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 -- start start 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb01045d0 0x7f8cb019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb0106f30 0x7f8cb019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb0109890 0x7f8cb01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb010f5e0 con 0x7f8cb0106f30 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb010f460 con 0x7f8cb0109890 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb7092640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb010f760 con 0x7f8cb01045d0 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb0106f30 0x7f8cb019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb0106f30 0x7f8cb019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42598/0 (socket says 172.21.15.78:42598) 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 -- 172.21.15.78:0/1132947483 learned_addr learned my addr 172.21.15.78:0/1132947483 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb01045d0 0x7f8cb019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 -- 172.21.15.78:0/1132947483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb01045d0 msgr2=0x7f8cb019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb01045d0 0x7f8cb019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 -- 172.21.15.78:0/1132947483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb0109890 msgr2=0x7f8cb01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb562f640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb0109890 0x7f8cb01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb0109890 0x7f8cb01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8caffff640 1 -- 172.21.15.78:0/1132947483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb01a20d0 con 0x7f8cb0106f30 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.971+0000 7f8cb562f640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb0109890 0x7f8cb01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.972+0000 7f8caffff640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb0106f30 0x7f8cb019aad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00d770 tx=0x7f8c9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.972+0000 7f8cadffb640 1 -- 172.21.15.78:0/1132947483 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c017070 con 0x7f8cb0106f30 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.972+0000 7f8cadffb640 1 -- 172.21.15.78:0/1132947483 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0054d0 con 0x7f8cb0106f30 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.972+0000 7f8cadffb640 1 -- 172.21.15.78:0/1132947483 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c004c30 con 0x7f8cb0106f30 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.972+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb01a23c0 con 0x7f8cb0106f30 2024-04-25T13:59:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.972+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb01a28c0 con 0x7f8cb0106f30 2024-04-25T13:59:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 Processor -- start 2024-04-25T13:59:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 -- start start 2024-04-25T13:59:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e6434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cadffb640 1 -- 172.21.15.78:0/1132947483 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8c9c004e10 con 0x7f8cb0106f30 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e643a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56349e5a8b00 con 0x56349e281730 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56349e544920 con 0x56349e2a2410 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349e5a4ac0 con 0x56349e279550 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42612/0 (socket says 172.21.15.78:42612) 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8caffff640 1 -- 172.21.15.78:0/3307486893 learned_addr learned my addr 172.21.15.78:0/3307486893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.975+0000 7f8cb562f640 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e6434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e643a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8caffff640 1 -- 172.21.15.78:0/3307486893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 msgr2=0x56349e643a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8caffff640 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e643a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8caffff640 1 -- 172.21.15.78:0/3307486893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 msgr2=0x56349e6434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8caffff640 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e6434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8caffff640 1 -- 172.21.15.78:0/3307486893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349e3827e0 con 0x56349e281730 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8caffff640 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64a920 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0e4c10 tx=0x7f8c9c0e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5600d3e0c8aac0db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8cadffb640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ca00afaf0 0x7f8ca00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8c9affd640 1 -- 172.21.15.78:0/3307486893 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0ed070 con 0x56349e281730 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8c9affd640 1 -- 172.21.15.78:0/3307486893 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0b0b50 con 0x56349e281730 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8c9affd640 1 -- 172.21.15.78:0/3307486893 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0a63e0 con 0x56349e281730 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8cb562f640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ca00afaf0 0x7f8ca00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.976+0000 7f8cadffb640 1 -- 172.21.15.78:0/1132947483 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8c9c0de7a0 con 0x7f8cb0106f30 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 -- 172.21.15.78:0/3307486893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 msgr2=0x56349e64a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cb562f640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ca00afaf0 0x7f8ca00b2000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56349e644670 tx=0x7f8ca8009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64a920 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0e4c10 tx=0x7f8c9c0e6690 comp rx=0 tx=0).stop 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 -- 172.21.15.78:0/3307486893 shutdown_connections 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64a920 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e643a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/3307486893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e6434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 -- 172.21.15.78:0/3307486893 >> 172.21.15.78:0/3307486893 conn(0x56349e296560 msgr2=0x56349e63ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 -- 172.21.15.78:0/3307486893 shutdown_connections 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 -- 172.21.15.78:0/3307486893 wait complete. 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.977+0000 7f8cbe01d740 1 -- start start 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 1 Processor -- start 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e64d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e654650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56349e64b370 con 0x56349e281730 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56349e5425d0 con 0x56349e2a2410 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349e64b4f0 con 0x56349e279550 2024-04-25T13:59:49.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cbe01d740 10 client.0 _ll_register_callbacks cb 0x56349e530080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42614/0 (socket says 172.21.15.78:42614) 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 -- 172.21.15.78:0/278004311 learned_addr learned my addr 172.21.15.78:0/278004311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e64d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cb562f640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e654650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 -- 172.21.15.78:0/278004311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 msgr2=0x56349e64d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e64d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 -- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 msgr2=0x56349e654650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e654650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 -- 172.21.15.78:0/278004311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349e3e2f70 con 0x56349e281730 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8cb562f640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e654650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:59:49.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8caffff640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64d6f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0a6970 tx=0x7f8c9c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ba0bb727c6c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.978+0000 7f8c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8c8ffff640 1 -- 172.21.15.78:0/278004311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c06c150 con 0x56349e281730 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8c8ffff640 1 -- 172.21.15.78:0/278004311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8c9c06ae00 con 0x56349e281730 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c0b0b50 con 0x56349e281730 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8c8ffff640 1 -- 172.21.15.78:0/278004311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c020020 con 0x56349e281730 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 10 client.15561 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56349e27a170 con 0x56349e281730 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8c8ffff640 1 -- 172.21.15.78:0/278004311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0b0b50 con 0x56349e281730 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8c8ffff640 1 -- 172.21.15.78:0/278004311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8c9c0a2f00 con 0x56349e281730 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 10 client.15561 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 20 client.15561 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e2611f0 0x56349e65a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 4 client.15561 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.979+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56349e5b6710 con 0x56349e2611f0 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.980+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e2611f0 0x56349e65a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.980+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e2611f0 0x56349e65a710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192fbaf5b4784899 server_cookie=c8b97c7970538601 in_seq=0 out_seq=0 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.980+0000 7f8c8ffff640 10 client.15561 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8c8ffff640 1 -- 172.21.15.78:0/278004311 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56349e5b6710 con 0x56349e2611f0 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8c8ffff640 10 client.15561 handle_command_reply: tid=0 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 client.15561 shutdown 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 msgr2=0x56349e64d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64d6f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0a6970 tx=0x7f8c9c009e90 comp rx=0 tx=0).stop 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 shutdown_connections 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e2a2410 0x56349e654650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e281730 0x56349e64d6f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e2611f0 0x56349e65a710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/278004311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e64d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8c8ffff640 0 client.15561 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 >> 172.21.15.78:0/278004311 conn(0x56349e296560 msgr2=0x56349e63def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 shutdown_connections 2024-04-25T13:59:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 1 -- 172.21.15.78:0/278004311 wait complete. 2024-04-25T13:59:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.981+0000 7f8cbe01d740 20 client.15561 trim_cache size 0 max 16384 2024-04-25T13:59:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 Processor -- start 2024-04-25T13:59:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 -- start start 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e65ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e65f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e65f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01adb70 con 0x56349e2a2410 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01ab440 con 0x56349e281730 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01a9290 con 0x56349e279550 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e65f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e65f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42620/0 (socket says 172.21.15.78:42620) 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 -- 172.21.15.78:0/1016946423 learned_addr learned my addr 172.21.15.78:0/1016946423 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 -- 172.21.15.78:0/1016946423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 msgr2=0x56349e65ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e65ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e65ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 -- 172.21.15.78:0/1016946423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 msgr2=0x56349e65f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8caffff640 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e65f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e65f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:49.999+0000 7f8cb562f640 1 -- 172.21.15.78:0/1016946423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349e648060 con 0x56349e2a2410 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cb562f640 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e65f7b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8010730 tx=0x7f8ca8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b246f55558106f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8c9a7fc640 1 -- 172.21.15.78:0/1016946423 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800f800 con 0x56349e2a2410 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8c9a7fc640 1 -- 172.21.15.78:0/1016946423 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca8023070 con 0x56349e2a2410 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8c9a7fc640 1 -- 172.21.15.78:0/1016946423 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8028e30 con 0x56349e2a2410 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 -- 172.21.15.78:0/1016946423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 msgr2=0x56349e65f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e65f7b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8010730 tx=0x7f8ca8015000 comp rx=0 tx=0).stop 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 -- 172.21.15.78:0/1016946423 shutdown_connections 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e65f7b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e65f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/1016946423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e65ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 -- 172.21.15.78:0/1016946423 >> 172.21.15.78:0/1016946423 conn(0x56349e296560 msgr2=0x56349e6466b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 -- 172.21.15.78:0/1016946423 shutdown_connections 2024-04-25T13:59:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.000+0000 7f8cbe01d740 1 -- 172.21.15.78:0/1016946423 wait complete. 2024-04-25T13:59:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 -- start start 2024-04-25T13:59:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 Processor -- start 2024-04-25T13:59:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e651a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e651fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e652e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01a9900 con 0x56349e2a2410 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01aad70 con 0x56349e281730 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01a9fd0 con 0x56349e279550 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cbe01d740 10 client.0 _ll_register_callbacks cb 0x7f8c9c0e4a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cb562f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e652e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cb562f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e652e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42628/0 (socket says 172.21.15.78:42628) 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cb562f640 1 -- 172.21.15.78:0/632096795 learned_addr learned my addr 172.21.15.78:0/632096795 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.001+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e651a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8caffff640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e651fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cb562f640 1 -- 172.21.15.78:0/632096795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 msgr2=0x56349e651a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cb562f640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e651a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cb562f640 1 -- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 msgr2=0x56349e651fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cb562f640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e651fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cb562f640 1 -- 172.21.15.78:0/632096795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349e653430 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e651a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cb562f640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e652e70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8012fd0 tx=0x7f8ca80286a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57ff69dd23298fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8c8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8c8ffff640 1 -- 172.21.15.78:0/632096795 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8038060 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8c8ffff640 1 -- 172.21.15.78:0/632096795 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8ca8028b70 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca8023070 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8c8ffff640 1 -- 172.21.15.78:0/632096795 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8ca8028d70 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cbe01d740 10 client.15567 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56349e6c2e10 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8c8ffff640 1 -- 172.21.15.78:0/632096795 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca8023070 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8c8ffff640 1 -- 172.21.15.78:0/632096795 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8ca802a230 con 0x56349e2a2410 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cbe01d740 10 client.15567 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.002+0000 7f8cbe01d740 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:50.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.003+0000 7f8cbe01d740 20 client.15567 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.003+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e236630 0x56349e6c5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.003+0000 7f8cbe01d740 4 client.15567 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.003+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56349e5b0580 con 0x56349e236630 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.003+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e236630 0x56349e6c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.003+0000 7f8c8ffff640 10 client.15567 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.003+0000 7f8cb4e2e640 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e236630 0x56349e6c5780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8fa4fb0ef265d47 server_cookie=4af82ad5a9a8f5b6 in_seq=0 out_seq=0 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.004+0000 7f8c8ffff640 1 -- 172.21.15.78:0/632096795 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56349e5b0580 con 0x56349e236630 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.004+0000 7f8c8ffff640 10 client.15567 handle_command_reply: tid=0 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.004+0000 7f8cbe01d740 1 client.15567 shutdown 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 msgr2=0x56349e652e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e652e70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8012fd0 tx=0x7f8ca80286a0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 shutdown_connections 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56349e2a2410 0x56349e652e70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56349e281730 0x56349e651fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56349e236630 0x56349e6c5780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 --2- 172.21.15.78:0/632096795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349e279550 0x56349e651a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 >> 172.21.15.78:0/632096795 conn(0x56349e296560 msgr2=0x56349e641cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8c8ffff640 0 client.15567 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 shutdown_connections 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 1 -- 172.21.15.78:0/632096795 wait complete. 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cbe01d740 20 client.15567 trim_cache size 0 max 16384 2024-04-25T13:59:50.011 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:50.012 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T13:59:50.012 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c4c0dd68-4c31-45d6-aff1-a0ca900d1d20", 2024-04-25T13:59:50.012 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T13:59:50.012 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:50.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ca00afaf0 msgr2=0x7f8ca00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ca00afaf0 0x7f8ca00b2000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56349e644670 tx=0x7f8ca8009700 comp rx=0 tx=0).stop 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb0106f30 msgr2=0x7f8cb019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb0106f30 0x7f8cb019aad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00d770 tx=0x7f8c9c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 shutdown_connections 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cb0109890 0x7f8cb01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8ca00afaf0 0x7f8ca00b2000 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb0106f30 0x7f8cb019aad0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 --2- 172.21.15.78:0/1132947483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb01045d0 0x7f8cb019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.005+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 >> 172.21.15.78:0/1132947483 conn(0x7f8cb00fdbd0 msgr2=0x7f8cb0104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.006+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 shutdown_connections 2024-04-25T13:59:50.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.006+0000 7f8cb7092640 1 -- 172.21.15.78:0/1132947483 wait complete. 2024-04-25T13:59:50.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T13:59:50.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T13:59:50.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- 172.21.15.78:0/2190726164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 msgr2=0x7f1de0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/2190726164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de0104040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009ca0 tx=0x7f1dd002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- 172.21.15.78:0/2190726164 shutdown_connections 2024-04-25T13:59:50.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/2190726164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1de0108930 0x7f1de010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/2190726164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de0104040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/2190726164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de01029f0 0x7f1de01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- 172.21.15.78:0/2190726164 >> 172.21.15.78:0/2190726164 conn(0x7f1de00fdbe0 msgr2=0x7f1de0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- 172.21.15.78:0/2190726164 shutdown_connections 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- 172.21.15.78:0/2190726164 wait complete. 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 Processor -- start 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- start start 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de01029f0 0x7f1de006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1de0108930 0x7f1de01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010e510 con 0x7f1de0103c30 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010e390 con 0x7f1de0108930 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.127+0000 7f1de6f0d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1de010e690 con 0x7f1de01029f0 2024-04-25T13:59:50.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42634/0 (socket says 172.21.15.78:42634) 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 -- 172.21.15.78:0/1553689905 learned_addr learned my addr 172.21.15.78:0/1553689905 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1de54aa640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1de0108930 0x7f1de01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de01029f0 0x7f1de006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 -- 172.21.15.78:0/1553689905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de01029f0 msgr2=0x7f1de006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de01029f0 0x7f1de006bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 -- 172.21.15.78:0/1553689905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1de0108930 msgr2=0x7f1de01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1de0108930 0x7f1de01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 -- 172.21.15.78:0/1553689905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de01a1d60 con 0x7f1de0103c30 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1de54aa640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1de0108930 0x7f1de01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1ddffff640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de019f080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1dd002ade0 tx=0x7f1dd00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1dddffb640 1 -- 172.21.15.78:0/1553689905 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0003c70 con 0x7f1de0103c30 2024-04-25T13:59:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1dddffb640 1 -- 172.21.15.78:0/1553689905 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd0003e10 con 0x7f1de0103c30 2024-04-25T13:59:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1dddffb640 1 -- 172.21.15.78:0/1553689905 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd002fca0 con 0x7f1de0103c30 2024-04-25T13:59:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de01a1ff0 con 0x7f1de0103c30 2024-04-25T13:59:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.128+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de01a24b0 con 0x7f1de0103c30 2024-04-25T13:59:50.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 Processor -- start 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 -- start start 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1dddffb640 1 -- 172.21.15.78:0/1553689905 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1dd0004040 con 0x7f1de0103c30 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e6bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb67dcfd80 con 0x55fb67ac9390 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb67d898b0 con 0x55fb67aa04d0 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1deddc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb67dcbd40 con 0x55fb67aa86b0 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1de54aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1de54aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e6aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42636/0 (socket says 172.21.15.78:42636) 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1de54aa640 1 -- 172.21.15.78:0/3189990369 learned_addr learned my addr 172.21.15.78:0/3189990369 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.131+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1ddffff640 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e6bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1de54aa640 1 -- 172.21.15.78:0/3189990369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 msgr2=0x55fb67e6bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1de54aa640 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e6bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1de54aa640 1 -- 172.21.15.78:0/3189990369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 msgr2=0x55fb67e6afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1de54aa640 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e6afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1de54aa640 1 -- 172.21.15.78:0/3189990369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb67d5aa40 con 0x55fb67ac9390 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1dddffb640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1dcc0afb70 0x7f1dcc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1de54aa640 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e6aa60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800ac90 tx=0x7f1dc800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc974d6470599b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1dcc0afb70 0x7f1dcc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1dddffb640 1 -- 172.21.15.78:0/1553689905 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1dd01001d0 con 0x7f1de0103c30 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1dd6ffd640 1 -- 172.21.15.78:0/3189990369 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8005220 con 0x55fb67ac9390 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1dd6ffd640 1 -- 172.21.15.78:0/3189990369 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc8004c50 con 0x55fb67ac9390 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.132+0000 7f1dd6ffd640 1 -- 172.21.15.78:0/3189990369 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80107e0 con 0x55fb67ac9390 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 -- 172.21.15.78:0/3189990369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 msgr2=0x55fb67e6aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1dcc0afb70 0x7f1dcc0b2080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55fb67e6a780 tx=0x7f1dd800b4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e6aa60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800ac90 tx=0x7f1dc800e720 comp rx=0 tx=0).stop 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 -- 172.21.15.78:0/3189990369 shutdown_connections 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e6bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3189990369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e6aa60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 -- 172.21.15.78:0/3189990369 >> 172.21.15.78:0/3189990369 conn(0x55fb67abd4e0 msgr2=0x55fb67e66270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 -- 172.21.15.78:0/3189990369 shutdown_connections 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 -- 172.21.15.78:0/3189990369 wait complete. 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.133+0000 7f1deddc4740 1 -- start start 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 1 Processor -- start 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e7be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb67e6c780 con 0x55fb67ac9390 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fb67d56d10 con 0x55fb67aa04d0 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb67e6c900 con 0x55fb67aa86b0 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1deddc4740 10 client.0 _ll_register_callbacks cb 0x55fb67d83210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e7be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e7be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42638/0 (socket says 172.21.15.78:42638) 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 -- 172.21.15.78:0/3221298638 learned_addr learned my addr 172.21.15.78:0/3221298638 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1ddffff640 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 -- 172.21.15.78:0/3221298638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 msgr2=0x55fb67e74ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 -- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 msgr2=0x55fb67e749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.134+0000 7f1de54aa640 1 -- 172.21.15.78:0/3221298638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb67bab7d0 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1de54aa640 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e7be40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800aa00 tx=0x7f1dc8004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dda8fe3f8f7a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1dbffff640 1 -- 172.21.15.78:0/3221298638 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8029060 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1dbffff640 1 -- 172.21.15.78:0/3221298638 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1dc80042d0 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc8004c50 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1dbffff640 1 -- 172.21.15.78:0/3221298638 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1dc80044d0 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 10 client.15579 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fb67ae13f0 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1dbffff640 1 -- 172.21.15.78:0/3221298638 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc8004c50 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1dbffff640 1 -- 172.21.15.78:0/3221298638 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1dc8003c70 con 0x55fb67ac9390 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 10 client.15579 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:50.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 20 client.15579 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a87f60 0x55fb67e82010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 4 client.15579 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.135+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb67ddd990 con 0x55fb67a87f60 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.136+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a87f60 0x55fb67e82010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.136+0000 7f1dbffff640 10 client.15579 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.136+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a87f60 0x55fb67e82010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f806387460b3bfb9 server_cookie=b1d74ed215950250 in_seq=0 out_seq=0 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1dbffff640 1 -- 172.21.15.78:0/3221298638 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb67ddd990 con 0x55fb67a87f60 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1dbffff640 10 client.15579 handle_command_reply: tid=0 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 client.15579 shutdown 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 msgr2=0x55fb67e7be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e7be40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800aa00 tx=0x7f1dc8004b60 comp rx=0 tx=0).stop 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 shutdown_connections 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67e7be40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a87f60 0x55fb67e82010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 --2- 172.21.15.78:0/3221298638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1dbffff640 0 client.15579 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:50.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 >> 172.21.15.78:0/3221298638 conn(0x55fb67abd4e0 msgr2=0x55fb67e653f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 shutdown_connections 2024-04-25T13:59:50.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 1 -- 172.21.15.78:0/3221298638 wait complete. 2024-04-25T13:59:50.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.137+0000 7f1deddc4740 20 client.15579 trim_cache size 0 max 16384 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 Processor -- start 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 -- start start 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa04d0 0x55fb67e749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67ac9390 0x55fb67e866d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01adb20 con 0x55fb67aa86b0 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01ab3f0 con 0x55fb67ac9390 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1deddc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01a9240 con 0x55fb67aa04d0 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42652/0 (socket says 172.21.15.78:42652) 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 -- 172.21.15.78:0/4215885042 learned_addr learned my addr 172.21.15.78:0/4215885042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:50.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 -- 172.21.15.78:0/4215885042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa04d0 msgr2=0x55fb67e749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 --2- 172.21.15.78:0/4215885042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa04d0 0x55fb67e749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 -- 172.21.15.78:0/4215885042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67ac9390 msgr2=0x55fb67e866d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 --2- 172.21.15.78:0/4215885042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67ac9390 0x55fb67e866d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 -- 172.21.15.78:0/4215885042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb67aa10f0 con 0x55fb67aa86b0 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1ddffff640 1 --2- 172.21.15.78:0/4215885042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00c4720 tx=0x7f1dd00c4560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dee41e9f2adda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1dd67fc640 1 -- 172.21.15.78:0/4215885042 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd00c7e10 con 0x55fb67aa86b0 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1dd67fc640 1 -- 172.21.15.78:0/4215885042 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd00ff030 con 0x55fb67aa86b0 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.155+0000 7f1dd67fc640 1 -- 172.21.15.78:0/4215885042 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd002fe40 con 0x55fb67aa86b0 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 -- 172.21.15.78:0/4215885042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67aa86b0 msgr2=0x55fb67e74ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 --2- 172.21.15.78:0/4215885042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00c4720 tx=0x7f1dd00c4560 comp rx=0 tx=0).stop 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 -- 172.21.15.78:0/4215885042 shutdown_connections 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 --2- 172.21.15.78:0/4215885042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67ac9390 0x55fb67e866d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 --2- 172.21.15.78:0/4215885042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67aa86b0 0x55fb67e74ee0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 --2- 172.21.15.78:0/4215885042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa04d0 0x55fb67e749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 -- 172.21.15.78:0/4215885042 >> 172.21.15.78:0/4215885042 conn(0x55fb67abd4e0 msgr2=0x55fb67e6f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 -- 172.21.15.78:0/4215885042 shutdown_connections 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.156+0000 7f1deddc4740 1 -- 172.21.15.78:0/4215885042 wait complete. 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 -- start start 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 Processor -- start 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e78040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67ee8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01a98b0 con 0x55fb67ac9390 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01aad20 con 0x55fb67aa04d0 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01a9f80 con 0x55fb67aa86b0 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1deddc4740 10 client.0 _ll_register_callbacks cb 0x7f1dc8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67ee8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67ee8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42662/0 (socket says 172.21.15.78:42662) 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 -- 172.21.15.78:0/351223865 learned_addr learned my addr 172.21.15.78:0/351223865 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e78040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1ddffff640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 -- 172.21.15.78:0/351223865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 msgr2=0x55fb67e78580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e78580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 -- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 msgr2=0x55fb67e78040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e78040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de54aa640 1 -- 172.21.15.78:0/351223865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb67d904b0 con 0x55fb67ac9390 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.157+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e78040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1de54aa640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67ee8280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800ac90 tx=0x7f1dc8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d65d9cbdde51b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:50.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1dbffff640 1 -- 172.21.15.78:0/351223865 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8010430 con 0x55fb67ac9390 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1dbffff640 1 -- 172.21.15.78:0/351223865 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1dc800a5b0 con 0x55fb67ac9390 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd00ff030 con 0x55fb67ac9390 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1dbffff640 1 -- 172.21.15.78:0/351223865 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1dc803f050 con 0x55fb67ac9390 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 10 client.15585 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fb67ee93f0 con 0x55fb67ac9390 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1dbffff640 1 -- 172.21.15.78:0/351223865 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd00ff030 con 0x55fb67ac9390 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1dbffff640 1 -- 172.21.15.78:0/351223865 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1dc8024790 con 0x55fb67ac9390 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 10 client.15585 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 20 client.15585 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a5d3a0 0x55fb67eebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 4 client.15585 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.158+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb67dd7800 con 0x55fb67a5d3a0 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a5d3a0 0x55fb67eebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:50.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1de4ca9640 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a5d3a0 0x55fb67eebde0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d669946803959b server_cookie=49d4487e78770265 in_seq=0 out_seq=0 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1dbffff640 10 client.15585 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1dbffff640 1 -- 172.21.15.78:0/351223865 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fb67dd7800 con 0x55fb67a5d3a0 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1dbffff640 10 client.15585 handle_command_reply: tid=0 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 client.15585 shutdown 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 msgr2=0x55fb67ee8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67ee8280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800ac90 tx=0x7f1dc8005fd0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 shutdown_connections 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb67ac9390 0x55fb67ee8280 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fb67a5d3a0 0x55fb67eebde0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb67aa86b0 0x55fb67e78580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 --2- 172.21.15.78:0/351223865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fb67aa04d0 0x55fb67e78040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1dbffff640 0 client.15585 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.159+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 >> 172.21.15.78:0/351223865 conn(0x55fb67abd4e0 msgr2=0x55fb67e749a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 shutdown_connections 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1deddc4740 1 -- 172.21.15.78:0/351223865 wait complete. 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1deddc4740 20 client.15585 trim_cache size 0 max 16384 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stdout: "c4c0dd68-4c31-45d6-aff1-a0ca900d1d20": { 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T13:59:50.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c4c0dd68-4c31-45d6-aff1-a0ca900d1d20", 2024-04-25T13:59:50.168 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T13:59:50.168 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T13:59:50.168 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T13:59:50.168 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T13:59:50.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1dcc0afb70 msgr2=0x7f1dcc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1dcc0afb70 0x7f1dcc0b2080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55fb67e6a780 tx=0x7f1dd800b4d0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 msgr2=0x7f1de019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de019f080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1dd002ade0 tx=0x7f1dd00319f0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 shutdown_connections 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1de0108930 0x7f1de01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1dcc0afb70 0x7f1dcc0b2080 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1de0103c30 0x7f1de019f080 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 --2- 172.21.15.78:0/1553689905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de01029f0 0x7f1de006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 >> 172.21.15.78:0/1553689905 conn(0x7f1de00fdbe0 msgr2=0x7f1de0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 shutdown_connections 2024-04-25T13:59:50.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T13:59:50.160+0000 7f1de6f0d640 1 -- 172.21.15.78:0/1553689905 wait complete. 2024-04-25T13:59:50.175 INFO:tasks.cephfs.filesystem:scrub status for tag:c4c0dd68-4c31-45d6-aff1-a0ca900d1d20 - {'path': '/', 'tag': 'c4c0dd68-4c31-45d6-aff1-a0ca900d1d20', 'options': 'recursive,force'} 2024-04-25T13:59:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:50 smithi078 ceph-mon[29163]: pgmap v358: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T13:59:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:50 smithi129 ceph-mon[32569]: pgmap v358: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T13:59:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:50 smithi112 ceph-mon[32617]: pgmap v358: 97 pgs: 97 active+clean; 653 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T13:59:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:51 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T13:59:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:51 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:51 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T13:59:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:51 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:51 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T13:59:51.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:51 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T13:59:51.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:51 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:51.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:51 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T13:59:51.668 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:51 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:51.668 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:51 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T13:59:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:51 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T13:59:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:51 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:51 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T13:59:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:51 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T13:59:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:51 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T13:59:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:52 smithi129 ceph-mon[32569]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T13:59:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:52 smithi078 ceph-mon[29163]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T13:59:52.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:52 smithi112 ceph-mon[32617]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T13:59:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:54 smithi129 ceph-mon[32569]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T13:59:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:54 smithi078 ceph-mon[29163]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T13:59:54.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:54 smithi112 ceph-mon[32617]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T13:59:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:56 smithi129 ceph-mon[32569]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T13:59:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:56 smithi078 ceph-mon[29163]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T13:59:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:56 smithi112 ceph-mon[32617]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-25T13:59:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 -- 172.21.15.78:0/3239080200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c104d90 msgr2=0x7fb79c1051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3239080200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c104d90 0x7fb79c1051a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb790009900 tx=0x7fb79002a410 comp rx=0 tx=0).stop 2024-04-25T13:59:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 -- 172.21.15.78:0/3239080200 shutdown_connections 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3239080200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb79c109a90 0x7fb79c10beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3239080200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c104d90 0x7fb79c1051a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3239080200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c103b50 0x7fb79c104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 -- 172.21.15.78:0/3239080200 >> 172.21.15.78:0/3239080200 conn(0x7fb79c0fed20 msgr2=0x7fb79c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 -- 172.21.15.78:0/3239080200 shutdown_connections 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.949+0000 7fb7a250d640 1 -- 172.21.15.78:0/3239080200 wait complete. 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 Processor -- start 2024-04-25T13:59:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 -- start start 2024-04-25T13:59:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c103b50 0x7fb79c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb79c104d90 0x7fb79c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c109a90 0x7fb79c1a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10f680 con 0x7fb79c109a90 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10f500 con 0x7fb79c103b50 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a250d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10f800 con 0x7fb79c104d90 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a0aaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c109a90 0x7fb79c1a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a0aaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c109a90 0x7fb79c1a3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37492/0 (socket says 172.21.15.78:37492) 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb7a0aaa640 1 -- 172.21.15.78:0/3442246073 learned_addr learned my addr 172.21.15.78:0/3442246073 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb79c104d90 0x7fb79c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.950+0000 7fb79bfff640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c103b50 0x7fb79c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a0aaa640 1 -- 172.21.15.78:0/3442246073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb79c104d90 msgr2=0x7fb79c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a0aaa640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb79c104d90 0x7fb79c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a0aaa640 1 -- 172.21.15.78:0/3442246073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c103b50 msgr2=0x7fb79c19c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a0aaa640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c103b50 0x7fb79c19c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a0aaa640 1 -- 172.21.15.78:0/3442246073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c1a3f70 con 0x7fb79c109a90 2024-04-25T13:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a0aaa640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c109a90 0x7fb79c1a3900 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb79400eaa0 tx=0x7fb79400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7997fa640 1 -- 172.21.15.78:0/3442246073 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79400cdb0 con 0x7fb79c109a90 2024-04-25T13:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c1a4260 con 0x7fb79c109a90 2024-04-25T13:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7997fa640 1 -- 172.21.15.78:0/3442246073 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb794005020 con 0x7fb79c109a90 2024-04-25T13:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7997fa640 1 -- 172.21.15.78:0/3442246073 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940116d0 con 0x7fb79c109a90 2024-04-25T13:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.951+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c1a47b0 con 0x7fb79c109a90 2024-04-25T13:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.953+0000 7fb7997fa640 1 -- 172.21.15.78:0/3442246073 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb7940051d0 con 0x7fb79c109a90 2024-04-25T13:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 Processor -- start 2024-04-25T13:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 -- start start 2024-04-25T13:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c9b1b0 0x55933403b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x55933403bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334042df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559333fa1460 con 0x559333c9b1b0 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559333f5b590 con 0x559333c7a4d0 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a93be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559333f9d420 con 0x559333c722f0 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c9b1b0 0x55933403b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c9b1b0 0x55933403b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37504/0 (socket says 172.21.15.78:37504) 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 -- 172.21.15.78:0/18694434 learned_addr learned my addr 172.21.15.78:0/18694434 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79bfff640 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x55933403bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7997fa640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7840afae0 0x7fb7840b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334042df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 -- 172.21.15.78:0/18694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 msgr2=0x55933403bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x55933403bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 -- 172.21.15.78:0/18694434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 msgr2=0x559334042df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334042df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb79b7fe640 1 -- 172.21.15.78:0/18694434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559333f680d0 con 0x559333c9b1b0 2024-04-25T13:59:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.954+0000 7fb7a0aaa640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7840afae0 0x7fb7840b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.955+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c9b1b0 0x55933403b970 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb7900096c0 tx=0x7fb790009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b2121532bcbb81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.955+0000 7fb7997fa640 1 -- 172.21.15.78:0/3442246073 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb7940ded00 con 0x7fb79c109a90 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.955+0000 7fb7827fc640 1 -- 172.21.15.78:0/18694434 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790035070 con 0x559333c9b1b0 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.955+0000 7fb7827fc640 1 -- 172.21.15.78:0/18694434 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7900318a0 con 0x559333c9b1b0 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.955+0000 7fb7827fc640 1 -- 172.21.15.78:0/18694434 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79003b460 con 0x559333c9b1b0 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 -- 172.21.15.78:0/18694434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c9b1b0 msgr2=0x55933403b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c9b1b0 0x55933403b970 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb7900096c0 tx=0x7fb790009610 comp rx=0 tx=0).stop 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a0aaa640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7840afae0 0x7fb7840b1ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a3300 tx=0x7fb7940a2b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 -- 172.21.15.78:0/18694434 shutdown_connections 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334042df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x55933403bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 --2- 172.21.15.78:0/18694434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c9b1b0 0x55933403b970 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 -- 172.21.15.78:0/18694434 >> 172.21.15.78:0/18694434 conn(0x559333c8f300 msgr2=0x559334037300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 -- 172.21.15.78:0/18694434 shutdown_connections 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 -- 172.21.15.78:0/18694434 wait complete. 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 -- start start 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.956+0000 7fb7a93be740 1 Processor -- start 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x559334045620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334045b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 0x55933404cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5593340437e0 con 0x559333c722f0 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559333f28b60 con 0x559333c7a4d0 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559334043960 con 0x559333c9b1b0 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 10 client.0 _ll_register_callbacks cb 0x559333f4d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x559334045620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x559334045620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37514/0 (socket says 172.21.15.78:37514) 2024-04-25T13:59:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 -- 172.21.15.78:0/2214463763 learned_addr learned my addr 172.21.15.78:0/2214463763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79bfff640 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334045b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 -- 172.21.15.78:0/2214463763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 msgr2=0x55933404cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 0x55933404cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 -- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 msgr2=0x559334045b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334045b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 -- 172.21.15.78:0/2214463763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559333cb3210 con 0x559333c722f0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79bfff640 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334045b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x559334045620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb790031840 tx=0x7fb790031d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a1a65470d8bcab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb77b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb77b7fe640 1 -- 172.21.15.78:0/2214463763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790050070 con 0x559333c722f0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb77b7fe640 1 -- 172.21.15.78:0/2214463763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb790003fb0 con 0x559333c722f0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.957+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7900318a0 con 0x559333c722f0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb77b7fe640 1 -- 172.21.15.78:0/2214463763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb790035070 con 0x559333c722f0 2024-04-25T13:59:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 10 client.15597 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559333d7f620 con 0x559333c722f0 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb77b7fe640 1 -- 172.21.15.78:0/2214463763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7900318a0 con 0x559333c722f0 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb77b7fe640 1 -- 172.21.15.78:0/2214463763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb79003bb00 con 0x559333c722f0 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 10 client.15597 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 20 client.15597 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c59d80 0x559334052f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 4 client.15597 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559333faf070 con 0x559333c59d80 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.958+0000 7fb79bfff640 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c59d80 0x559334052f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.959+0000 7fb79bfff640 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c59d80 0x559334052f10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f1a3e18cb823e7 server_cookie=314c727ddeb0484a in_seq=0 out_seq=0 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.959+0000 7fb77b7fe640 10 client.15597 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.959+0000 7fb77b7fe640 1 -- 172.21.15.78:0/2214463763 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559333faf070 con 0x559333c59d80 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.959+0000 7fb77b7fe640 10 client.15597 handle_command_reply: tid=0 2024-04-25T13:59:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.959+0000 7fb7a93be740 1 client.15597 shutdown 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 msgr2=0x559334045620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x559334045620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb790031840 tx=0x7fb790031d40 comp rx=0 tx=0).stop 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 shutdown_connections 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c59d80 0x559334052f10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 0x55933404cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x559334045b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb77b7fe640 0 client.15597 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 --2- 172.21.15.78:0/2214463763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x559334045620 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 >> 172.21.15.78:0/2214463763 conn(0x559333c8f300 msgr2=0x55933403a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 shutdown_connections 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 1 -- 172.21.15.78:0/2214463763 wait complete. 2024-04-25T13:59:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.960+0000 7fb7a93be740 20 client.15597 trim_cache size 0 max 16384 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 Processor -- start 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 -- start start 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x559334037360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c7a4d0 0x55933404ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c9b1b0 0x55933403a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1afd30 con 0x559333c7a4d0 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1ad600 con 0x559333c9b1b0 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb7a93be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1ab450 con 0x559333c722f0 2024-04-25T13:59:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb79bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c7a4d0 0x55933404ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb79bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c7a4d0 0x55933404ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37522/0 (socket says 172.21.15.78:37522) 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.975+0000 7fb79bfff640 1 -- 172.21.15.78:0/3906663943 learned_addr learned my addr 172.21.15.78:0/3906663943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c9b1b0 0x55933403a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x559334037360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79bfff640 1 -- 172.21.15.78:0/3906663943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 msgr2=0x559334037360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79bfff640 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x559334037360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79bfff640 1 -- 172.21.15.78:0/3906663943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c9b1b0 msgr2=0x55933403a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79bfff640 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c9b1b0 0x55933403a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79bfff640 1 -- 172.21.15.78:0/3906663943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559334040500 con 0x559333c7a4d0 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c9b1b0 0x55933403a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb79bfff640 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c7a4d0 0x55933404ca50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00bde0 tx=0x7fb78c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4573abd1c45d2ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb781ffb640 1 -- 172.21.15.78:0/3906663943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c01d430 con 0x559333c7a4d0 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb781ffb640 1 -- 172.21.15.78:0/3906663943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb78c01da80 con 0x559333c7a4d0 2024-04-25T13:59:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.976+0000 7fb781ffb640 1 -- 172.21.15.78:0/3906663943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c011d80 con 0x559333c7a4d0 2024-04-25T13:59:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 -- 172.21.15.78:0/3906663943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c7a4d0 msgr2=0x55933404ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c7a4d0 0x55933404ca50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00bde0 tx=0x7fb78c0044d0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 -- 172.21.15.78:0/3906663943 shutdown_connections 2024-04-25T13:59:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c9b1b0 0x55933403a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c7a4d0 0x55933404ca50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 --2- 172.21.15.78:0/3906663943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c722f0 0x559334037360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 -- 172.21.15.78:0/3906663943 >> 172.21.15.78:0/3906663943 conn(0x559333c8f300 msgr2=0x559334036c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 -- 172.21.15.78:0/3906663943 shutdown_connections 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 -- 172.21.15.78:0/3906663943 wait complete. 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.977+0000 7fb7a93be740 1 -- start start 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 1 Processor -- start 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x55933404a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x55933404aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 0x55933404b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1abac0 con 0x559333c722f0 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1acf30 con 0x559333c7a4d0 2024-04-25T13:59:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1ac190 con 0x559333c9b1b0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb7a93be740 10 client.0 _ll_register_callbacks cb 0x7fb790031f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x55933404a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x55933404a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37526/0 (socket says 172.21.15.78:37526) 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 -- 172.21.15.78:0/624853447 learned_addr learned my addr 172.21.15.78:0/624853447 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79bfff640 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x55933404aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 0x55933404b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 -- 172.21.15.78:0/624853447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 msgr2=0x55933404b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 0x55933404b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 -- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 msgr2=0x55933404aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x55933404aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.978+0000 7fb79b7fe640 1 -- 172.21.15.78:0/624853447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933404bf70 con 0x559333c722f0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb79b7fe640 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x55933404a540 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb79002a930 tx=0x7fb790031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df573363ae847c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb77b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T13:59:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb77b7fe640 1 -- 172.21.15.78:0/624853447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79005d040 con 0x559333c722f0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb77b7fe640 1 -- 172.21.15.78:0/624853447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb7900043c0 con 0x559333c722f0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78c01da80 con 0x559333c722f0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb77b7fe640 1 -- 172.21.15.78:0/624853447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb790050040 con 0x559333c722f0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 10 client.15603 fetch_fsmap learned FSMap version 22 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5593340bb460 con 0x559333c722f0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb77b7fe640 1 -- 172.21.15.78:0/624853447 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb78c01da80 con 0x559333c722f0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb77b7fe640 1 -- 172.21.15.78:0/624853447 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb790061060 con 0x559333c722f0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 10 client.15603 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 20 client.15603 populate_metadata read hostname 'smithi078' 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c2f1c0 0x5593340bddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 4 client.15603 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.979+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559333fa8ee0 con 0x559333c2f1c0 2024-04-25T13:59:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.980+0000 7fb79bfff640 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c2f1c0 0x5593340bddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:59:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.980+0000 7fb77b7fe640 10 client.15603 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.980+0000 7fb79bfff640 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c2f1c0 0x5593340bddd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a266925d6ea4a77 server_cookie=c4c3755e6592617d in_seq=0 out_seq=0 2024-04-25T13:59:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.987+0000 7fb77b7fe640 1 -- 172.21.15.78:0/624853447 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559333fa8ee0 con 0x559333c2f1c0 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.987+0000 7fb77b7fe640 10 client.15603 handle_command_reply: tid=0 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.987+0000 7fb7a93be740 1 client.15603 shutdown 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 msgr2=0x55933404a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x55933404a540 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb79002a930 tx=0x7fb790031a50 comp rx=0 tx=0).stop 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 shutdown_connections 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559333c2f1c0 0x5593340bddd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559333c9b1b0 0x55933404b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559333c7a4d0 0x55933404aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 --2- 172.21.15.78:0/624853447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559333c722f0 0x55933404a540 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb77b7fe640 0 client.15603 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 >> 172.21.15.78:0/624853447 conn(0x559333c8f300 msgr2=0x5593340451f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 shutdown_connections 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 1 -- 172.21.15.78:0/624853447 wait complete. 2024-04-25T13:59:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a93be740 20 client.15603 trim_cache size 0 max 16384 2024-04-25T13:59:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T13:59:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T13:59:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T13:59:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7840afae0 msgr2=0x7fb7840b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7840afae0 0x7fb7840b1ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a3300 tx=0x7fb7940a2b00 comp rx=0 tx=0).stop 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c109a90 msgr2=0x7fb79c1a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c109a90 0x7fb79c1a3900 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb79400eaa0 tx=0x7fb79400ef60 comp rx=0 tx=0).stop 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 shutdown_connections 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb7840afae0 0x7fb7840b1ff0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb79c109a90 0x7fb79c1a3900 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb79c104d90 0x7fb79c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 --2- 172.21.15.78:0/3442246073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb79c103b50 0x7fb79c19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.988+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 >> 172.21.15.78:0/3442246073 conn(0x7fb79c0fed20 msgr2=0x7fb79c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:59:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.989+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 shutdown_connections 2024-04-25T13:59:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T13:59:56.989+0000 7fb7a250d640 1 -- 172.21.15.78:0/3442246073 wait complete. 2024-04-25T13:59:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 13:59:58 smithi129 ceph-mon[32569]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T13:59:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 13:59:58 smithi078 ceph-mon[29163]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T13:59:58.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 13:59:58 smithi112 ceph-mon[32617]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T14:00:00.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:00 smithi129 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T14:00:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:00 smithi078 ceph-mon[29163]: overall HEALTH_OK 2024-04-25T14:00:00.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:00 smithi112 ceph-mon[32617]: overall HEALTH_OK 2024-04-25T14:00:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:01 smithi129 ceph-mon[32569]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T14:00:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:01 smithi078 ceph-mon[29163]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T14:00:01.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:01 smithi112 ceph-mon[32617]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T14:00:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:02 smithi129 ceph-mon[32569]: pgmap v364: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T14:00:02.712 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:02 smithi078 ceph-mon[29163]: pgmap v364: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T14:00:02.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:02 smithi112 ceph-mon[32617]: pgmap v364: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T14:00:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:04 smithi129 ceph-mon[32569]: pgmap v365: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:00:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:04 smithi078 ceph-mon[29163]: pgmap v365: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:00:04.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:04 smithi112 ceph-mon[32617]: pgmap v365: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:00:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:06 smithi129 ceph-mon[32569]: pgmap v366: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:00:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:06 smithi078 ceph-mon[29163]: pgmap v366: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:00:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:06 smithi112 ceph-mon[32617]: pgmap v366: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T14:00:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:08 smithi129 ceph-mon[32569]: pgmap v367: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T14:00:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:08 smithi078 ceph-mon[29163]: pgmap v367: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T14:00:08.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:08 smithi112 ceph-mon[32617]: pgmap v367: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T14:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.086+0000 7f629b7fe640 1 -- 172.21.15.78:0/2871828654 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62880318a0 con 0x7f629c106f70 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- 172.21.15.78:0/2871828654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c106f70 msgr2=0x7f629c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2871828654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c106f70 0x7f629c109390 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6288009900 tx=0x7f628802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- 172.21.15.78:0/2871828654 shutdown_connections 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2871828654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f629c1098d0 0x7f629c10bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2871828654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c106f70 0x7f629c109390 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2871828654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c104610 0x7f629c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- 172.21.15.78:0/2871828654 >> 172.21.15.78:0/2871828654 conn(0x7f629c0fdc10 msgr2=0x7f629c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- 172.21.15.78:0/2871828654 shutdown_connections 2024-04-25T14:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- 172.21.15.78:0/2871828654 wait complete. 2024-04-25T14:00:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 Processor -- start 2024-04-25T14:00:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- start start 2024-04-25T14:00:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c104610 0x7f629c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f629c106f70 0x7f629c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1098d0 0x7f629c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f629c10f650 con 0x7f629c104610 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f629c10f4d0 con 0x7f629c1098d0 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a2c51640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f629c10f7d0 con 0x7f629c106f70 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c104610 0x7f629c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c104610 0x7f629c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35332/0 (socket says 172.21.15.78:35332) 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 -- 172.21.15.78:0/2115305767 learned_addr learned my addr 172.21.15.78:0/2115305767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f629bfff640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f629c106f70 0x7f629c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a11ee640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1098d0 0x7f629c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 -- 172.21.15.78:0/2115305767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f629c106f70 msgr2=0x7f629c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f629c106f70 0x7f629c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 -- 172.21.15.78:0/2115305767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1098d0 msgr2=0x7f629c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1098d0 0x7f629c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a09ed640 1 -- 172.21.15.78:0/2115305767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c1a1dc0 con 0x7f629c104610 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.087+0000 7f62a11ee640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1098d0 0x7f629c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.088+0000 7f62a09ed640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c104610 0x7f629c19a280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6290005fd0 tx=0x7f629000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.088+0000 7f6299ffb640 1 -- 172.21.15.78:0/2115305767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290017070 con 0x7f629c104610 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.088+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c1a20b0 con 0x7f629c104610 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.088+0000 7f6299ffb640 1 -- 172.21.15.78:0/2115305767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6290009c30 con 0x7f629c104610 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.088+0000 7f6299ffb640 1 -- 172.21.15.78:0/2115305767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290003d30 con 0x7f629c104610 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.088+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c1a2650 con 0x7f629c104610 2024-04-25T14:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.091+0000 7f6299ffb640 1 -- 172.21.15.78:0/2115305767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6290003ed0 con 0x7f629c104610 2024-04-25T14:00:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.091+0000 7f62a9be2740 1 Processor -- start 2024-04-25T14:00:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.091+0000 7f6299ffb640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f628c0afae0 0x7f628c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.091+0000 7f62a9be2740 1 -- start start 2024-04-25T14:00:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.091+0000 7f62a11ee640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f628c0afae0 0x7f628c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f6299ffb640 1 -- 172.21.15.78:0/2115305767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f62900de900 con 0x7f629c104610 2024-04-25T14:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b431b0 0x5576b9ee3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b224d0 0x5576b9eeadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a9be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5576b9e49460 con 0x5576b9b224d0 2024-04-25T14:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a9be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5576b9e03590 con 0x5576b9b431b0 2024-04-25T14:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a9be2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5576b9e45420 con 0x5576b9b1a2f0 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b224d0 0x5576b9eeadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a09ed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b431b0 0x5576b9ee3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b224d0 0x5576b9eeadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35344/0 (socket says 172.21.15.78:35344) 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 -- 172.21.15.78:0/4147656438 learned_addr learned my addr 172.21.15.78:0/4147656438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f629bfff640 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f628c0afae0 0x7f628c0b1ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f62940092f0 tx=0x7f6294009280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 -- 172.21.15.78:0/4147656438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b1a2f0 msgr2=0x5576b9ee3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 -- 172.21.15.78:0/4147656438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b431b0 msgr2=0x5576b9ee3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b431b0 0x5576b9ee3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.092+0000 7f62a11ee640 1 -- 172.21.15.78:0/4147656438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b9e100d0 con 0x5576b9b224d0 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.093+0000 7f62a11ee640 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b224d0 0x5576b9eeadf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6294012610 tx=0x7f6294013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a4a713eb184635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.093+0000 7f6286ffd640 1 -- 172.21.15.78:0/4147656438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294023070 con 0x5576b9b224d0 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.093+0000 7f6286ffd640 1 -- 172.21.15.78:0/4147656438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6294011bd0 con 0x5576b9b224d0 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.093+0000 7f6286ffd640 1 -- 172.21.15.78:0/4147656438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629401fc50 con 0x5576b9b224d0 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 -- 172.21.15.78:0/4147656438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b224d0 msgr2=0x5576b9eeadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b224d0 0x5576b9eeadf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6294012610 tx=0x7f6294013000 comp rx=0 tx=0).stop 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 -- 172.21.15.78:0/4147656438 shutdown_connections 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b224d0 0x5576b9eeadf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 --2- 172.21.15.78:0/4147656438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b431b0 0x5576b9ee3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 -- 172.21.15.78:0/4147656438 >> 172.21.15.78:0/4147656438 conn(0x5576b9b37300 msgr2=0x5576b9edf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 -- 172.21.15.78:0/4147656438 shutdown_connections 2024-04-25T14:00:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 -- 172.21.15.78:0/4147656438 wait complete. 2024-04-25T14:00:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 -- start start 2024-04-25T14:00:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.095+0000 7f62a9be2740 1 Processor -- start 2024-04-25T14:00:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b431b0 0x5576b9ef8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a9be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5576b9eefc50 con 0x5576b9b431b0 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a9be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5576b9dd0b60 con 0x5576b9b1a2f0 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a9be2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5576b9eefdd0 con 0x5576b9b224d0 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a9be2740 10 client.0 _ll_register_callbacks cb 0x5576b9df52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b431b0 0x5576b9ef8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b431b0 0x5576b9ef8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35358/0 (socket says 172.21.15.78:35358) 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 -- 172.21.15.78:0/1456172612 learned_addr learned my addr 172.21.15.78:0/1456172612 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f629bfff640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a09ed640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 -- 172.21.15.78:0/1456172612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 msgr2=0x5576b9ef1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 -- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 msgr2=0x5576b9ef19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a11ee640 1 -- 172.21.15.78:0/1456172612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b9b5b210 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.096+0000 7f62a09ed640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f62a11ee640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b431b0 0x5576b9ef8e60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6294009c60 tx=0x7f6294011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c207852cef559be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f627bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f627bfff640 1 -- 172.21.15.78:0/1456172612 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294037060 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f627bfff640 1 -- 172.21.15.78:0/1456172612 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f629401f510 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6294011bd0 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f627bfff640 1 -- 172.21.15.78:0/1456172612 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6294023050 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f62a9be2740 10 client.15615 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5576b9c27620 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f627bfff640 1 -- 172.21.15.78:0/1456172612 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6294011bd0 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f627bfff640 1 -- 172.21.15.78:0/1456172612 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f629400f040 con 0x5576b9b431b0 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f62a9be2740 10 client.15615 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.097+0000 7f62a9be2740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.098+0000 7f62a9be2740 20 client.15615 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.098+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b01d80 0x5576b9eff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.098+0000 7f62a9be2740 4 client.15615 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.098+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576b9e57070 con 0x5576b9b01d80 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.098+0000 7f62a09ed640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b01d80 0x5576b9eff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.098+0000 7f627bfff640 10 client.15615 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.098+0000 7f62a09ed640 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b01d80 0x5576b9eff3c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41be0def424771c server_cookie=fafef61505a3b4aa in_seq=0 out_seq=0 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f627bfff640 1 -- 172.21.15.78:0/1456172612 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576b9e57070 con 0x5576b9b01d80 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f627bfff640 10 client.15615 handle_command_reply: tid=0 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 client.15615 shutdown 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b431b0 msgr2=0x5576b9ef8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b431b0 0x5576b9ef8e60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6294009c60 tx=0x7f6294011fb0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 shutdown_connections 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b431b0 0x5576b9ef8e60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b01d80 0x5576b9eff3c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f627bfff640 0 client.15615 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1456172612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 >> 172.21.15.78:0/1456172612 conn(0x5576b9b37300 msgr2=0x5576b9ee2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 shutdown_connections 2024-04-25T14:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.099+0000 7f62a9be2740 1 -- 172.21.15.78:0/1456172612 wait complete. 2024-04-25T14:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.100+0000 7f62a9be2740 20 client.15615 trim_cache size 0 max 16384 2024-04-25T14:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 Processor -- start 2024-04-25T14:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 -- start start 2024-04-25T14:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b01d80 0x5576b9ee7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b224d0 0x5576b9ef8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1abab0 con 0x5576b9b1a2f0 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1a9380 con 0x5576b9b224d0 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a9be2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1a71d0 con 0x5576b9b01d80 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f629bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f629bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35368/0 (socket says 172.21.15.78:35368) 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f629bfff640 1 -- 172.21.15.78:0/848314340 learned_addr learned my addr 172.21.15.78:0/848314340 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a09ed640 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b01d80 0x5576b9ee7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.128+0000 7f62a11ee640 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b224d0 0x5576b9ef8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f629bfff640 1 -- 172.21.15.78:0/848314340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b01d80 msgr2=0x5576b9ee7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f629bfff640 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b01d80 0x5576b9ee7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f629bfff640 1 -- 172.21.15.78:0/848314340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b224d0 msgr2=0x5576b9ef8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f629bfff640 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b224d0 0x5576b9ef8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f629bfff640 1 -- 172.21.15.78:0/848314340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b9ef92c0 con 0x5576b9b1a2f0 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f62a11ee640 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b224d0 0x5576b9ef8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f629bfff640 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee7f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f628802a930 tx=0x7f6288031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35aa7f27c290ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f62867fc640 1 -- 172.21.15.78:0/848314340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288035040 con 0x5576b9b1a2f0 2024-04-25T14:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f62867fc640 1 -- 172.21.15.78:0/848314340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6288004030 con 0x5576b9b1a2f0 2024-04-25T14:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.129+0000 7f62867fc640 1 -- 172.21.15.78:0/848314340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6288003b90 con 0x5576b9b1a2f0 2024-04-25T14:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 -- 172.21.15.78:0/848314340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b1a2f0 msgr2=0x5576b9ee7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee7f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f628802a930 tx=0x7f6288031ec0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 -- 172.21.15.78:0/848314340 shutdown_connections 2024-04-25T14:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b224d0 0x5576b9ef8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b1a2f0 0x5576b9ee7f70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 --2- 172.21.15.78:0/848314340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b01d80 0x5576b9ee7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 -- 172.21.15.78:0/848314340 >> 172.21.15.78:0/848314340 conn(0x5576b9b37300 msgr2=0x5576b9ee38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 -- 172.21.15.78:0/848314340 shutdown_connections 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.130+0000 7f62a9be2740 1 -- 172.21.15.78:0/848314340 wait complete. 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 -- start start 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 Processor -- start 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b01d80 0x5576b9ef61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1a7840 con 0x5576b9b01d80 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1a8cb0 con 0x5576b9b1a2f0 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a9be2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1a7f10 con 0x5576b9b224d0 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.131+0000 7f62a09ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b01d80 0x5576b9ef61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b01d80 0x5576b9ef61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35376/0 (socket says 172.21.15.78:35376) 2024-04-25T14:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 -- 172.21.15.78:0/1331208911 learned_addr learned my addr 172.21.15.78:0/1331208911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a9be2740 10 client.0 _ll_register_callbacks cb 0x7f62900c9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f629bfff640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a11ee640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 -- 172.21.15.78:0/1331208911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 msgr2=0x5576b9ef7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 -- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 msgr2=0x5576b9ef6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 -- 172.21.15.78:0/1331208911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b9ef7bf0 con 0x5576b9b01d80 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f629bfff640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a09ed640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b01d80 0x5576b9ef61c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f62900a4040 tx=0x7f62900a6ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47390819c5a2b386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f627bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f627bfff640 1 -- 172.21.15.78:0/1331208911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62900aa040 con 0x5576b9b01d80 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f627bfff640 1 -- 172.21.15.78:0/1331208911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f62900a3060 con 0x5576b9b01d80 2024-04-25T14:00:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6288004030 con 0x5576b9b01d80 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f627bfff640 1 -- 172.21.15.78:0/1331208911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f62900041c0 con 0x5576b9b01d80 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.132+0000 7f62a9be2740 10 client.15621 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5576b9ef8e60 con 0x5576b9b01d80 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f627bfff640 1 -- 172.21.15.78:0/1331208911 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6288004030 con 0x5576b9b01d80 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f627bfff640 1 -- 172.21.15.78:0/1331208911 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f62900a5aa0 con 0x5576b9b01d80 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f62a9be2740 10 client.15621 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f62a9be2740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f62a9be2740 20 client.15621 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b431b0 0x5576b9f69b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f62a9be2740 4 client.15621 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576b9e50ee0 con 0x5576b9b431b0 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.133+0000 7f629bfff640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b431b0 0x5576b9f69b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.134+0000 7f629bfff640 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b431b0 0x5576b9f69b90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b714f91c278c60ba server_cookie=6a114e7c842c1a88 in_seq=0 out_seq=0 2024-04-25T14:00:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.134+0000 7f627bfff640 10 client.15621 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f627bfff640 1 -- 172.21.15.78:0/1331208911 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576b9e50ee0 con 0x5576b9b431b0 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f627bfff640 10 client.15621 handle_command_reply: tid=0 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 client.15621 shutdown 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b01d80 msgr2=0x5576b9ef61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b01d80 0x5576b9ef61c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f62900a4040 tx=0x7f62900a6ee0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 shutdown_connections 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5576b9b431b0 0x5576b9f69b90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5576b9b224d0 0x5576b9ef7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5576b9b1a2f0 0x5576b9ef6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 --2- 172.21.15.78:0/1331208911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576b9b01d80 0x5576b9ef61c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 >> 172.21.15.78:0/1331208911 conn(0x5576b9b37300 msgr2=0x5576b9ee7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f627bfff640 0 client.15621 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 shutdown_connections 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 1 -- 172.21.15.78:0/1331208911 wait complete. 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.138+0000 7f62a9be2740 20 client.15621 trim_cache size 0 max 16384 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:00:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:00:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f628c0afae0 msgr2=0x7f628c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f628c0afae0 0x7f628c0b1ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f62940092f0 tx=0x7f6294009280 comp rx=0 tx=0).stop 2024-04-25T14:00:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c104610 msgr2=0x7f629c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c104610 0x7f629c19a280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6290005fd0 tx=0x7f629000dde0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 shutdown_connections 2024-04-25T14:00:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f629c1098d0 0x7f629c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f629c106f70 0x7f629c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f628c0afae0 0x7f628c0b1ff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 --2- 172.21.15.78:0/2115305767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f629c104610 0x7f629c19a280 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 >> 172.21.15.78:0/2115305767 conn(0x7f629c0fdc10 msgr2=0x7f629c105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 shutdown_connections 2024-04-25T14:00:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:10.139+0000 7f62a2c51640 1 -- 172.21.15.78:0/2115305767 wait complete. 2024-04-25T14:00:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:10 smithi129 ceph-mon[32569]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:00:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:10 smithi078 ceph-mon[29163]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:00:10.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:10 smithi112 ceph-mon[32617]: pgmap v368: 97 pgs: 97 active+clean; 664 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-25T14:00:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:12 smithi078 ceph-mon[29163]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-25T14:00:12.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:12 smithi129 ceph-mon[32569]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-25T14:00:12.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:12 smithi112 ceph-mon[32617]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-25T14:00:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:14 smithi129 ceph-mon[32569]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:14 smithi078 ceph-mon[29163]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:14 smithi112 ceph-mon[32617]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:16.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:16 smithi129 ceph-mon[32569]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:16.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:16 smithi078 ceph-mon[29163]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:16.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:16 smithi112 ceph-mon[32617]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:18 smithi129 ceph-mon[32569]: pgmap v372: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:18.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:18 smithi078 ceph-mon[29163]: pgmap v372: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:18.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:18 smithi112 ceph-mon[32617]: pgmap v372: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:20.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:20.176 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 -- 172.21.15.78:0/3978391269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b01029e0 msgr2=0x7f85b01036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 --2- 172.21.15.78:0/3978391269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b01029e0 0x7f85b01036e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009900 tx=0x7f85ac02a410 comp rx=0 tx=0).stop 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 -- 172.21.15.78:0/3978391269 shutdown_connections 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 --2- 172.21.15.78:0/3978391269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b0108920 0x7f85b010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 --2- 172.21.15.78:0/3978391269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b0103c20 0x7f85b0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 --2- 172.21.15.78:0/3978391269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b01029e0 0x7f85b01036e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 -- 172.21.15.78:0/3978391269 >> 172.21.15.78:0/3978391269 conn(0x7f85b00fdbd0 msgr2=0x7f85b0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 -- 172.21.15.78:0/3978391269 shutdown_connections 2024-04-25T14:00:20.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.349+0000 7f85b844c640 1 -- 172.21.15.78:0/3978391269 wait complete. 2024-04-25T14:00:20.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.350+0000 7f85b844c640 1 Processor -- start 2024-04-25T14:00:20.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.350+0000 7f85b844c640 1 -- start start 2024-04-25T14:00:20.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.350+0000 7f85b844c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b01029e0 0x7f85b01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.350+0000 7f85b844c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b0103c20 0x7f85b01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b844c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b0108920 0x7f85b019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b01029e0 0x7f85b01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b01029e0 0x7f85b01981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57116/0 (socket says 172.21.15.78:57116) 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 -- 172.21.15.78:0/733182823 learned_addr learned my addr 172.21.15.78:0/733182823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010d3f0 con 0x7f85b0108920 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010d270 con 0x7f85b01029e0 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010d570 con 0x7f85b0103c20 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b59e7640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b0103c20 0x7f85b01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b69e9640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b0108920 0x7f85b019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 -- 172.21.15.78:0/733182823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b0103c20 msgr2=0x7f85b01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b0103c20 0x7f85b01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 -- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b0108920 msgr2=0x7f85b019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b0108920 0x7f85b019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b61e8640 1 -- 172.21.15.78:0/733182823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b019fce0 con 0x7f85b01029e0 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b59e7640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b0103c20 0x7f85b01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.351+0000 7f85b69e9640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b0108920 0x7f85b019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.352+0000 7f85b61e8640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b01029e0 0x7f85b01981a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85ac02a930 tx=0x7f85ac0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.352+0000 7f85a77fe640 1 -- 172.21.15.78:0/733182823 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac03f030 con 0x7f85b01029e0 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.352+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b019ff70 con 0x7f85b01029e0 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.352+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01a03d0 con 0x7f85b01029e0 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.352+0000 7f85a77fe640 1 -- 172.21.15.78:0/733182823 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ac003cb0 con 0x7f85b01029e0 2024-04-25T14:00:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.353+0000 7f85a77fe640 1 -- 172.21.15.78:0/733182823 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac035070 con 0x7f85b01029e0 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.355+0000 7f85bf3db740 1 Processor -- start 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85a77fe640 1 -- 172.21.15.78:0/733182823 <== mon.2 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f85ac042030 con 0x7f85b01029e0 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85bf3db740 1 -- start start 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f66a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52b9c4d0 0x55db52f66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f67c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85bf3db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db52ecbd80 con 0x55db52bc5390 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85bf3db740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db52e858b0 con 0x55db52b9c4d0 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85bf3db740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55db52ec7d40 con 0x55db52ba46b0 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85b69e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f66a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85b69e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f66a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39180/0 (socket says 172.21.15.78:39180) 2024-04-25T14:00:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85b69e9640 1 -- 172.21.15.78:0/3751124999 learned_addr learned my addr 172.21.15.78:0/3751124999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85b61e8640 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52b9c4d0 0x55db52f66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.356+0000 7f85b59e7640 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f67c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85b69e9640 1 -- 172.21.15.78:0/3751124999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 msgr2=0x55db52f67c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85b69e9640 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f67c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85a77fe640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f859c0afb70 0x7f859c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85b69e9640 1 -- 172.21.15.78:0/3751124999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52b9c4d0 msgr2=0x55db52f66fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85b69e9640 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52b9c4d0 0x55db52f66fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85b69e9640 1 -- 172.21.15.78:0/3751124999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db52e56a40 con 0x55db52bc5390 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85a77fe640 1 -- 172.21.15.78:0/733182823 <== mon.2 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f85ac0ff100 con 0x7f85b01029e0 2024-04-25T14:00:20.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85b69e9640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f859c0afb70 0x7f859c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.357+0000 7f85b69e9640 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f66a60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85a8011820 tx=0x7f85a8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e262484fbb7607e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.358+0000 7f85b69e9640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f859c0afb70 0x7f859c0b2080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a802b000 tx=0x7f85a802c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.358+0000 7f85a4bfe640 1 -- 172.21.15.78:0/3751124999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a801ad20 con 0x55db52bc5390 2024-04-25T14:00:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.358+0000 7f85a4bfe640 1 -- 172.21.15.78:0/3751124999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a80258e0 con 0x55db52bc5390 2024-04-25T14:00:20.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.358+0000 7f85a4bfe640 1 -- 172.21.15.78:0/3751124999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8010650 con 0x55db52bc5390 2024-04-25T14:00:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 -- 172.21.15.78:0/3751124999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 msgr2=0x55db52f66a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f66a60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85a8011820 tx=0x7f85a8011c20 comp rx=0 tx=0).stop 2024-04-25T14:00:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 -- 172.21.15.78:0/3751124999 shutdown_connections 2024-04-25T14:00:20.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f67c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52b9c4d0 0x55db52f66fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3751124999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f66a60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 -- 172.21.15.78:0/3751124999 >> 172.21.15.78:0/3751124999 conn(0x55db52bb94e0 msgr2=0x55db52f62270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 -- 172.21.15.78:0/3751124999 shutdown_connections 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 -- 172.21.15.78:0/3751124999 wait complete. 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 -- start start 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.359+0000 7f85bf3db740 1 Processor -- start 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f70940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f70e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52bc5390 0x55db52f77de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85bf3db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db52f68780 con 0x55db52b9c4d0 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85bf3db740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55db52e52d10 con 0x55db52bc5390 2024-04-25T14:00:20.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85bf3db740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55db52f68900 con 0x55db52ba46b0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f70940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f70940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39192/0 (socket says 172.21.15.78:39192) 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 -- 172.21.15.78:0/3045982464 learned_addr learned my addr 172.21.15.78:0/3045982464 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85bf3db740 10 client.0 _ll_register_callbacks cb 0x55db52e7f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b69e9640 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52bc5390 0x55db52f77de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b59e7640 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f70e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 -- 172.21.15.78:0/3045982464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 msgr2=0x55db52f70e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f70e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 -- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52bc5390 msgr2=0x55db52f77de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52bc5390 0x55db52f77de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 -- 172.21.15.78:0/3045982464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db52ca77d0 con 0x55db52b9c4d0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f85b61e8640 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f70940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0c47e0 tx=0x7f85ac0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271cf15a1ddf7d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.360+0000 7f8599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:20.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.361+0000 7f8599ffb640 1 -- 172.21.15.78:0/3045982464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0c7340 con 0x55db52b9c4d0 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.361+0000 7f8599ffb640 1 -- 172.21.15.78:0/3045982464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f85ac03b460 con 0x55db52b9c4d0 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.361+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a80258e0 con 0x55db52b9c4d0 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.361+0000 7f8599ffb640 1 -- 172.21.15.78:0/3045982464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f85ac003e60 con 0x55db52b9c4d0 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.361+0000 7f85bf3db740 10 client.15633 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.361+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55db52bdd3f0 con 0x55db52b9c4d0 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.361+0000 7f8599ffb640 1 -- 172.21.15.78:0/3045982464 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a80258e0 con 0x55db52b9c4d0 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.362+0000 7f8599ffb640 1 -- 172.21.15.78:0/3045982464 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f85ac0c3800 con 0x55db52b9c4d0 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.362+0000 7f85bf3db740 10 client.15633 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.362+0000 7f85bf3db740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.362+0000 7f85bf3db740 20 client.15633 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.362+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b83f60 0x55db52f7e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.362+0000 7f85bf3db740 4 client.15633 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:20.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.362+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db52ed9990 con 0x55db52b83f60 2024-04-25T14:00:20.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.363+0000 7f85b59e7640 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b83f60 0x55db52f7e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.363+0000 7f8599ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.363+0000 7f85b59e7640 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b83f60 0x55db52f7e530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810075010603a528 server_cookie=96570c7b808f33f4 in_seq=0 out_seq=0 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f8599ffb640 1 -- 172.21.15.78:0/3045982464 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db52ed9990 con 0x55db52b83f60 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f8599ffb640 10 client.15633 handle_command_reply: tid=0 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 client.15633 shutdown 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 msgr2=0x55db52f70940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f70940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0c47e0 tx=0x7f85ac0057e0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 shutdown_connections 2024-04-25T14:00:20.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52bc5390 0x55db52f77de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b83f60 0x55db52f7e530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52ba46b0 0x55db52f70e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3045982464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f70940 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 >> 172.21.15.78:0/3045982464 conn(0x55db52bb94e0 msgr2=0x55db52f65180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f8599ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 shutdown_connections 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 1 -- 172.21.15.78:0/3045982464 wait complete. 2024-04-25T14:00:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.364+0000 7f85bf3db740 20 client.15633 trim_cache size 0 max 16384 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.399+0000 7f85bf3db740 1 Processor -- start 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.399+0000 7f85bf3db740 1 -- start start 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52b9c4d0 0x55db52f6f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f77d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85bf3db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8594010d40 con 0x55db52bc5390 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85bf3db740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f859400e610 con 0x55db52ba46b0 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85bf3db740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f859400c460 con 0x55db52b9c4d0 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39204/0 (socket says 172.21.15.78:39204) 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 -- 172.21.15.78:0/3960658281 learned_addr learned my addr 172.21.15.78:0/3960658281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b61e8640 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52b9c4d0 0x55db52f6f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b59e7640 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f77d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 -- 172.21.15.78:0/3960658281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52b9c4d0 msgr2=0x55db52f6f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52b9c4d0 0x55db52f6f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 -- 172.21.15.78:0/3960658281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 msgr2=0x55db52f77d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f77d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b69e9640 1 -- 172.21.15.78:0/3960658281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db52b9d0f0 con 0x55db52bc5390 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.400+0000 7f85b61e8640 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52b9c4d0 0x55db52f6f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.401+0000 7f85b69e9640 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f623b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009f60 tx=0x7f85a8011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a650b0855e007b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.401+0000 7f859bfff640 1 -- 172.21.15.78:0/3960658281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8010430 con 0x55db52bc5390 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.401+0000 7f859bfff640 1 -- 172.21.15.78:0/3960658281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a80105d0 con 0x55db52bc5390 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.401+0000 7f859bfff640 1 -- 172.21.15.78:0/3960658281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8032470 con 0x55db52bc5390 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 -- 172.21.15.78:0/3960658281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 msgr2=0x55db52f623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f623b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009f60 tx=0x7f85a8011fa0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 -- 172.21.15.78:0/3960658281 shutdown_connections 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52bc5390 0x55db52f623b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f77d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 --2- 172.21.15.78:0/3960658281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52b9c4d0 0x55db52f6f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 -- 172.21.15.78:0/3960658281 >> 172.21.15.78:0/3960658281 conn(0x55db52bb94e0 msgr2=0x55db52f61ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 -- 172.21.15.78:0/3960658281 shutdown_connections 2024-04-25T14:00:20.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.402+0000 7f85bf3db740 1 -- 172.21.15.78:0/3960658281 wait complete. 2024-04-25T14:00:20.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 -- start start 2024-04-25T14:00:20.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 Processor -- start 2024-04-25T14:00:20.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f77c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85b61e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f77c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85b61e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f77c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39214/0 (socket says 172.21.15.78:39214) 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85b61e8640 1 -- 172.21.15.78:0/494338276 learned_addr learned my addr 172.21.15.78:0/494338276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52bc5390 0x55db52f74d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f859400cad0 con 0x55db52b9c4d0 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f859400df40 con 0x55db52ba46b0 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.403+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f859400d1a0 con 0x55db52bc5390 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85b59e7640 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85b61e8640 1 -- 172.21.15.78:0/494338276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52bc5390 msgr2=0x55db52f74d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85b61e8640 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52bc5390 0x55db52f74d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85b61e8640 1 -- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 msgr2=0x55db52f747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85b61e8640 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85b61e8640 1 -- 172.21.15.78:0/494338276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db52e8c4b0 con 0x55db52b9c4d0 2024-04-25T14:00:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85bf3db740 10 client.0 _ll_register_callbacks cb 0x7f85ac009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85b61e8640 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f77c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f85ac03b800 tx=0x7f85ac0cb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d5a158c35390e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f8599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a80105d0 con 0x55db52b9c4d0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f8599ffb640 1 -- 172.21.15.78:0/494338276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0e1e60 con 0x55db52b9c4d0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f8599ffb640 1 -- 172.21.15.78:0/494338276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f85ac03b460 con 0x55db52b9c4d0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f8599ffb640 1 -- 172.21.15.78:0/494338276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f85ac0c6ce0 con 0x55db52b9c4d0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85bf3db740 10 client.15639 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.404+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55db52fe5750 con 0x55db52b9c4d0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f8599ffb640 1 -- 172.21.15.78:0/494338276 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f85ac0cff40 con 0x55db52b9c4d0 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85bf3db740 10 client.15639 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85bf3db740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85bf3db740 20 client.15639 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85bf3db740 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b593a0 0x55db52fe7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85bf3db740 4 client.15639 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f85a80105d0 con 0x55db52b593a0 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85b59e7640 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b593a0 0x55db52fe7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f8599ffb640 10 client.15639 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.405+0000 7f85b59e7640 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b593a0 0x55db52fe7d40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ae679645717b4 server_cookie=828ac4f9beeaee11 in_seq=0 out_seq=0 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f8599ffb640 1 -- 172.21.15.78:0/494338276 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f85a80105d0 con 0x55db52b593a0 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f8599ffb640 10 client.15639 handle_command_reply: tid=0 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 client.15639 shutdown 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 msgr2=0x55db52f77c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f77c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f85ac03b800 tx=0x7f85ac0cb920 comp rx=0 tx=0).stop 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 shutdown_connections 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55db52b593a0 0x55db52fe7d40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55db52bc5390 0x55db52f74d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f8599ffb640 0 client.15639 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55db52ba46b0 0x55db52f747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 --2- 172.21.15.78:0/494338276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db52b9c4d0 0x55db52f77c30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 >> 172.21.15.78:0/494338276 conn(0x55db52bb94e0 msgr2=0x55db52f70510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 shutdown_connections 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 1 -- 172.21.15.78:0/494338276 wait complete. 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.406+0000 7f85bf3db740 20 client.15639 trim_cache size 0 max 16384 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:00:20.411 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:00:20.412 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:00:20.412 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:00:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f859c0afb70 msgr2=0x7f859c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f859c0afb70 0x7f859c0b2080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a802b000 tx=0x7f85a802c000 comp rx=0 tx=0).stop 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b01029e0 msgr2=0x7f85b01981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b01029e0 0x7f85b01981a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85ac02a930 tx=0x7f85ac0042f0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 shutdown_connections 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f859c0afb70 0x7f859c0b2080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85b0108920 0x7f85b019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b0103c20 0x7f85b01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 --2- 172.21.15.78:0/733182823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b01029e0 0x7f85b01981a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 >> 172.21.15.78:0/733182823 conn(0x7f85b00fdbd0 msgr2=0x7f85b0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 shutdown_connections 2024-04-25T14:00:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.407+0000 7f85b844c640 1 -- 172.21.15.78:0/733182823 wait complete. 2024-04-25T14:00:20.421 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:00:20.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:00:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:20 smithi078 ceph-mon[29163]: pgmap v373: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T14:00:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.538+0000 7f78e6fee640 1 -- 172.21.15.78:0/938301056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e01045b0 msgr2=0x7f78e01069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.538+0000 7f78e6fee640 1 --2- 172.21.15.78:0/938301056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e01045b0 0x7f78e01069d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009900 tx=0x7f78d402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 -- 172.21.15.78:0/938301056 shutdown_connections 2024-04-25T14:00:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 --2- 172.21.15.78:0/938301056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78e0109870 0x7f78e010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 --2- 172.21.15.78:0/938301056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78e0106f10 0x7f78e0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 --2- 172.21.15.78:0/938301056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e01045b0 0x7f78e01069d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 -- 172.21.15.78:0/938301056 >> 172.21.15.78:0/938301056 conn(0x7f78e00fdbd0 msgr2=0x7f78e0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 -- 172.21.15.78:0/938301056 shutdown_connections 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 -- 172.21.15.78:0/938301056 wait complete. 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 Processor -- start 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 -- start start 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78e01045b0 0x7f78e0103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.539+0000 7f78e6fee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78e0106f10 0x7f78e01020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e6fee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0109870 0x7f78e01025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e6fee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010f5a0 con 0x7f78e0109870 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e6fee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010f420 con 0x7f78e0106f10 2024-04-25T14:00:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e6fee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010f720 con 0x7f78e01045b0 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0109870 0x7f78e01025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e4d8a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78e01045b0 0x7f78e0103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78e0106f10 0x7f78e01020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0109870 0x7f78e01025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39234/0 (socket says 172.21.15.78:39234) 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 -- 172.21.15.78:0/335749212 learned_addr learned my addr 172.21.15.78:0/335749212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 -- 172.21.15.78:0/335749212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78e01045b0 msgr2=0x7f78e0103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78e01045b0 0x7f78e0103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 -- 172.21.15.78:0/335749212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78e0106f10 msgr2=0x7f78e01020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78e0106f10 0x7f78e01020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 -- 172.21.15.78:0/335749212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e0102d10 con 0x7f78e0109870 2024-04-25T14:00:20.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78e558b640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0109870 0x7f78e01025e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78d800eb40 tx=0x7f78d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.540+0000 7f78ddffb640 1 -- 172.21.15.78:0/335749212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8004b20 con 0x7f78e0109870 2024-04-25T14:00:20.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.541+0000 7f78ddffb640 1 -- 172.21.15.78:0/335749212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d8004cc0 con 0x7f78e0109870 2024-04-25T14:00:20.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.541+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01a42c0 con 0x7f78e0109870 2024-04-25T14:00:20.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.541+0000 7f78ddffb640 1 -- 172.21.15.78:0/335749212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8010720 con 0x7f78e0109870 2024-04-25T14:00:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.541+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01a4730 con 0x7f78e0109870 2024-04-25T14:00:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.542+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e00ff6f0 con 0x7f78e0109870 2024-04-25T14:00:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.543+0000 7f78ddffb640 1 -- 172.21.15.78:0/335749212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f78d8005560 con 0x7f78e0109870 2024-04-25T14:00:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.544+0000 7f78ddffb640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78d00afaf0 0x7f78d00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.545+0000 7f78e4d8a640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78d00afaf0 0x7f78d00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.545+0000 7f78ddffb640 1 -- 172.21.15.78:0/335749212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f78d80deff0 con 0x7f78e0109870 2024-04-25T14:00:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.545+0000 7f78e4d8a640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78d00afaf0 0x7f78d00b2000 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009230 tx=0x7f78d4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.549+0000 7f78ddffb640 1 -- 172.21.15.78:0/335749212 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78d80a7bc0 con 0x7f78e0109870 2024-04-25T14:00:20.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:20 smithi129 ceph-mon[32569]: pgmap v373: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T14:00:20.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:20 smithi112 ceph-mon[32617]: pgmap v373: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T14:00:20.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.785+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e00637a0 con 0x7f78e0109870 2024-04-25T14:00:20.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.785+0000 7f78ddffb640 1 -- 172.21.15.78:0/335749212 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f78d80a74c0 con 0x7f78e0109870 2024-04-25T14:00:20.785 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:00:20.785 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:00:20.786 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:00:20.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78d00afaf0 msgr2=0x7f78d00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78d00afaf0 0x7f78d00b2000 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009230 tx=0x7f78d4031400 comp rx=0 tx=0).stop 2024-04-25T14:00:20.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0109870 msgr2=0x7f78e01025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0109870 0x7f78e01025e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f78d800eb40 tx=0x7f78d800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 shutdown_connections 2024-04-25T14:00:20.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78d00afaf0 0x7f78d00b2000 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78e0109870 0x7f78e01025e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78e0106f10 0x7f78e01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 --2- 172.21.15.78:0/335749212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78e01045b0 0x7f78e0103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 >> 172.21.15.78:0/335749212 conn(0x7f78e00fdbd0 msgr2=0x7f78e0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 shutdown_connections 2024-04-25T14:00:20.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.788+0000 7f78e6fee640 1 -- 172.21.15.78:0/335749212 wait complete. 2024-04-25T14:00:20.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:00:20.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 -- 172.21.15.78:0/319305186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd98106f30 msgr2=0x7efd98109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/319305186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd98106f30 0x7efd98109350 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009900 tx=0x7efd8c02a410 comp rx=0 tx=0).stop 2024-04-25T14:00:20.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 -- 172.21.15.78:0/319305186 shutdown_connections 2024-04-25T14:00:20.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/319305186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd98109890 0x7efd9810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/319305186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd98106f30 0x7efd98109350 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/319305186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd981045d0 0x7efd981069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 -- 172.21.15.78:0/319305186 >> 172.21.15.78:0/319305186 conn(0x7efd980fdbd0 msgr2=0x7efd98100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 -- 172.21.15.78:0/319305186 shutdown_connections 2024-04-25T14:00:20.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 -- 172.21.15.78:0/319305186 wait complete. 2024-04-25T14:00:20.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.918+0000 7efd9d5d7640 1 Processor -- start 2024-04-25T14:00:20.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd9d5d7640 1 -- start start 2024-04-25T14:00:20.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd9d5d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd981045d0 0x7efd981983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd9d5d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd98106f30 0x7efd981988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd9d5d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd98109890 0x7efd9819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd9d5d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810d430 con 0x7efd981045d0 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd9d5d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810d2b0 con 0x7efd98109890 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd9d5d7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd9810d5b0 con 0x7efd98106f30 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd981045d0 0x7efd981983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd981045d0 0x7efd981983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39252/0 (socket says 172.21.15.78:39252) 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 -- 172.21.15.78:0/652473437 learned_addr learned my addr 172.21.15.78:0/652473437 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd967fc640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd98106f30 0x7efd981988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd977fe640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd98109890 0x7efd9819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 -- 172.21.15.78:0/652473437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd98106f30 msgr2=0x7efd981988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd98106f30 0x7efd981988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 -- 172.21.15.78:0/652473437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd98109890 msgr2=0x7efd9819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd98109890 0x7efd9819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd96ffd640 1 -- 172.21.15.78:0/652473437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9819fee0 con 0x7efd981045d0 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd977fe640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd98109890 0x7efd9819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.919+0000 7efd967fc640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd98106f30 0x7efd981988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.920+0000 7efd96ffd640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd981045d0 0x7efd981983a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efd8800e9f0 tx=0x7efd8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.920+0000 7efd7ffff640 1 -- 172.21.15.78:0/652473437 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8800ccc0 con 0x7efd981045d0 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.920+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd981a01d0 con 0x7efd981045d0 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.920+0000 7efd7ffff640 1 -- 172.21.15.78:0/652473437 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd880054d0 con 0x7efd981045d0 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.920+0000 7efd7ffff640 1 -- 172.21.15.78:0/652473437 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88010430 con 0x7efd981045d0 2024-04-25T14:00:20.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.920+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd981a0720 con 0x7efd981045d0 2024-04-25T14:00:20.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.922+0000 7efda4565740 1 Processor -- start 2024-04-25T14:00:20.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.922+0000 7efda4565740 1 -- start start 2024-04-25T14:00:20.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efda4565740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae22810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efda4565740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efda4565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efda4565740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c74ad86d80 con 0x55c74aa574d0 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efda4565740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c74ad40a20 con 0x55c74aa5f6b0 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efda4565740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c74ad82d40 con 0x55c74aa80390 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae22810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae22810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:59626/0 (socket says 172.21.15.78:59626) 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 -- 172.21.15.78:0/4222933101 learned_addr learned my addr 172.21.15.78:0/4222933101 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd967fc640 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 -- 172.21.15.78:0/4222933101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 msgr2=0x55c74ae23a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae23a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 -- 172.21.15.78:0/4222933101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 msgr2=0x55c74ae22d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae22d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.923+0000 7efd977fe640 1 -- 172.21.15.78:0/4222933101 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74ad4bc30 con 0x55c74aa80390 2024-04-25T14:00:20.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.924+0000 7efd977fe640 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae22810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd90009f60 tx=0x7efd90009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=399d53a5a948de62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.924+0000 7efd7ffff640 1 -- 172.21.15.78:0/652473437 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efd880046c0 con 0x7efd981045d0 2024-04-25T14:00:20.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.924+0000 7efd7d7fa640 1 -- 172.21.15.78:0/4222933101 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90007640 con 0x55c74aa80390 2024-04-25T14:00:20.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.924+0000 7efd7d7fa640 1 -- 172.21.15.78:0/4222933101 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9001e070 con 0x55c74aa80390 2024-04-25T14:00:20.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.924+0000 7efd7d7fa640 1 -- 172.21.15.78:0/4222933101 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90019bd0 con 0x55c74aa80390 2024-04-25T14:00:20.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.924+0000 7efd7ffff640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efd800afaf0 0x7efd800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efd7ffff640 1 -- 172.21.15.78:0/652473437 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efd880df0e0 con 0x7efd981045d0 2024-04-25T14:00:20.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efd96ffd640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efd800afaf0 0x7efd800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 -- 172.21.15.78:0/4222933101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 msgr2=0x55c74ae22810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae22810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd90009f60 tx=0x7efd90009f90 comp rx=0 tx=0).stop 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 -- 172.21.15.78:0/4222933101 shutdown_connections 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae23a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae22d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 --2- 172.21.15.78:0/4222933101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae22810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 -- 172.21.15.78:0/4222933101 >> 172.21.15.78:0/4222933101 conn(0x55c74aa744e0 msgr2=0x55c74ae1e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efd96ffd640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efd800afaf0 0x7efd800b2000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd880a3630 tx=0x7efd88090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 -- 172.21.15.78:0/4222933101 shutdown_connections 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.925+0000 7efda4565740 1 -- 172.21.15.78:0/4222933101 wait complete. 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.926+0000 7efda4565740 1 -- start start 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.926+0000 7efda4565740 1 Processor -- start 2024-04-25T14:00:20.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.926+0000 7efda4565740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.926+0000 7efda4565740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efda4565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa80390 0x55c74ae339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39264/0 (socket says 172.21.15.78:39264) 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 -- 172.21.15.78:0/2362100037 learned_addr learned my addr 172.21.15.78:0/2362100037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efda4565740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c74ae24530 con 0x55c74aa574d0 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c74ad0ddc0 con 0x55c74aa80390 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c74ae246b0 con 0x55c74aa5f6b0 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd967fc640 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efda4565740 10 client.0 _ll_register_callbacks cb 0x55c74ad29600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 -- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa5f6b0 msgr2=0x55c74ae2cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 -- 172.21.15.78:0/2362100037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa80390 msgr2=0x55c74ae339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa80390 0x55c74ae339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 -- 172.21.15.78:0/2362100037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74aa983f0 con 0x55c74aa574d0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd967fc640 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd977fe640 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c5c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efd90005c40 tx=0x7efd90009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c789aad7203ac450 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.927+0000 7efd7aff5640 1 -- 172.21.15.78:0/2362100037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90003d80 con 0x55c74aa574d0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efd7aff5640 1 -- 172.21.15.78:0/2362100037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efd90019950 con 0x55c74aa574d0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9001e070 con 0x55c74aa574d0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efd7aff5640 1 -- 172.21.15.78:0/2362100037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efd90031050 con 0x55c74aa574d0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 10 client.15654 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c74ab627d0 con 0x55c74aa574d0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efd7aff5640 1 -- 172.21.15.78:0/2362100037 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9001e070 con 0x55c74aa574d0 2024-04-25T14:00:20.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efd7aff5640 1 -- 172.21.15.78:0/2362100037 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7efd90005040 con 0x55c74aa574d0 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 10 client.15654 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 10 client.15654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 20 client.15654 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa3ef60 0x55c74ae39f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 4 client.15654 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.928+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c74ad94990 con 0x55c74aa3ef60 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.929+0000 7efd967fc640 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa3ef60 0x55c74ae39f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.929+0000 7efd7aff5640 10 client.15654 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.929+0000 7efd967fc640 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa3ef60 0x55c74ae39f70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89f83fa16a546240 server_cookie=c225d7a07c4acd88 in_seq=0 out_seq=0 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efd7aff5640 1 -- 172.21.15.78:0/2362100037 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c74ad94990 con 0x55c74aa3ef60 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efd7aff5640 10 client.15654 handle_command_reply: tid=0 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 client.15654 shutdown 2024-04-25T14:00:20.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 msgr2=0x55c74ae2c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c5c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efd90005c40 tx=0x7efd90009d50 comp rx=0 tx=0).stop 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 shutdown_connections 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa3ef60 0x55c74ae39f70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa80390 0x55c74ae339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 --2- 172.21.15.78:0/2362100037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c5c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 >> 172.21.15.78:0/2362100037 conn(0x55c74aa744e0 msgr2=0x55c74ae1d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efd7aff5640 0 client.15654 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 shutdown_connections 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 1 -- 172.21.15.78:0/2362100037 wait complete. 2024-04-25T14:00:20.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.930+0000 7efda4565740 20 client.15654 trim_cache size 0 max 16384 2024-04-25T14:00:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 Processor -- start 2024-04-25T14:00:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 -- start start 2024-04-25T14:00:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae1c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd981aba50 con 0x55c74aa574d0 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd981a9320 con 0x55c74aa5f6b0 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.957+0000 7efda4565740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd981a7170 con 0x55c74aa80390 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39278/0 (socket says 172.21.15.78:39278) 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 -- 172.21.15.78:0/2534471108 learned_addr learned my addr 172.21.15.78:0/2534471108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae1c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd967fc640 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 -- 172.21.15.78:0/2534471108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 msgr2=0x55c74ae1c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae1c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 -- 172.21.15.78:0/2534471108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 msgr2=0x55c74ae2c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 -- 172.21.15.78:0/2534471108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74ae1ebb0 con 0x55c74aa574d0 2024-04-25T14:00:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd977fe640 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efd90009a60 tx=0x7efd90000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17daccf35c203302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd7cff9640 1 -- 172.21.15.78:0/2534471108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9003c020 con 0x55c74aa574d0 2024-04-25T14:00:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd7cff9640 1 -- 172.21.15.78:0/2534471108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9001e040 con 0x55c74aa574d0 2024-04-25T14:00:20.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.958+0000 7efd7cff9640 1 -- 172.21.15.78:0/2534471108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9002b690 con 0x55c74aa574d0 2024-04-25T14:00:20.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 -- 172.21.15.78:0/2534471108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 msgr2=0x55c74ae2c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efd90009a60 tx=0x7efd90000f40 comp rx=0 tx=0).stop 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 -- 172.21.15.78:0/2534471108 shutdown_connections 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae1c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 --2- 172.21.15.78:0/2534471108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa574d0 0x55c74ae2c190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 -- 172.21.15.78:0/2534471108 >> 172.21.15.78:0/2534471108 conn(0x55c74aa744e0 msgr2=0x55c74ae27310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 -- 172.21.15.78:0/2534471108 shutdown_connections 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 -- 172.21.15.78:0/2534471108 wait complete. 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.960+0000 7efda4565740 1 -- start start 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efda4565740 1 Processor -- start 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efda4565740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa574d0 0x55c74ae32c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efda4565740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efd967fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efd967fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39286/0 (socket says 172.21.15.78:39286) 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efd967fc640 1 -- 172.21.15.78:0/1395177112 learned_addr learned my addr 172.21.15.78:0/1395177112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efda4565740 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae2fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd981a77e0 con 0x55c74aa5f6b0 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd981a8c50 con 0x55c74aa574d0 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efd977fe640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa574d0 0x55c74ae32c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.961+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd981a7eb0 con 0x55c74aa80390 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efda4565740 10 client.0 _ll_register_callbacks cb 0x7efd900093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd977fe640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae2fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd967fc640 1 -- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 msgr2=0x55c74ae2fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd967fc640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae2fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd967fc640 1 -- 172.21.15.78:0/1395177112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa574d0 msgr2=0x55c74ae32c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd967fc640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa574d0 0x55c74ae32c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd967fc640 1 -- 172.21.15.78:0/1395177112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74ae332d0 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd977fe640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae2fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd967fc640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2f7b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efd8c02a930 tx=0x7efd8c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a534e7cbd4110fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd7aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd7aff5640 1 -- 172.21.15.78:0/1395177112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c035040 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9001e040 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.962+0000 7efd7aff5640 1 -- 172.21.15.78:0/1395177112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efd8c005040 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efd7aff5640 1 -- 172.21.15.78:0/1395177112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efd8c005240 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efda4565740 10 client.15660 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c74aea11f0 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efd7aff5640 1 -- 172.21.15.78:0/1395177112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9001e040 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efd7aff5640 1 -- 172.21.15.78:0/1395177112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7efd8c045560 con 0x55c74aa5f6b0 2024-04-25T14:00:20.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efda4565740 10 client.15660 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efda4565740 10 client.15660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efda4565740 20 client.15660 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.963+0000 7efda4565740 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa143a0 0x55c74aea3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.964+0000 7efda4565740 4 client.15660 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.964+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c74ad8e800 con 0x55c74aa143a0 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.964+0000 7efd977fe640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa143a0 0x55c74aea3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.964+0000 7efd977fe640 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa143a0 0x55c74aea3c10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8690a778ebaf764 server_cookie=48f8774abcdeec7b in_seq=0 out_seq=0 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.964+0000 7efd7aff5640 10 client.15660 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efd7aff5640 1 -- 172.21.15.78:0/1395177112 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c74ad8e800 con 0x55c74aa143a0 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efd7aff5640 10 client.15660 handle_command_reply: tid=0 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 client.15660 shutdown 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa5f6b0 msgr2=0x55c74ae2f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2f7b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efd8c02a930 tx=0x7efd8c004040 comp rx=0 tx=0).stop 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 shutdown_connections 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c74aa143a0 0x55c74aea3c10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efd7aff5640 0 client.15660 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c74aa80390 0x55c74ae2fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c74aa5f6b0 0x55c74ae2f7b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 --2- 172.21.15.78:0/1395177112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c74aa574d0 0x55c74ae32c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 >> 172.21.15.78:0/1395177112 conn(0x55c74aa744e0 msgr2=0x55c74ae20e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 shutdown_connections 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 1 -- 172.21.15.78:0/1395177112 wait complete. 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.965+0000 7efda4565740 20 client.15660 trim_cache size 0 max 16384 2024-04-25T14:00:20.969 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efd800afaf0 msgr2=0x7efd800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efd800afaf0 0x7efd800b2000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd880a3630 tx=0x7efd88090040 comp rx=0 tx=0).stop 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd981045d0 msgr2=0x7efd981983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd981045d0 0x7efd981983a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efd8800e9f0 tx=0x7efd8800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 shutdown_connections 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd98109890 0x7efd9819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd98106f30 0x7efd981988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efd800afaf0 0x7efd800b2000 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 --2- 172.21.15.78:0/652473437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd981045d0 0x7efd981983a0 secure :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efd8800e9f0 tx=0x7efd8800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:00:20.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 >> 172.21.15.78:0/652473437 conn(0x7efd980fdbd0 msgr2=0x7efd98105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 shutdown_connections 2024-04-25T14:00:20.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:20.966+0000 7efd9d5d7640 1 -- 172.21.15.78:0/652473437 wait complete. 2024-04-25T14:00:20.981 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.095+0000 7f7a91096640 1 -- 172.21.15.78:0/1441894956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c104d20 msgr2=0x7f7a8c105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.095+0000 7f7a91096640 1 --2- 172.21.15.78:0/1441894956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c104d20 0x7f7a8c105130 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009900 tx=0x7f7a8002a410 comp rx=0 tx=0).stop 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.095+0000 7f7a91096640 1 -- 172.21.15.78:0/1441894956 shutdown_connections 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.095+0000 7f7a91096640 1 --2- 172.21.15.78:0/1441894956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c109a20 0x7f7a8c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.095+0000 7f7a91096640 1 --2- 172.21.15.78:0/1441894956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c104d20 0x7f7a8c105130 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.095+0000 7f7a91096640 1 --2- 172.21.15.78:0/1441894956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a8c103ae0 0x7f7a8c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.095+0000 7f7a91096640 1 -- 172.21.15.78:0/1441894956 >> 172.21.15.78:0/1441894956 conn(0x7f7a8c0fecd0 msgr2=0x7f7a8c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.096+0000 7f7a91096640 1 -- 172.21.15.78:0/1441894956 shutdown_connections 2024-04-25T14:00:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.096+0000 7f7a91096640 1 -- 172.21.15.78:0/1441894956 wait complete. 2024-04-25T14:00:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 Processor -- start 2024-04-25T14:00:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 -- start start 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c103ae0 0x7f7a8c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c109a20 0x7f7a8c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a8c19f7b0 0x7f7a8c1a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c109a20 0x7f7a8c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c109a20 0x7f7a8c19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39304/0 (socket says 172.21.15.78:39304) 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a8a59c640 1 -- 172.21.15.78:0/475235449 learned_addr learned my addr 172.21.15.78:0/475235449 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10f600 con 0x7f7a8c109a20 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10f480 con 0x7f7a8c103ae0 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.097+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10f780 con 0x7f7a8c19f7b0 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c103ae0 0x7f7a8c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a8c19f7b0 0x7f7a8c1a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8a59c640 1 -- 172.21.15.78:0/475235449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a8c19f7b0 msgr2=0x7f7a8c1a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a8c19f7b0 0x7f7a8c1a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8a59c640 1 -- 172.21.15.78:0/475235449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c103ae0 msgr2=0x7f7a8c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c103ae0 0x7f7a8c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8a59c640 1 -- 172.21.15.78:0/475235449 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a2300 con 0x7f7a8c109a20 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c109a20 0x7f7a8c19a920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a80006010 tx=0x7f7a800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a73fff640 1 -- 172.21.15.78:0/475235449 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8003f030 con 0x7f7a8c109a20 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a73fff640 1 -- 172.21.15.78:0/475235449 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a80031910 con 0x7f7a8c109a20 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a2590 con 0x7f7a8c109a20 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.098+0000 7f7a73fff640 1 -- 172.21.15.78:0/475235449 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80035070 con 0x7f7a8c109a20 2024-04-25T14:00:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.099+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a2a30 con 0x7f7a8c109a20 2024-04-25T14:00:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.101+0000 7f7a73fff640 1 -- 172.21.15.78:0/475235449 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7a80042030 con 0x7f7a8c109a20 2024-04-25T14:00:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 Processor -- start 2024-04-25T14:00:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 -- start start 2024-04-25T14:00:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a73fff640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7a740afaf0 0x7f7a740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd7583810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd7583d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7584a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561dd74e7d80 con 0x561dd71b84d0 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561dd74a1a20 con 0x561dd71c06b0 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.102+0000 7f7a98025740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561dd74e3d40 con 0x561dd71e1390 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd7583810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7a740afaf0 0x7f7a740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a73fff640 1 -- 172.21.15.78:0/475235449 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7a800fef90 con 0x7f7a8c109a20 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd7583d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd7583810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:59708/0 (socket says 172.21.15.78:59708) 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 -- 172.21.15.78:0/710990974 learned_addr learned my addr 172.21.15.78:0/710990974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7584a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 -- 172.21.15.78:0/710990974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 msgr2=0x561dd7584a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7584a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 -- 172.21.15.78:0/710990974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 msgr2=0x561dd7583d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd7583d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 -- 172.21.15.78:0/710990974 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd74acc30 con 0x561dd71e1390 2024-04-25T14:00:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7a740afaf0 0x7f7a740b2000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a84005fd0 tx=0x7f7a84014ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.103+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd7583810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009c20 tx=0x7f7a7c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5323a5f1c63a29d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.104+0000 7f7a717fa640 1 -- 172.21.15.78:0/710990974 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c004020 con 0x561dd71e1390 2024-04-25T14:00:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.104+0000 7f7a717fa640 1 -- 172.21.15.78:0/710990974 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7c0041c0 con 0x561dd71e1390 2024-04-25T14:00:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.104+0000 7f7a717fa640 1 -- 172.21.15.78:0/710990974 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c016690 con 0x561dd71e1390 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 -- 172.21.15.78:0/710990974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 msgr2=0x561dd7583810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd7583810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009c20 tx=0x7f7a7c011810 comp rx=0 tx=0).stop 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 -- 172.21.15.78:0/710990974 shutdown_connections 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7584a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd7583d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 --2- 172.21.15.78:0/710990974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd7583810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 -- 172.21.15.78:0/710990974 >> 172.21.15.78:0/710990974 conn(0x561dd71d54e0 msgr2=0x561dd757f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 -- 172.21.15.78:0/710990974 shutdown_connections 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 -- 172.21.15.78:0/710990974 wait complete. 2024-04-25T14:00:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 -- start start 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.105+0000 7f7a98025740 1 Processor -- start 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a98025740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd758d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a98025740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd758d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a98025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd75947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd758d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd758d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39324/0 (socket says 172.21.15.78:39324) 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8ad9d640 1 -- 172.21.15.78:0/1498499567 learned_addr learned my addr 172.21.15.78:0/1498499567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a98025740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561dd7585530 con 0x561dd71b84d0 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561dd746edc0 con 0x561dd71c06b0 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561dd75856b0 con 0x561dd71e1390 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd75947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a98025740 10 client.0 _ll_register_callbacks cb 0x561dd748a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd758d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8b59e640 1 -- 172.21.15.78:0/1498499567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 msgr2=0x561dd758d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd758d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8b59e640 1 -- 172.21.15.78:0/1498499567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 msgr2=0x561dd758d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd758d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.106+0000 7f7a8b59e640 1 -- 172.21.15.78:0/1498499567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd71f93f0 con 0x561dd71e1390 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd758d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd758d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd75947e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a84014fb0 tx=0x7f7a84004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c22932ef3504928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a6eff5640 1 -- 172.21.15.78:0/1498499567 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84025070 con 0x561dd71e1390 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a6eff5640 1 -- 172.21.15.78:0/1498499567 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7a840208e0 con 0x561dd71e1390 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a7c0041c0 con 0x561dd71e1390 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a6eff5640 1 -- 172.21.15.78:0/1498499567 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7a84020b20 con 0x561dd71e1390 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a98025740 10 client.25165 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561dd72c37d0 con 0x561dd71e1390 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.107+0000 7f7a6eff5640 1 -- 172.21.15.78:0/1498499567 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7c0041c0 con 0x561dd71e1390 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a6eff5640 1 -- 172.21.15.78:0/1498499567 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7a8400f040 con 0x561dd71e1390 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a98025740 10 client.25165 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a98025740 10 client.25165 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a98025740 20 client.25165 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a98025740 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd719ff60 0x561dd759b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a98025740 4 client.25165 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dd74f5990 con 0x561dd719ff60 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.108+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd719ff60 0x561dd759b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.109+0000 7f7a6eff5640 10 client.25165 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.109+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd719ff60 0x561dd759b090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1e8f3d782207610 server_cookie=152a8d22e4885b6 in_seq=0 out_seq=0 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.109+0000 7f7a6eff5640 1 -- 172.21.15.78:0/1498499567 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dd74f5990 con 0x561dd719ff60 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.109+0000 7f7a6eff5640 10 client.25165 handle_command_reply: tid=0 2024-04-25T14:00:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.109+0000 7f7a98025740 1 client.25165 shutdown 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.109+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 msgr2=0x561dd75947e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.109+0000 7f7a98025740 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd75947e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a84014fb0 tx=0x7f7a84004ab0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 shutdown_connections 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd719ff60 0x561dd759b090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71e1390 0x561dd75947e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd758d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 --2- 172.21.15.78:0/1498499567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71b84d0 0x561dd758d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 >> 172.21.15.78:0/1498499567 conn(0x561dd71d54e0 msgr2=0x561dd757e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a6eff5640 0 client.25165 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 shutdown_connections 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 1 -- 172.21.15.78:0/1498499567 wait complete. 2024-04-25T14:00:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.110+0000 7f7a98025740 20 client.25165 trim_cache size 0 max 16384 2024-04-25T14:00:21.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.123+0000 7f7a98025740 1 Processor -- start 2024-04-25T14:00:21.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.123+0000 7f7a98025740 1 -- start start 2024-04-25T14:00:21.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.123+0000 7f7a98025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7579d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.123+0000 7f7a98025740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7594700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.123+0000 7f7a98025740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd7588110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a98025740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1abf00 con 0x561dd71e1390 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a98025740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1a97d0 con 0x561dd71c06b0 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a98025740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1a7620 con 0x561dd71b84d0 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7594700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7594700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57284/0 (socket says 172.21.15.78:57284) 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8a59c640 1 -- 172.21.15.78:0/3239237073 learned_addr learned my addr 172.21.15.78:0/3239237073 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd7588110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7579d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8b59e640 1 -- 172.21.15.78:0/3239237073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 msgr2=0x561dd7579d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7579d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8b59e640 1 -- 172.21.15.78:0/3239237073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 msgr2=0x561dd7594700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7594700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8b59e640 1 -- 172.21.15.78:0/3239237073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd7594c40 con 0x561dd71e1390 2024-04-25T14:00:21.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.124+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd7588110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7a84013f70 tx=0x7f7a840098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a923cabc35198907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.125+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3239237073 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84025040 con 0x561dd71e1390 2024-04-25T14:00:21.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.125+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3239237073 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a84039070 con 0x561dd71e1390 2024-04-25T14:00:21.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.125+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3239237073 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8402a440 con 0x561dd71e1390 2024-04-25T14:00:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 -- 172.21.15.78:0/3239237073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 msgr2=0x561dd7588110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd7588110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7a84013f70 tx=0x7f7a840098c0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 -- 172.21.15.78:0/3239237073 shutdown_connections 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd7588110 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7594700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 --2- 172.21.15.78:0/3239237073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7579d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 -- 172.21.15.78:0/3239237073 >> 172.21.15.78:0/3239237073 conn(0x561dd71d54e0 msgr2=0x561dd7583790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 -- 172.21.15.78:0/3239237073 shutdown_connections 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 -- 172.21.15.78:0/3239237073 wait complete. 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 -- start start 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 Processor -- start 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7594570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.126+0000 7f7a98025740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7591160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a98025740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd75916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a98025740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1a7c90 con 0x561dd71e1390 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a98025740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1a9100 con 0x561dd71c06b0 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a98025740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1a8360 con 0x561dd71b84d0 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd75916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7591160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7594570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7591160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57290/0 (socket says 172.21.15.78:57290) 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7594570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:59728/0 (socket says 172.21.15.78:59728) 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 -- 172.21.15.78:0/4125367867 learned_addr learned my addr 172.21.15.78:0/4125367867 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a98025740 10 client.0 _ll_register_callbacks cb 0x7f7a84009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 -- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 msgr2=0x561dd7594570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7594570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 -- 172.21.15.78:0/4125367867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 msgr2=0x561dd75916a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd75916a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 -- 172.21.15.78:0/4125367867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd76013f0 con 0x561dd71c06b0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7594570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8b59e640 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd75916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a8a59c640 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7591160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005ab0 tx=0x7f7a800c7230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7a7969078a9176 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.127+0000 7f7a6eff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.128+0000 7f7a6eff5640 1 -- 172.21.15.78:0/4125367867 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80049040 con 0x561dd71c06b0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.128+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a84039070 con 0x561dd71c06b0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.128+0000 7f7a6eff5640 1 -- 172.21.15.78:0/4125367867 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7a800c76c0 con 0x561dd71c06b0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.128+0000 7f7a6eff5640 1 -- 172.21.15.78:0/4125367867 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7a8008c630 con 0x561dd71c06b0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.128+0000 7f7a98025740 10 client.25208 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.128+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561dd760ae10 con 0x561dd71c06b0 2024-04-25T14:00:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.128+0000 7f7a6eff5640 1 -- 172.21.15.78:0/4125367867 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a84039070 con 0x561dd71c06b0 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a6eff5640 1 -- 172.21.15.78:0/4125367867 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7a8004c060 con 0x561dd71c06b0 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a98025740 10 client.25208 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a98025740 10 client.25208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a98025740 20 client.25208 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a98025740 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd71753a0 0x561dd760d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a98025740 4 client.25208 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dd74ef800 con 0x561dd71753a0 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.129+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd71753a0 0x561dd760d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a6eff5640 10 client.25208 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a8ad9d640 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd71753a0 0x561dd760d7b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8913becdba5aa80d server_cookie=7f7ed02abf3140c in_seq=0 out_seq=0 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a6eff5640 1 -- 172.21.15.78:0/4125367867 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dd74ef800 con 0x561dd71753a0 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a6eff5640 10 client.25208 handle_command_reply: tid=0 2024-04-25T14:00:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 client.25208 shutdown 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 msgr2=0x561dd7591160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7591160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005ab0 tx=0x7f7a800c7230 comp rx=0 tx=0).stop 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 shutdown_connections 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561dd71e1390 0x561dd75916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561dd71c06b0 0x561dd7591160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x561dd71753a0 0x561dd760d7b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 --2- 172.21.15.78:0/4125367867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dd71b84d0 0x561dd7594570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 >> 172.21.15.78:0/4125367867 conn(0x561dd71d54e0 msgr2=0x561dd7588110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a6eff5640 0 client.25208 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 shutdown_connections 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 1 -- 172.21.15.78:0/4125367867 wait complete. 2024-04-25T14:00:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.130+0000 7f7a98025740 20 client.25208 trim_cache size 0 max 16384 2024-04-25T14:00:21.137 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:00:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7a740afaf0 msgr2=0x7f7a740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7a740afaf0 0x7f7a740b2000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a84005fd0 tx=0x7f7a84014ab0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c109a20 msgr2=0x7f7a8c19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c109a20 0x7f7a8c19a920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a80006010 tx=0x7f7a800042f0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 shutdown_connections 2024-04-25T14:00:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a8c19f7b0 0x7f7a8c1a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7a740afaf0 0x7f7a740b2000 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a8c109a20 0x7f7a8c19a920 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 --2- 172.21.15.78:0/475235449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c103ae0 0x7f7a8c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 >> 172.21.15.78:0/475235449 conn(0x7f7a8c0fecd0 msgr2=0x7f7a8c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 shutdown_connections 2024-04-25T14:00:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.131+0000 7f7a91096640 1 -- 172.21.15.78:0/475235449 wait complete. 2024-04-25T14:00:21.143 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.257+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/2331931683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 msgr2=0x7fdc58101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.257+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/2331931683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc58101bb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009900 tx=0x7fdc4c02a410 comp rx=0 tx=0).stop 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.257+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/2331931683 shutdown_connections 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.257+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/2331931683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc58102240 0x7fdc5810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.257+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/2331931683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc58101bb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.257+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/2331931683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc58105090 0x7fdc581054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.257+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/2331931683 >> 172.21.15.78:0/2331931683 conn(0x7fdc580fd340 msgr2=0x7fdc580ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.258+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/2331931683 shutdown_connections 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.258+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/2331931683 wait complete. 2024-04-25T14:00:21.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.258+0000 7fdc5dd4e640 1 Processor -- start 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc5dd4e640 1 -- start start 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc5dd4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc5819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc5dd4e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc58102240 0x7fdc5819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc5dd4e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc58105090 0x7fdc581a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc5dd4e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5810f670 con 0x7fdc58101740 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc5dd4e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5810f4f0 con 0x7fdc58102240 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc5dd4e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5810f7f0 con 0x7fdc58105090 2024-04-25T14:00:21.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc5819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc5819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39354/0 (socket says 172.21.15.78:39354) 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 -- 172.21.15.78:0/244744142 learned_addr learned my addr 172.21.15.78:0/244744142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc58102240 0x7fdc5819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc57fff640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc58105090 0x7fdc581a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 -- 172.21.15.78:0/244744142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc58105090 msgr2=0x7fdc581a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc58105090 0x7fdc581a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 -- 172.21.15.78:0/244744142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc58102240 msgr2=0x7fdc5819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc58102240 0x7fdc5819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc577fe640 1 -- 172.21.15.78:0/244744142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc581a1d90 con 0x7fdc58101740 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.259+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc58102240 0x7fdc5819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.260+0000 7fdc577fe640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc5819a250 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800d740 tx=0x7fdc4800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.260+0000 7fdc54ff9640 1 -- 172.21.15.78:0/244744142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48017070 con 0x7fdc58101740 2024-04-25T14:00:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.260+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc581a2080 con 0x7fdc58101740 2024-04-25T14:00:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.260+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc581a25f0 con 0x7fdc58101740 2024-04-25T14:00:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.260+0000 7fdc54ff9640 1 -- 172.21.15.78:0/244744142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4800de50 con 0x7fdc58101740 2024-04-25T14:00:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.260+0000 7fdc54ff9640 1 -- 172.21.15.78:0/244744142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48020060 con 0x7fdc58101740 2024-04-25T14:00:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.263+0000 7fdc64bfd740 1 Processor -- start 2024-04-25T14:00:21.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.263+0000 7fdc64bfd740 1 -- start start 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de6823e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6823ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e7b6b0 0x55de6823fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de681a2d80 con 0x55de67e734d0 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de6815ca20 con 0x55de67e7b6b0 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de6819ed40 con 0x55de67e9c390 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc54ff9640 1 -- 172.21.15.78:0/244744142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdc48003bb0 con 0x7fdc58101740 2024-04-25T14:00:21.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de6823e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de6823e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:59762/0 (socket says 172.21.15.78:59762) 2024-04-25T14:00:21.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc57fff640 1 -- 172.21.15.78:0/1529963962 learned_addr learned my addr 172.21.15.78:0/1529963962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e7b6b0 0x55de6823fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc577fe640 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6823ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc57fff640 1 -- 172.21.15.78:0/1529963962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e7b6b0 msgr2=0x55de6823fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e7b6b0 0x55de6823fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.264+0000 7fdc57fff640 1 -- 172.21.15.78:0/1529963962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 msgr2=0x55de6823ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6823ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc57fff640 1 -- 172.21.15.78:0/1529963962 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de68167c30 con 0x55de67e9c390 2024-04-25T14:00:21.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc54ff9640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc400afaf0 0x7fdc400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc577fe640 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6823ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc54ff9640 1 -- 172.21.15.78:0/244744142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdc480de4a0 con 0x7fdc58101740 2024-04-25T14:00:21.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de6823e810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdc50010370 tx=0x7fdc50010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc90c98c9900385f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc577fe640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc400afaf0 0x7fdc400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc3dffb640 1 -- 172.21.15.78:0/1529963962 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50016070 con 0x55de67e9c390 2024-04-25T14:00:21.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc3dffb640 1 -- 172.21.15.78:0/1529963962 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc50004d20 con 0x55de67e9c390 2024-04-25T14:00:21.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.265+0000 7fdc3dffb640 1 -- 172.21.15.78:0/1529963962 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50005420 con 0x55de67e9c390 2024-04-25T14:00:21.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1529963962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 msgr2=0x55de6823e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc577fe640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc400afaf0 0x7fdc400b2000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55de6823e530 tx=0x7fdc48000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de6823e810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdc50010370 tx=0x7fdc50010830 comp rx=0 tx=0).stop 2024-04-25T14:00:21.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1529963962 shutdown_connections 2024-04-25T14:00:21.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e7b6b0 0x55de6823fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6823ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1529963962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de6823e810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1529963962 >> 172.21.15.78:0/1529963962 conn(0x55de67e904e0 msgr2=0x55de6823a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1529963962 shutdown_connections 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.266+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1529963962 wait complete. 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.267+0000 7fdc64bfd740 1 -- start start 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.267+0000 7fdc64bfd740 1 Processor -- start 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6824c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6824cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e9c390 0x55de68253e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de682405d0 con 0x55de67e734d0 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de68129dc0 con 0x55de67e9c390 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de68240750 con 0x55de67e7b6b0 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc64bfd740 10 client.0 _ll_register_callbacks cb 0x55de68145600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:21.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6824c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6824cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6824c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39370/0 (socket says 172.21.15.78:39370) 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc57fff640 1 -- 172.21.15.78:0/4275388885 learned_addr learned my addr 172.21.15.78:0/4275388885 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc57fff640 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e9c390 0x55de68253e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc56ffd640 1 -- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e9c390 msgr2=0x55de68253e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e9c390 0x55de68253e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc56ffd640 1 -- 172.21.15.78:0/4275388885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 msgr2=0x55de6824c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6824c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc56ffd640 1 -- 172.21.15.78:0/4275388885 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de67eb43f0 con 0x55de67e7b6b0 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.268+0000 7fdc57fff640 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e9c390 0x55de68253e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6824cf10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c02aa00 tx=0x7fdc4c003960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e30e0369cde0a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc36ffd640 1 -- 172.21.15.78:0/4275388885 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c035070 con 0x55de67e7b6b0 2024-04-25T14:00:21.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc36ffd640 1 -- 172.21.15.78:0/4275388885 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdc4c003da0 con 0x55de67e7b6b0 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc50004d20 con 0x55de67e7b6b0 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc36ffd640 1 -- 172.21.15.78:0/4275388885 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c0315a0 con 0x55de67e7b6b0 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc64bfd740 10 client.25183 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.269+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55de67f7e7d0 con 0x55de67e7b6b0 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.270+0000 7fdc36ffd640 1 -- 172.21.15.78:0/4275388885 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc50004d20 con 0x55de67e7b6b0 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc36ffd640 1 -- 172.21.15.78:0/4275388885 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdc4c031740 con 0x55de67e7b6b0 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc64bfd740 10 client.25183 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc64bfd740 10 client.25183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc64bfd740 20 client.25183 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e5af60 0x55de6825a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc64bfd740 4 client.25183 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de681b0990 con 0x55de67e5af60 2024-04-25T14:00:21.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.271+0000 7fdc57fff640 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e5af60 0x55de6825a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.272+0000 7fdc36ffd640 10 client.25183 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:21.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.272+0000 7fdc57fff640 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e5af60 0x55de6825a680 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3375c80c256aaf8d server_cookie=b7795cdb239bf357 in_seq=0 out_seq=0 2024-04-25T14:00:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.272+0000 7fdc36ffd640 1 -- 172.21.15.78:0/4275388885 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de681b0990 con 0x55de67e5af60 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.272+0000 7fdc36ffd640 10 client.25183 handle_command_reply: tid=0 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.272+0000 7fdc64bfd740 1 client.25183 shutdown 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 msgr2=0x55de6824cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6824cf10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c02aa00 tx=0x7fdc4c003960 comp rx=0 tx=0).stop 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 shutdown_connections 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e9c390 0x55de68253e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e5af60 0x55de6825a680 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6824cf10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc36ffd640 0 client.25183 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/4275388885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e734d0 0x55de6824c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 >> 172.21.15.78:0/4275388885 conn(0x55de67e904e0 msgr2=0x55de68239040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 shutdown_connections 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 1 -- 172.21.15.78:0/4275388885 wait complete. 2024-04-25T14:00:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.273+0000 7fdc64bfd740 20 client.25183 trim_cache size 0 max 16384 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 Processor -- start 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 -- start start 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de6824b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6825ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e9c390 0x55de6825f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44010d60 con 0x55de67e9c390 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4400e630 con 0x55de67e734d0 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.287+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4400c480 con 0x55de67e7b6b0 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e9c390 0x55de6825f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e9c390 0x55de6825f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39380/0 (socket says 172.21.15.78:39380) 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 -- 172.21.15.78:0/1964187944 learned_addr learned my addr 172.21.15.78:0/1964187944 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de6824b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6825ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 -- 172.21.15.78:0/1964187944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 msgr2=0x55de6825ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6825ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 -- 172.21.15.78:0/1964187944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 msgr2=0x55de6824b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de6824b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 -- 172.21.15.78:0/1964187944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de68243ce0 con 0x55de67e9c390 2024-04-25T14:00:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.288+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e9c390 0x55de6825f3b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc5002bfd0 tx=0x7fdc50004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd9a562768f0b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.289+0000 7fdc3d7fa640 1 -- 172.21.15.78:0/1964187944 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50016040 con 0x55de67e9c390 2024-04-25T14:00:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.289+0000 7fdc3d7fa640 1 -- 172.21.15.78:0/1964187944 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc50012bc0 con 0x55de67e9c390 2024-04-25T14:00:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.289+0000 7fdc3d7fa640 1 -- 172.21.15.78:0/1964187944 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50005580 con 0x55de67e9c390 2024-04-25T14:00:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.289+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1964187944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e9c390 msgr2=0x55de6825f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e9c390 0x55de6825f3b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc5002bfd0 tx=0x7fdc50004ef0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1964187944 shutdown_connections 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e9c390 0x55de6825f3b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e7b6b0 0x55de6825ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1964187944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de6824b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1964187944 >> 172.21.15.78:0/1964187944 conn(0x55de67e904e0 msgr2=0x55de6824b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1964187944 shutdown_connections 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1964187944 wait complete. 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 -- start start 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.290+0000 7fdc64bfd740 1 Processor -- start 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de68250f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e7b6b0 0x55de68251490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc64bfd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de68252320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4400caf0 con 0x55de67e7b6b0 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4400df60 con 0x55de67e734d0 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc64bfd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4400d1c0 con 0x55de67e9c390 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc64bfd740 10 client.0 _ll_register_callbacks cb 0x7fdc4c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de68250f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de68250f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57334/0 (socket says 172.21.15.78:57334) 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc57fff640 1 -- 172.21.15.78:0/1597916715 learned_addr learned my addr 172.21.15.78:0/1597916715 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e7b6b0 0x55de68251490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de68252320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc57fff640 1 -- 172.21.15.78:0/1597916715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 msgr2=0x55de68252320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de68252320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.291+0000 7fdc57fff640 1 -- 172.21.15.78:0/1597916715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e7b6b0 msgr2=0x55de68251490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e7b6b0 0x55de68251490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc57fff640 1 -- 172.21.15.78:0/1597916715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6824bdb0 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e7b6b0 0x55de68251490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc57fff640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de68250f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005f10 tx=0x7fdc50010fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db57e9e2e41c3619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc36ffd640 1 -- 172.21.15.78:0/1597916715 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50016040 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc50012bc0 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc36ffd640 1 -- 172.21.15.78:0/1597916715 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdc50012510 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc36ffd640 1 -- 172.21.15.78:0/1597916715 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdc50012e20 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc64bfd740 10 client.25223 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.292+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55de682c2a30 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc36ffd640 1 -- 172.21.15.78:0/1597916715 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc50012bc0 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc36ffd640 1 -- 172.21.15.78:0/1597916715 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdc5000f040 con 0x55de67e734d0 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc64bfd740 10 client.25223 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:21.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc64bfd740 10 client.25223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc64bfd740 20 client.25223 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e303a0 0x55de682c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc64bfd740 4 client.25223 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de681aa800 con 0x55de67e303a0 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.293+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e303a0 0x55de682c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc36ffd640 10 client.25223 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc56ffd640 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e303a0 0x55de682c5220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7d3408add2b903 server_cookie=ceacb05a9c2cae25 in_seq=0 out_seq=0 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc36ffd640 1 -- 172.21.15.78:0/1597916715 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de681aa800 con 0x55de67e303a0 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc36ffd640 10 client.25223 handle_command_reply: tid=0 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 client.25223 shutdown 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 msgr2=0x55de68250f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de68250f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005f10 tx=0x7fdc50010fa0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 shutdown_connections 2024-04-25T14:00:21.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de67e303a0 0x55de682c5220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc36ffd640 0 client.25223 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de67e9c390 0x55de68252320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de67e7b6b0 0x55de68251490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 --2- 172.21.15.78:0/1597916715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de67e734d0 0x55de68250f50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 >> 172.21.15.78:0/1597916715 conn(0x55de67e904e0 msgr2=0x55de6823ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 shutdown_connections 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.294+0000 7fdc64bfd740 1 -- 172.21.15.78:0/1597916715 wait complete. 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc64bfd740 20 client.25223 trim_cache size 0 max 16384 2024-04-25T14:00:21.300 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:00:21.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc400afaf0 msgr2=0x7fdc400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc400afaf0 0x7fdc400b2000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55de6823e530 tx=0x7fdc48000bc0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 msgr2=0x7fdc5819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc5819a250 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800d740 tx=0x7fdc4800dc00 comp rx=0 tx=0).stop 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 shutdown_connections 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc58105090 0x7fdc581a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc58102240 0x7fdc5819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc400afaf0 0x7fdc400b2000 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 --2- 172.21.15.78:0/244744142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc58101740 0x7fdc5819a250 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.295+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 >> 172.21.15.78:0/244744142 conn(0x7fdc580fd340 msgr2=0x7fdc58105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.296+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 shutdown_connections 2024-04-25T14:00:21.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:21.296+0000 7fdc5dd4e640 1 -- 172.21.15.78:0/244744142 wait complete. 2024-04-25T14:00:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:21 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/335749212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:21.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:21 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/335749212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:21.667 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:21 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/335749212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:22.308 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:00:22.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:00:22.309 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:00:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 -- 172.21.15.78:0/3984606680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc106c60 msgr2=0x7febcc1090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 --2- 172.21.15.78:0/3984606680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc106c60 0x7febcc1090e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7febb8009900 tx=0x7febb802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 -- 172.21.15.78:0/3984606680 shutdown_connections 2024-04-25T14:00:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 --2- 172.21.15.78:0/3984606680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febcc109770 0x7febcc10bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 --2- 172.21.15.78:0/3984606680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc106c60 0x7febcc1090e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 --2- 172.21.15.78:0/3984606680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febcc104270 0x7febcc106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 -- 172.21.15.78:0/3984606680 >> 172.21.15.78:0/3984606680 conn(0x7febcc0fdbd0 msgr2=0x7febcc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 -- 172.21.15.78:0/3984606680 shutdown_connections 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.436+0000 7febd2d7d640 1 -- 172.21.15.78:0/3984606680 wait complete. 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 Processor -- start 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 -- start start 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc104270 0x7febcc19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febcc106c60 0x7febcc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febcc109770 0x7febcc1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10f600 con 0x7febcc104270 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10f480 con 0x7febcc106c60 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd2d7d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febcc10f780 con 0x7febcc109770 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd0b19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc104270 0x7febcc19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd0b19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc104270 0x7febcc19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39404/0 (socket says 172.21.15.78:39404) 2024-04-25T14:00:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.437+0000 7febd0b19640 1 -- 172.21.15.78:0/4284370962 learned_addr learned my addr 172.21.15.78:0/4284370962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febcbfff640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febcc106c60 0x7febcc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd131a640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febcc109770 0x7febcc1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd0b19640 1 -- 172.21.15.78:0/4284370962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febcc109770 msgr2=0x7febcc1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd0b19640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febcc109770 0x7febcc1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd0b19640 1 -- 172.21.15.78:0/4284370962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febcc106c60 msgr2=0x7febcc19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd0b19640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febcc106c60 0x7febcc19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd0b19640 1 -- 172.21.15.78:0/4284370962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc1a2090 con 0x7febcc104270 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd0b19640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc104270 0x7febcc19a550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7febc000e9f0 tx=0x7febc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febc9ffb640 1 -- 172.21.15.78:0/4284370962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000ccc0 con 0x7febcc104270 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc1a2380 con 0x7febcc104270 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febc9ffb640 1 -- 172.21.15.78:0/4284370962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc00054d0 con 0x7febcc104270 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febc9ffb640 1 -- 172.21.15.78:0/4284370962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0010430 con 0x7febcc104270 2024-04-25T14:00:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.438+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc1a28d0 con 0x7febcc104270 2024-04-25T14:00:22.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.441+0000 7febd9c2e740 1 Processor -- start 2024-04-25T14:00:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.442+0000 7febc9ffb640 1 -- 172.21.15.78:0/4284370962 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7febc00105d0 con 0x7febcc104270 2024-04-25T14:00:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.442+0000 7febd9c2e740 1 -- start start 2024-04-25T14:00:22.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.442+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x5588886eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.442+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888321550 0x5588886eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.442+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886f2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd9c2e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558888650b00 con 0x55888834a410 2024-04-25T14:00:22.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd9c2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5588885ec920 con 0x558888329730 2024-04-25T14:00:22.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd9c2e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55888864cac0 con 0x558888321550 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febc9ffb640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7febbc0afb70 0x7febbc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd131a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x5588886eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd131a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x5588886eb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39412/0 (socket says 172.21.15.78:39412) 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd131a640 1 -- 172.21.15.78:0/580799949 learned_addr learned my addr 172.21.15.78:0/580799949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febcbfff640 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febc9ffb640 1 -- 172.21.15.78:0/4284370962 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7febc00defe0 con 0x7febcc104270 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febcbfff640 1 -- 172.21.15.78:0/580799949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888321550 msgr2=0x5588886eba20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd0b19640 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888321550 0x5588886eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febcbfff640 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888321550 0x5588886eba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febcbfff640 1 -- 172.21.15.78:0/580799949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 msgr2=0x5588886eb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febcbfff640 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x5588886eb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febcbfff640 1 -- 172.21.15.78:0/580799949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55888842a7e0 con 0x558888329730 2024-04-25T14:00:22.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.443+0000 7febd131a640 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x5588886eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.444+0000 7febd131a640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7febbc0afb70 0x7febbc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.444+0000 7febcbfff640 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886f2920 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febcc19b5f0 tx=0x7febb8032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c414654b1fe9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.444+0000 7febb6ffd640 1 -- 172.21.15.78:0/580799949 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb803f070 con 0x558888329730 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.444+0000 7febb6ffd640 1 -- 172.21.15.78:0/580799949 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb8004040 con 0x558888329730 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.444+0000 7febb6ffd640 1 -- 172.21.15.78:0/580799949 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8035070 con 0x558888329730 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd9c2e740 1 -- 172.21.15.78:0/580799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 msgr2=0x5588886f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd9c2e740 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886f2920 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febcc19b5f0 tx=0x7febb8032b20 comp rx=0 tx=0).stop 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd131a640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7febbc0afb70 0x7febbc0b2080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5588886ec670 tx=0x7febc4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd9c2e740 1 -- 172.21.15.78:0/580799949 shutdown_connections 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd9c2e740 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886f2920 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd9c2e740 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888321550 0x5588886eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd9c2e740 1 --2- 172.21.15.78:0/580799949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x5588886eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.445+0000 7febd9c2e740 1 -- 172.21.15.78:0/580799949 >> 172.21.15.78:0/580799949 conn(0x55888833e560 msgr2=0x5588886e6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.446+0000 7febd9c2e740 1 -- 172.21.15.78:0/580799949 shutdown_connections 2024-04-25T14:00:22.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.446+0000 7febd9c2e740 1 -- 172.21.15.78:0/580799949 wait complete. 2024-04-25T14:00:22.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.446+0000 7febd9c2e740 1 -- start start 2024-04-25T14:00:22.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.446+0000 7febd9c2e740 1 Processor -- start 2024-04-25T14:00:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888329730 0x5588886f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd9c2e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588886f3310 con 0x558888329730 2024-04-25T14:00:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd9c2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5588885ea5d0 con 0x558888321550 2024-04-25T14:00:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd9c2e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5588886f3490 con 0x55888834a410 2024-04-25T14:00:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febcbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888329730 0x5588886f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febcbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888329730 0x5588886f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39428/0 (socket says 172.21.15.78:39428) 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febcbfff640 1 -- 172.21.15.78:0/131795228 learned_addr learned my addr 172.21.15.78:0/131795228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd131a640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd9c2e740 10 client.0 _ll_register_callbacks cb 0x5588885d8080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd0b19640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd131a640 1 -- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 msgr2=0x5588886f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd131a640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd131a640 1 -- 172.21.15.78:0/131795228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888329730 msgr2=0x5588886f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd131a640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888329730 0x5588886f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febd131a640 1 -- 172.21.15.78:0/131795228 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55888848af70 con 0x55888834a410 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.447+0000 7febcbfff640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888329730 0x5588886f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febd131a640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fc710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febc4013830 tx=0x7febc4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3afa3aa318908d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febabfff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febabfff640 1 -- 172.21.15.78:0/131795228 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4011d50 con 0x55888834a410 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febabfff640 1 -- 172.21.15.78:0/131795228 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7febc401e420 con 0x55888834a410 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb8004040 con 0x55888834a410 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febabfff640 1 -- 172.21.15.78:0/131795228 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7febc401e620 con 0x55888834a410 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febd9c2e740 10 client.25198 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558888322170 con 0x55888834a410 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.448+0000 7febabfff640 1 -- 172.21.15.78:0/131795228 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc4023030 con 0x55888834a410 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febabfff640 1 -- 172.21.15.78:0/131795228 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7febc4028800 con 0x55888834a410 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febd9c2e740 10 client.25198 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febd9c2e740 10 client.25198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febd9c2e740 20 client.25198 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febd9c2e740 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588883091f0 0x5588887070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febd9c2e740 4 client.25198 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7febb8004040 con 0x5588883091f0 2024-04-25T14:00:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.449+0000 7febd0b19640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588883091f0 0x5588887070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.450+0000 7febd0b19640 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588883091f0 0x5588887070e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b903389420ea02c server_cookie=24205b3a8009f6c4 in_seq=0 out_seq=0 2024-04-25T14:00:22.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.450+0000 7febabfff640 10 client.25198 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.450+0000 7febabfff640 1 -- 172.21.15.78:0/131795228 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7febb8004040 con 0x5588883091f0 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.450+0000 7febabfff640 10 client.25198 handle_command_reply: tid=0 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.450+0000 7febd9c2e740 1 client.25198 shutdown 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.450+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 msgr2=0x5588886fc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.450+0000 7febd9c2e740 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fc710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febc4013830 tx=0x7febc4004ab0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 shutdown_connections 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588883091f0 0x5588887070e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fc710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888329730 0x5588886f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 --2- 172.21.15.78:0/131795228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 >> 172.21.15.78:0/131795228 conn(0x55888833e560 msgr2=0x5588886e5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febabfff640 0 client.25198 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 shutdown_connections 2024-04-25T14:00:22.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 1 -- 172.21.15.78:0/131795228 wait complete. 2024-04-25T14:00:22.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.451+0000 7febd9c2e740 20 client.25198 trim_cache size 0 max 16384 2024-04-25T14:00:22.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 Processor -- start 2024-04-25T14:00:22.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 -- start start 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888321550 0x5588886f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1ab8f0 con 0x558888321550 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1a91c0 con 0x558888329730 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.486+0000 7febd9c2e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1a7010 con 0x55888834a410 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888321550 0x5588886f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888321550 0x5588886f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39436/0 (socket says 172.21.15.78:39436) 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 -- 172.21.15.78:0/1850767533 learned_addr learned my addr 172.21.15.78:0/1850767533 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febcbfff640 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd131a640 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 -- 172.21.15.78:0/1850767533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 msgr2=0x5588886fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 -- 172.21.15.78:0/1850767533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 msgr2=0x5588886fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 -- 172.21.15.78:0/1850767533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588886f0180 con 0x558888321550 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febcbfff640 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febd0b19640 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888321550 0x5588886f3f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7febc00a7250 tx=0x7febc0090040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d0ccee5b3cc6b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.487+0000 7febb67fc640 1 -- 172.21.15.78:0/1850767533 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00a7c50 con 0x558888321550 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febb67fc640 1 -- 172.21.15.78:0/1850767533 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc00a7550 con 0x558888321550 2024-04-25T14:00:22.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febb67fc640 1 -- 172.21.15.78:0/1850767533 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00aff20 con 0x558888321550 2024-04-25T14:00:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febd9c2e740 1 -- 172.21.15.78:0/1850767533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888321550 msgr2=0x5588886f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febd9c2e740 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888321550 0x5588886f3f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7febc00a7250 tx=0x7febc0090040 comp rx=0 tx=0).stop 2024-04-25T14:00:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febd9c2e740 1 -- 172.21.15.78:0/1850767533 shutdown_connections 2024-04-25T14:00:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febd9c2e740 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55888834a410 0x5588886fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febd9c2e740 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888329730 0x5588886fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febd9c2e740 1 --2- 172.21.15.78:0/1850767533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558888321550 0x5588886f3f40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.488+0000 7febd9c2e740 1 -- 172.21.15.78:0/1850767533 >> 172.21.15.78:0/1850767533 conn(0x55888833e560 msgr2=0x5588886f3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 -- 172.21.15.78:0/1850767533 shutdown_connections 2024-04-25T14:00:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 -- 172.21.15.78:0/1850767533 wait complete. 2024-04-25T14:00:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 -- start start 2024-04-25T14:00:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 Processor -- start 2024-04-25T14:00:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888329730 0x5588886fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x55888876d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1a7680 con 0x55888834a410 2024-04-25T14:00:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1a8af0 con 0x558888321550 2024-04-25T14:00:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.489+0000 7febd9c2e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1a7d50 con 0x558888329730 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd9c2e740 10 client.0 _ll_register_callbacks cb 0x7febc4009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x55888876d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x55888876d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39442/0 (socket says 172.21.15.78:39442) 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 -- 172.21.15.78:0/3750562931 learned_addr learned my addr 172.21.15.78:0/3750562931 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febcbfff640 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888329730 0x5588886fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd0b19640 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 -- 172.21.15.78:0/3750562931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888329730 msgr2=0x5588886fa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888329730 0x5588886fa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 -- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 msgr2=0x5588886f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 -- 172.21.15.78:0/3750562931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55888876d980 con 0x55888834a410 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febd131a640 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x55888876d440 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febc4028e10 tx=0x7febc401cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c92f14e13f66229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febabfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febabfff640 1 -- 172.21.15.78:0/3750562931 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4037070 con 0x55888834a410 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.490+0000 7febabfff640 1 -- 172.21.15.78:0/3750562931 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7febc4004d30 con 0x55888834a410 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc00a7550 con 0x55888834a410 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febabfff640 1 -- 172.21.15.78:0/3750562931 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7febc4043050 con 0x55888834a410 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 10 client.15714 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55888876eb10 con 0x55888834a410 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febabfff640 1 -- 172.21.15.78:0/3750562931 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc00a7550 con 0x55888834a410 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febabfff640 1 -- 172.21.15.78:0/3750562931 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7febc401e8e0 con 0x55888834a410 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 10 client.15714 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 20 client.15714 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588882de630 0x558888771510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 4 client.15714 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.491+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55888865e710 con 0x5588882de630 2024-04-25T14:00:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.492+0000 7febd0b19640 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588882de630 0x558888771510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.492+0000 7febabfff640 10 client.15714 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.492+0000 7febd0b19640 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588882de630 0x558888771510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d890806e8b811a12 server_cookie=7f8deae3fe3a5cfd in_seq=0 out_seq=0 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.493+0000 7febabfff640 1 -- 172.21.15.78:0/3750562931 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55888865e710 con 0x5588882de630 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.493+0000 7febabfff640 10 client.15714 handle_command_reply: tid=0 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.493+0000 7febd9c2e740 1 client.15714 shutdown 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.493+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 msgr2=0x55888876d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.493+0000 7febd9c2e740 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x55888876d440 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febc4028e10 tx=0x7febc401cf80 comp rx=0 tx=0).stop 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 shutdown_connections 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55888834a410 0x55888876d440 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5588882de630 0x558888771510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febabfff640 0 client.15714 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558888329730 0x5588886fa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 --2- 172.21.15.78:0/3750562931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558888321550 0x5588886f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 >> 172.21.15.78:0/3750562931 conn(0x55888833e560 msgr2=0x5588886e9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 shutdown_connections 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 1 -- 172.21.15.78:0/3750562931 wait complete. 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.494+0000 7febd9c2e740 20 client.15714 trim_cache size 0 max 16384 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "ac043da7-fcf9-4b3b-abd9-051272d19156", 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:00:22.506 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:00:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7febbc0afb70 msgr2=0x7febbc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7febbc0afb70 0x7febbc0b2080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5588886ec670 tx=0x7febc4004a40 comp rx=0 tx=0).stop 2024-04-25T14:00:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc104270 msgr2=0x7febcc19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc104270 0x7febcc19a550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7febc000e9f0 tx=0x7febc000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 shutdown_connections 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febcc109770 0x7febcc1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febcc106c60 0x7febcc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7febbc0afb70 0x7febbc0b2080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 --2- 172.21.15.78:0/4284370962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febcc104270 0x7febcc19a550 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 >> 172.21.15.78:0/4284370962 conn(0x7febcc0fdbd0 msgr2=0x7febcc1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 shutdown_connections 2024-04-25T14:00:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.495+0000 7febd2d7d640 1 -- 172.21.15.78:0/4284370962 wait complete. 2024-04-25T14:00:22.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:22.509 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:22.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:22 smithi078 ceph-mon[29163]: pgmap v374: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:22.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 -- 172.21.15.78:0/2004433975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558102a50 msgr2=0x7fa558103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 --2- 172.21.15.78:0/2004433975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558102a50 0x7fa558103750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa554009900 tx=0x7fa55402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:22.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 -- 172.21.15.78:0/2004433975 shutdown_connections 2024-04-25T14:00:22.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 --2- 172.21.15.78:0/2004433975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa558108990 0x7fa55810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 --2- 172.21.15.78:0/2004433975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa558103c90 0x7fa5581040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 --2- 172.21.15.78:0/2004433975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558102a50 0x7fa558103750 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 -- 172.21.15.78:0/2004433975 >> 172.21.15.78:0/2004433975 conn(0x7fa5580fdc40 msgr2=0x7fa5581000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 -- 172.21.15.78:0/2004433975 shutdown_connections 2024-04-25T14:00:22.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.624+0000 7fa560034640 1 -- 172.21.15.78:0/2004433975 wait complete. 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 Processor -- start 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 -- start start 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa558102a50 0x7fa558075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa558103c90 0x7fa55819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558108990 0x7fa5581a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810e4c0 con 0x7fa558108990 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810e340 con 0x7fa558103c90 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa560034640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810e640 con 0x7fa558102a50 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa55e5d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558108990 0x7fa5581a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa55e5d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558108990 0x7fa5581a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39460/0 (socket says 172.21.15.78:39460) 2024-04-25T14:00:22.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa55e5d1640 1 -- 172.21.15.78:0/353188794 learned_addr learned my addr 172.21.15.78:0/353188794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa558102a50 0x7fa558075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.625+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa558103c90 0x7fa55819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa55e5d1640 1 -- 172.21.15.78:0/353188794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa558102a50 msgr2=0x7fa558075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa55e5d1640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa558102a50 0x7fa558075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa55e5d1640 1 -- 172.21.15.78:0/353188794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa558103c90 msgr2=0x7fa55819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa55e5d1640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa558103c90 0x7fa55819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa55e5d1640 1 -- 172.21.15.78:0/353188794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5581a1d50 con 0x7fa558108990 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa558103c90 0x7fa55819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa55e5d1640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558108990 0x7fa5581a1530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa55000eb10 tx=0x7fa55000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa54effd640 1 -- 172.21.15.78:0/353188794 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004b20 con 0x7fa558108990 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5581a1fe0 con 0x7fa558108990 2024-04-25T14:00:22.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5581a24a0 con 0x7fa558108990 2024-04-25T14:00:22.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa54effd640 1 -- 172.21.15.78:0/353188794 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa550004cc0 con 0x7fa558108990 2024-04-25T14:00:22.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.626+0000 7fa54effd640 1 -- 172.21.15.78:0/353188794 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550010670 con 0x7fa558108990 2024-04-25T14:00:22.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.629+0000 7fa566fc4740 1 Processor -- start 2024-04-25T14:00:22.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.630+0000 7fa54effd640 1 -- 172.21.15.78:0/353188794 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa550005020 con 0x7fa558108990 2024-04-25T14:00:22.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.630+0000 7fa566fc4740 1 -- start start 2024-04-25T14:00:22.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b29a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b29fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7847676b0 0x55d784b2ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa566fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d784a8ed80 con 0x55d784788390 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa566fc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d784a488b0 con 0x55d78475f4d0 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa566fc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d784a8ad40 con 0x55d7847676b0 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b29a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b29a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39462/0 (socket says 172.21.15.78:39462) 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1059042504 learned_addr learned my addr 172.21.15.78:0/1059042504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa54effd640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa5440afb70 0x7fa5440b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b29fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7847676b0 0x55d784b2ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1059042504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7847676b0 msgr2=0x55d784b2ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7847676b0 0x55d784b2ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1059042504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 msgr2=0x55d784b29fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b29fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1059042504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d784a19a40 con 0x55d784788390 2024-04-25T14:00:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.631+0000 7fa54effd640 1 -- 172.21.15.78:0/353188794 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa5500df040 con 0x7fa558108990 2024-04-25T14:00:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.632+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b29a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa548012980 tx=0x7fa54800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9b189eb66fe29a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.632+0000 7fa55e5d1640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa5440afb70 0x7fa5440b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.632+0000 7fa543fff640 1 -- 172.21.15.78:0/1059042504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548010af0 con 0x55d784788390 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.632+0000 7fa543fff640 1 -- 172.21.15.78:0/1059042504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548010c90 con 0x55d784788390 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.632+0000 7fa543fff640 1 -- 172.21.15.78:0/1059042504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54801c5f0 con 0x55d784788390 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa566fc4740 1 -- 172.21.15.78:0/1059042504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 msgr2=0x55d784b29a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b29a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa548012980 tx=0x7fa54800bcb0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa55e5d1640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa5440afb70 0x7fa5440b2080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa5500a7210 tx=0x7fa55000cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa566fc4740 1 -- 172.21.15.78:0/1059042504 shutdown_connections 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7847676b0 0x55d784b2ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b29fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1059042504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b29a60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.633+0000 7fa566fc4740 1 -- 172.21.15.78:0/1059042504 >> 172.21.15.78:0/1059042504 conn(0x55d78477c4e0 msgr2=0x55d784b25270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 -- 172.21.15.78:0/1059042504 shutdown_connections 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 -- 172.21.15.78:0/1059042504 wait complete. 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 -- start start 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 Processor -- start 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78475f4d0 0x55d784b37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7847676b0 0x55d784b38200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b3f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d784b2b780 con 0x55d784788390 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.634+0000 7fa566fc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d784a15d10 con 0x55d7847676b0 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa566fc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d784b2b900 con 0x55d78475f4d0 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa566fc4740 10 client.0 _ll_register_callbacks cb 0x55d784a42210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b3f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b3f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39478/0 (socket says 172.21.15.78:39478) 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 -- 172.21.15.78:0/303860830 learned_addr learned my addr 172.21.15.78:0/303860830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78475f4d0 0x55d784b37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7847676b0 0x55d784b38200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 -- 172.21.15.78:0/303860830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78475f4d0 msgr2=0x55d784b37cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78475f4d0 0x55d784b37cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 -- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7847676b0 msgr2=0x55d784b38200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7847676b0 0x55d784b38200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 -- 172.21.15.78:0/303860830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78486a7d0 con 0x55d784788390 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b3f160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa548012710 tx=0x7fa5480127d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f333f3076264e245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.635+0000 7fa5417fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa5417fa640 1 -- 172.21.15.78:0/303860830 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548004020 con 0x55d784788390 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa5417fa640 1 -- 172.21.15.78:0/303860830 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa5480041c0 con 0x55d784788390 2024-04-25T14:00:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa548010c90 con 0x55d784788390 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa5417fa640 1 -- 172.21.15.78:0/303860830 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa5480043c0 con 0x55d784788390 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 10 client.15726 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d7847a03f0 con 0x55d784788390 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa5417fa640 1 -- 172.21.15.78:0/303860830 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548010c90 con 0x55d784788390 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa5417fa640 1 -- 172.21.15.78:0/303860830 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa548029040 con 0x55d784788390 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 10 client.15726 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 20 client.15726 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d784746f60 0x55d784b45a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 4 client.15726 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.636+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d784a9c990 con 0x55d784746f60 2024-04-25T14:00:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.637+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d784746f60 0x55d784b45a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.637+0000 7fa5417fa640 10 client.15726 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.637+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d784746f60 0x55d784b45a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ece81d3f429474 server_cookie=7445f4b168714f52 in_seq=0 out_seq=0 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa5417fa640 1 -- 172.21.15.78:0/303860830 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d784a9c990 con 0x55d784746f60 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa5417fa640 10 client.15726 handle_command_reply: tid=0 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 client.15726 shutdown 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 msgr2=0x55d784b3f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b3f160 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa548012710 tx=0x7fa5480127d0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 shutdown_connections 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d784788390 0x55d784b3f160 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7847676b0 0x55d784b38200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d784746f60 0x55d784b45a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 --2- 172.21.15.78:0/303860830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78475f4d0 0x55d784b37cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa5417fa640 0 client.15726 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 >> 172.21.15.78:0/303860830 conn(0x55d78477c4e0 msgr2=0x55d784b243f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 shutdown_connections 2024-04-25T14:00:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 1 -- 172.21.15.78:0/303860830 wait complete. 2024-04-25T14:00:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.638+0000 7fa566fc4740 20 client.15726 trim_cache size 0 max 16384 2024-04-25T14:00:22.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.656+0000 7fa566fc4740 1 Processor -- start 2024-04-25T14:00:22.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.656+0000 7fa566fc4740 1 -- start start 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78477c4e0 0x55d784b24ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b2e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7847676b0 0x55d784b2eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa566fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c010d70 con 0x55d7847676b0 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa566fc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00e640 con 0x55d78475f4d0 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa566fc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00c490 con 0x55d78477c4e0 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7847676b0 0x55d784b2eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7847676b0 0x55d784b2eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39492/0 (socket says 172.21.15.78:39492) 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1190033049 learned_addr learned my addr 172.21.15.78:0/1190033049 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78477c4e0 0x55d784b24ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b2e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1190033049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78477c4e0 msgr2=0x55d784b24ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78477c4e0 0x55d784b24ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1190033049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 msgr2=0x55d784b2e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b2e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 -- 172.21.15.78:0/1190033049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d784b2a100 con 0x55d7847676b0 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.657+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7847676b0 0x55d784b2eae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa548010990 tx=0x7fa54800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c3f51d8f14a6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.658+0000 7fa5437fe640 1 -- 172.21.15.78:0/1190033049 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548005660 con 0x55d7847676b0 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.658+0000 7fa5437fe640 1 -- 172.21.15.78:0/1190033049 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548004710 con 0x55d7847676b0 2024-04-25T14:00:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.658+0000 7fa5437fe640 1 -- 172.21.15.78:0/1190033049 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54801d890 con 0x55d7847676b0 2024-04-25T14:00:22.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 -- 172.21.15.78:0/1190033049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7847676b0 msgr2=0x55d784b2eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7847676b0 0x55d784b2eae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa548010990 tx=0x7fa54800b410 comp rx=0 tx=0).stop 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 -- 172.21.15.78:0/1190033049 shutdown_connections 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7847676b0 0x55d784b2eae0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78475f4d0 0x55d784b2e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 --2- 172.21.15.78:0/1190033049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d78477c4e0 0x55d784b24ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 -- 172.21.15.78:0/1190033049 >> 172.21.15.78:0/1190033049 conn(0x55d784b25380 msgr2=0x55d784b36380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 -- 172.21.15.78:0/1190033049 shutdown_connections 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.659+0000 7fa566fc4740 1 -- 172.21.15.78:0/1190033049 wait complete. 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 -- start start 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 Processor -- start 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78475f4d0 0x55d784b3c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78477c4e0 0x55d784b3ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d784788390 0x55d784babe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00cb00 con 0x55d78475f4d0 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00df70 con 0x55d78477c4e0 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa566fc4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00d1d0 con 0x55d784788390 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78475f4d0 0x55d784b3c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa55ddd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78477c4e0 0x55d784b3ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa55ddd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78477c4e0 0x55d784b3ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57464/0 (socket says 172.21.15.78:57464) 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa55d5cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78475f4d0 0x55d784b3c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39508/0 (socket says 172.21.15.78:39508) 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.660+0000 7fa55ddd0640 1 -- 172.21.15.78:0/264452975 learned_addr learned my addr 172.21.15.78:0/264452975 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa566fc4740 10 client.0 _ll_register_callbacks cb 0x7fa548012860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa55ddd0640 1 -- 172.21.15.78:0/264452975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d784788390 msgr2=0x55d784babe20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d784788390 0x55d784babe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa55ddd0640 1 -- 172.21.15.78:0/264452975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78475f4d0 msgr2=0x55d784b3c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78475f4d0 0x55d784b3c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa55ddd0640 1 -- 172.21.15.78:0/264452975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7847600f0 con 0x55d78477c4e0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78475f4d0 0x55d784b3c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa55ddd0640 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78477c4e0 0x55d784b3ca90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa554009900 tx=0x7fa554031910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b75fbb64fe98413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa5417fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.661+0000 7fa5417fa640 1 -- 172.21.15.78:0/264452975 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554035040 con 0x55d78477c4e0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa548004710 con 0x55d78477c4e0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa5417fa640 1 -- 172.21.15.78:0/264452975 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa554004450 con 0x55d78477c4e0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa5417fa640 1 -- 172.21.15.78:0/264452975 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa554045310 con 0x55d78477c4e0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 10 client.25259 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d784a4f4b0 con 0x55d78477c4e0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa5417fa640 1 -- 172.21.15.78:0/264452975 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548004710 con 0x55d78477c4e0 2024-04-25T14:00:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa5417fa640 1 -- 172.21.15.78:0/264452975 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa554003bc0 con 0x55d78477c4e0 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 10 client.25259 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 10 client.25259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 20 client.25259 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d78471c3a0 0x55d784baf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 4 client.25259 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.662+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d784a96800 con 0x55d78471c3a0 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.663+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d78471c3a0 0x55d784baf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.663+0000 7fa5417fa640 10 client.25259 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.663+0000 7fa55d5cf640 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d78471c3a0 0x55d784baf8c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ca752f287fa0c2 server_cookie=d43cad1be622c849 in_seq=0 out_seq=0 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.663+0000 7fa5417fa640 1 -- 172.21.15.78:0/264452975 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d784a96800 con 0x55d78471c3a0 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.663+0000 7fa5417fa640 10 client.25259 handle_command_reply: tid=0 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.663+0000 7fa566fc4740 1 client.25259 shutdown 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78477c4e0 msgr2=0x55d784b3ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78477c4e0 0x55d784b3ca90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa554009900 tx=0x7fa554031910 comp rx=0 tx=0).stop 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 shutdown_connections 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d78471c3a0 0x55d784baf8c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa5417fa640 0 client.25259 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d784788390 0x55d784babe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d78477c4e0 0x55d784b3ca90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 --2- 172.21.15.78:0/264452975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d78475f4d0 0x55d784b3c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 >> 172.21.15.78:0/264452975 conn(0x55d784b25380 msgr2=0x55d784b2e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 shutdown_connections 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 1 -- 172.21.15.78:0/264452975 wait complete. 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa566fc4740 20 client.25259 trim_cache size 0 max 16384 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stdout: "ac043da7-fcf9-4b3b-abd9-051272d19156": { 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "ac043da7-fcf9-4b3b-abd9-051272d19156", 2024-04-25T14:00:22.669 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:00:22.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:00:22.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:00:22.670 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa5440afb70 msgr2=0x7fa5440b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa560034640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa5440afb70 0x7fa5440b2080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa5500a7210 tx=0x7fa55000cf70 comp rx=0 tx=0).stop 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558108990 msgr2=0x7fa5581a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.664+0000 7fa560034640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558108990 0x7fa5581a1530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa55000eb10 tx=0x7fa55000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 shutdown_connections 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa5440afb70 0x7fa5440b2080 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558108990 0x7fa5581a1530 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa558103c90 0x7fa55819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 --2- 172.21.15.78:0/353188794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa558102a50 0x7fa558075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 >> 172.21.15.78:0/353188794 conn(0x7fa5580fdc40 msgr2=0x7fa558104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 shutdown_connections 2024-04-25T14:00:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:22.665+0000 7fa560034640 1 -- 172.21.15.78:0/353188794 wait complete. 2024-04-25T14:00:22.672 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:22 smithi129 ceph-mon[32569]: pgmap v374: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:22.676 INFO:tasks.cephfs.filesystem:scrub status for tag:ac043da7-fcf9-4b3b-abd9-051272d19156 - {'path': '/', 'tag': 'ac043da7-fcf9-4b3b-abd9-051272d19156', 'options': 'recursive,force'} 2024-04-25T14:00:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:22 smithi112 ceph-mon[32617]: pgmap v374: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 -- 172.21.15.78:0/1812501454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102d80 msgr2=0x7f2a44103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/1812501454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102d80 0x7f2a44103190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009930 tx=0x7f2a3002a420 comp rx=0 tx=0).stop 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 -- 172.21.15.78:0/1812501454 shutdown_connections 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/1812501454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a441036d0 0x7f2a44103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/1812501454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102d80 0x7f2a44103190 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/1812501454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a44102430 0x7f2a44102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 -- 172.21.15.78:0/1812501454 >> 172.21.15.78:0/1812501454 conn(0x7f2a440fdc60 msgr2=0x7f2a441000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.287+0000 7f2a4acbd640 1 -- 172.21.15.78:0/1812501454 shutdown_connections 2024-04-25T14:00:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.288+0000 7f2a4acbd640 1 -- 172.21.15.78:0/1812501454 wait complete. 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.288+0000 7f2a4acbd640 1 Processor -- start 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.288+0000 7f2a4acbd640 1 -- start start 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4acbd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102430 0x7f2a4419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4acbd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44102d80 0x7f2a4419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4acbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a441036d0 0x7f2a441a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4acbd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441118c0 con 0x7f2a44102430 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4acbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44111740 con 0x7f2a441036d0 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4acbd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44111a40 con 0x7f2a44102d80 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102430 0x7f2a4419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102430 0x7f2a4419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39534/0 (socket says 172.21.15.78:39534) 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 -- 172.21.15.78:0/3473394934 learned_addr learned my addr 172.21.15.78:0/3473394934 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a43fff640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44102d80 0x7f2a4419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4925a640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a441036d0 0x7f2a441a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 -- 172.21.15.78:0/3473394934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44102d80 msgr2=0x7f2a4419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44102d80 0x7f2a4419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 -- 172.21.15.78:0/3473394934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a441036d0 msgr2=0x7f2a441a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a441036d0 0x7f2a441a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 -- 172.21.15.78:0/3473394934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a441a3fd0 con 0x7f2a44102430 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a4925a640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a441036d0 0x7f2a441a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.289+0000 7f2a48a59640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102430 0x7f2a4419c490 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800e9c0 tx=0x7f2a3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.290+0000 7f2a41ffb640 1 -- 172.21.15.78:0/3473394934 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800cd80 con 0x7f2a44102430 2024-04-25T14:00:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.290+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a441a42c0 con 0x7f2a44102430 2024-04-25T14:00:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.290+0000 7f2a41ffb640 1 -- 172.21.15.78:0/3473394934 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a380054d0 con 0x7f2a44102430 2024-04-25T14:00:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.290+0000 7f2a41ffb640 1 -- 172.21.15.78:0/3473394934 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38010430 con 0x7f2a44102430 2024-04-25T14:00:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.291+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a441a4860 con 0x7f2a44102430 2024-04-25T14:00:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 Processor -- start 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 -- start start 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001fa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001faef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c000201df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c000160460 con 0x55bfffe394d0 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c00011a590 con 0x55bfffe5a1b0 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a51b73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c00015c420 con 0x55bfffe312f0 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.294+0000 7f2a41ffb640 1 -- 172.21.15.78:0/3473394934 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2a38004fd0 con 0x7f2a44102430 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c000201df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001fa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57494/0 (socket says 172.21.15.78:57494) 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 -- 172.21.15.78:0/2855972228 learned_addr learned my addr 172.21.15.78:0/2855972228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a48a59640 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001faef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 -- 172.21.15.78:0/2855972228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 msgr2=0x55c0001faef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001faef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 -- 172.21.15.78:0/2855972228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 msgr2=0x55c000201df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c000201df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 -- 172.21.15.78:0/2855972228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0001270d0 con 0x55bfffe5a1b0 2024-04-25T14:00:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a43fff640 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c000201df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.295+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001fa970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009f60 tx=0x7f2a3c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6acbef873de4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.296+0000 7f2a2effd640 1 -- 172.21.15.78:0/2855972228 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c016070 con 0x55bfffe5a1b0 2024-04-25T14:00:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.296+0000 7f2a2effd640 1 -- 172.21.15.78:0/2855972228 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a3c01bb10 con 0x55bfffe5a1b0 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.296+0000 7f2a41ffb640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2a340afb60 0x7f2a340b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.296+0000 7f2a2effd640 1 -- 172.21.15.78:0/2855972228 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c012c10 con 0x55bfffe5a1b0 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.296+0000 7f2a48a59640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2a340afb60 0x7f2a340b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a41ffb640 1 -- 172.21.15.78:0/3473394934 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2a380df1d0 con 0x7f2a44102430 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 -- 172.21.15.78:0/2855972228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 msgr2=0x55c0001fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a48a59640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2a340afb60 0x7f2a340b2070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a380a75d0 tx=0x7f2a380e26d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001fa970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009f60 tx=0x7f2a3c009f90 comp rx=0 tx=0).stop 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 -- 172.21.15.78:0/2855972228 shutdown_connections 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c000201df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001faef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2855972228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001fa970 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 -- 172.21.15.78:0/2855972228 >> 172.21.15.78:0/2855972228 conn(0x55bfffe4e300 msgr2=0x55c0001f6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 -- 172.21.15.78:0/2855972228 shutdown_connections 2024-04-25T14:00:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.297+0000 7f2a51b73740 1 -- 172.21.15.78:0/2855972228 wait complete. 2024-04-25T14:00:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.298+0000 7f2a51b73740 1 -- start start 2024-04-25T14:00:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.298+0000 7f2a51b73740 1 Processor -- start 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe312f0 0x55c000204680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000204bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe5a1b0 0x55c00020bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a4925a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe312f0 0x55c000204680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a4925a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe312f0 0x55c000204680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:57504/0 (socket says 172.21.15.78:57504) 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a4925a640 1 -- 172.21.15.78:0/661569333 learned_addr learned my addr 172.21.15.78:0/661569333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c000202840 con 0x55bfffe5a1b0 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c0000e7b60 con 0x55bfffe312f0 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a43fff640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000204bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.299+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c0002029c0 con 0x55bfffe394d0 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a51b73740 10 client.0 _ll_register_callbacks cb 0x55c00010c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a4925a640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe5a1b0 0x55c00020bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a43fff640 1 -- 172.21.15.78:0/661569333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe312f0 msgr2=0x55c000204680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a43fff640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe312f0 0x55c000204680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a43fff640 1 -- 172.21.15.78:0/661569333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe5a1b0 msgr2=0x55c00020bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a43fff640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe5a1b0 0x55c00020bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a43fff640 1 -- 172.21.15.78:0/661569333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfffe72210 con 0x55bfffe394d0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a4925a640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe5a1b0 0x55c00020bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a43fff640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000204bc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002a940 tx=0x7f2a30005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3321b3ee86d1c630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a3c01bb10 con 0x55bfffe394d0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a23fff640 10 client.25231 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.300+0000 7f2a23fff640 1 -- 172.21.15.78:0/661569333 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30035040 con 0x55bfffe394d0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.301+0000 7f2a23fff640 1 -- 172.21.15.78:0/661569333 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2a300048c0 con 0x55bfffe394d0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.301+0000 7f2a23fff640 1 -- 172.21.15.78:0/661569333 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2a30004ac0 con 0x55bfffe394d0 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.301+0000 7f2a51b73740 10 client.25231 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.301+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55bffff3e620 con 0x55bfffe394d0 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.301+0000 7f2a23fff640 1 -- 172.21.15.78:0/661569333 <== mon.1 v2:172.21.15.129:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2a30004c60 con 0x55bfffe394d0 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.302+0000 7f2a51b73740 10 client.25231 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.302+0000 7f2a51b73740 10 client.25231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.302+0000 7f2a51b73740 20 client.25231 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.302+0000 7f2a51b73740 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffe18d80 0x55c000212760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.302+0000 7f2a51b73740 4 client.25231 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.302+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2a3c01bb10 con 0x55bfffe18d80 2024-04-25T14:00:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.302+0000 7f2a4925a640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffe18d80 0x55c000212760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.303+0000 7f2a23fff640 10 client.25231 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.303+0000 7f2a4925a640 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffe18d80 0x55c000212760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714bc73706da775b server_cookie=eeb5fc24a0a5570b in_seq=0 out_seq=0 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.303+0000 7f2a23fff640 1 -- 172.21.15.78:0/661569333 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2a3c01bb10 con 0x55bfffe18d80 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a23fff640 10 client.25231 handle_command_reply: tid=0 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 client.25231 shutdown 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 msgr2=0x55c000204bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000204bc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002a940 tx=0x7f2a30005270 comp rx=0 tx=0).stop 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 shutdown_connections 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe5a1b0 0x55c00020bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffe18d80 0x55c000212760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000204bc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 --2- 172.21.15.78:0/661569333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe312f0 0x55c000204680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 >> 172.21.15.78:0/661569333 conn(0x55bfffe4e300 msgr2=0x55c0001f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.304+0000 7f2a23fff640 0 client.25231 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.305+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 shutdown_connections 2024-04-25T14:00:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.305+0000 7f2a51b73740 1 -- 172.21.15.78:0/661569333 wait complete. 2024-04-25T14:00:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.305+0000 7f2a51b73740 20 client.25231 trim_cache size 0 max 16384 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.334+0000 7f2a51b73740 1 Processor -- start 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.334+0000 7f2a51b73740 1 -- start start 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c00020ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001f6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a51b73740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441aff30 con 0x55bfffe394d0 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a51b73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441ad800 con 0x55bfffe5a1b0 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a51b73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441ab650 con 0x55bfffe312f0 2024-04-25T14:00:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c00020ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c00020ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39568/0 (socket says 172.21.15.78:39568) 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 -- 172.21.15.78:0/2561838561 learned_addr learned my addr 172.21.15.78:0/2561838561 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 -- 172.21.15.78:0/2561838561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 msgr2=0x55c0001f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 -- 172.21.15.78:0/2561838561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 msgr2=0x55c0001f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.335+0000 7f2a43fff640 1 -- 172.21.15.78:0/2561838561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c00020bf80 con 0x55bfffe394d0 2024-04-25T14:00:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.336+0000 7f2a43fff640 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c00020ba40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2a30005670 tx=0x7f2a3003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ede20c09702192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.336+0000 7f2a2e7fc640 1 -- 172.21.15.78:0/2561838561 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30035070 con 0x55bfffe394d0 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.336+0000 7f2a2e7fc640 1 -- 172.21.15.78:0/2561838561 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a30040070 con 0x55bfffe394d0 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.336+0000 7f2a2e7fc640 1 -- 172.21.15.78:0/2561838561 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300315a0 con 0x55bfffe394d0 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 -- 172.21.15.78:0/2561838561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 msgr2=0x55c00020ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c00020ba40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2a30005670 tx=0x7f2a3003e000 comp rx=0 tx=0).stop 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 -- 172.21.15.78:0/2561838561 shutdown_connections 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c0001f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe394d0 0x55c00020ba40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2561838561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe312f0 0x55c0001f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 -- 172.21.15.78:0/2561838561 >> 172.21.15.78:0/2561838561 conn(0x55bfffe4e300 msgr2=0x55c0001feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 -- 172.21.15.78:0/2561838561 shutdown_connections 2024-04-25T14:00:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.337+0000 7f2a51b73740 1 -- 172.21.15.78:0/2561838561 wait complete. 2024-04-25T14:00:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.338+0000 7f2a51b73740 1 -- start start 2024-04-25T14:00:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.338+0000 7f2a51b73740 1 Processor -- start 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe312f0 0x55c000208af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000209030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c000278b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe312f0 0x55c000208af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe312f0 0x55c000208af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39584/0 (socket says 172.21.15.78:39584) 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 -- 172.21.15.78:0/2162146738 learned_addr learned my addr 172.21.15.78:0/2162146738 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441abcc0 con 0x55bfffe312f0 2024-04-25T14:00:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441ad130 con 0x55bfffe5a1b0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441ac390 con 0x55bfffe394d0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a43fff640 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000209030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 -- 172.21.15.78:0/2162146738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 msgr2=0x55c000209030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000209030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 -- 172.21.15.78:0/2162146738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 msgr2=0x55c000278b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c000278b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a4925a640 1 -- 172.21.15.78:0/2162146738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c000279070 con 0x55bfffe312f0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 10 client.0 _ll_register_callbacks cb 0x7f2a30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.339+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a30040070 con 0x55bfffe312f0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.340+0000 7f2a4925a640 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe312f0 0x55c000208af0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c002230 tx=0x7f2a3c010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8da99df2c09b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.340+0000 7f2a23fff640 10 client.15750 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.340+0000 7f2a23fff640 1 -- 172.21.15.78:0/2162146738 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c023040 con 0x55bfffe312f0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.340+0000 7f2a23fff640 1 -- 172.21.15.78:0/2162146738 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2a3c01b660 con 0x55bfffe312f0 2024-04-25T14:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.340+0000 7f2a23fff640 1 -- 172.21.15.78:0/2162146738 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2a3c016040 con 0x55bfffe312f0 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.340+0000 7f2a51b73740 10 client.15750 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.340+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55c00027a240 con 0x55bfffe312f0 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a23fff640 1 -- 172.21.15.78:0/2162146738 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2a3c029040 con 0x55bfffe312f0 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a51b73740 10 client.15750 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a51b73740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a51b73740 20 client.15750 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffdee1c0 0x55c00027c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a51b73740 4 client.15750 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c00016e070 con 0x55bfffdee1c0 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a43fff640 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffdee1c0 0x55c00027c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a23fff640 10 client.15750 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.341+0000 7f2a43fff640 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffdee1c0 0x55c00027c7b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7737273faf32a7c server_cookie=fae37928ecc4d456 in_seq=0 out_seq=0 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.345+0000 7f2a23fff640 1 -- 172.21.15.78:0/2162146738 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c00016e070 con 0x55bfffdee1c0 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.345+0000 7f2a23fff640 10 client.15750 handle_command_reply: tid=0 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.345+0000 7f2a51b73740 1 client.15750 shutdown 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.345+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe312f0 msgr2=0x55c000208af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.345+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe312f0 0x55c000208af0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c002230 tx=0x7f2a3c010fc0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 shutdown_connections 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfffe5a1b0 0x55c000278b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfffdee1c0 0x55c00027c7b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfffe394d0 0x55c000209030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 --2- 172.21.15.78:0/2162146738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfffe312f0 0x55c000208af0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a23fff640 0 client.15750 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 >> 172.21.15.78:0/2162146738 conn(0x55bfffe4e300 msgr2=0x55c0001f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 shutdown_connections 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 1 -- 172.21.15.78:0/2162146738 wait complete. 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.346+0000 7f2a51b73740 20 client.15750 trim_cache size 0 max 16384 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:00:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2a340afb60 msgr2=0x7f2a340b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2a340afb60 0x7f2a340b2070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a380a75d0 tx=0x7f2a380e26d0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102430 msgr2=0x7f2a4419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102430 0x7f2a4419c490 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800e9c0 tx=0x7f2a3800ee80 comp rx=0 tx=0).stop 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 shutdown_connections 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2a441036d0 0x7f2a441a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a44102d80 0x7f2a4419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2a340afb60 0x7f2a340b2070 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 --2- 172.21.15.78:0/3473394934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a44102430 0x7f2a4419c490 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 >> 172.21.15.78:0/3473394934 conn(0x7f2a440fdc60 msgr2=0x7f2a440ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 shutdown_connections 2024-04-25T14:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:23.347+0000 7f2a4acbd640 1 -- 172.21.15.78:0/3473394934 wait complete. 2024-04-25T14:00:23.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:23 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:00:23.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:23 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:23 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:00:23.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:23 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:23 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:00:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:23 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:23 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:00:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:23 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:23 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:00:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:23 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:23 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:00:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:23 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:24 smithi129 ceph-mon[32569]: pgmap v375: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T14:00:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:24 smithi078 ceph-mon[29163]: pgmap v375: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T14:00:24.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:24 smithi112 ceph-mon[32617]: pgmap v375: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T14:00:26.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:25 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:00:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:25 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:00:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:25 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:00:27.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:26 smithi112 ceph-mon[32617]: pgmap v376: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:27.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:26 smithi129 ceph-mon[32569]: pgmap v376: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:27.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:26 smithi078 ceph-mon[29163]: pgmap v376: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:28 smithi129 ceph-mon[32569]: pgmap v377: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:28.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:28 smithi078 ceph-mon[29163]: pgmap v377: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:28.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:28 smithi112 ceph-mon[32617]: pgmap v377: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:30 smithi129 ceph-mon[32569]: pgmap v378: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:30 smithi078 ceph-mon[29163]: pgmap v378: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:30.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:30 smithi112 ceph-mon[32617]: pgmap v378: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:32.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:32 smithi129 ceph-mon[32569]: pgmap v379: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:00:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:32 smithi078 ceph-mon[29163]: pgmap v379: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:00:32.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:32 smithi112 ceph-mon[32617]: pgmap v379: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T14:00:34.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:33 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:00:34.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:33 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:00:34.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:33 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:00:35.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:34 smithi078 ceph-mon[29163]: pgmap v380: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:35.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:34 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:34 smithi112 ceph-mon[32617]: pgmap v380: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:35.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:34 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:34 smithi129 ceph-mon[32569]: pgmap v380: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:35.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:35.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:34 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:36.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:00:36.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:00:36.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:00:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:00:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:00:36.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:00:36.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:00:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 -- 172.21.15.78:0/481796308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c109af0 msgr2=0x7f9f0c10bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 --2- 172.21.15.78:0/481796308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c109af0 0x7f9f0c10bf10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009ca0 tx=0x7f9f0402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 -- 172.21.15.78:0/481796308 shutdown_connections 2024-04-25T14:00:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 --2- 172.21.15.78:0/481796308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c109af0 0x7f9f0c10bf10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 --2- 172.21.15.78:0/481796308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f0c104df0 0x7f9f0c105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 --2- 172.21.15.78:0/481796308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f0c103bb0 0x7f9f0c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 -- 172.21.15.78:0/481796308 >> 172.21.15.78:0/481796308 conn(0x7f9f0c0feda0 msgr2=0x7f9f0c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 -- 172.21.15.78:0/481796308 shutdown_connections 2024-04-25T14:00:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.450+0000 7f9f13015640 1 -- 172.21.15.78:0/481796308 wait complete. 2024-04-25T14:00:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 Processor -- start 2024-04-25T14:00:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 -- start start 2024-04-25T14:00:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c103bb0 0x7f9f0c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f0c104df0 0x7f9f0c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f0c109af0 0x7f9f0c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10f670 con 0x7f9f0c103bb0 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10f4f0 con 0x7f9f0c109af0 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f13015640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10f7f0 con 0x7f9f0c104df0 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f10db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c103bb0 0x7f9f0c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f10db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c103bb0 0x7f9f0c19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45514/0 (socket says 172.21.15.78:45514) 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f10db1640 1 -- 172.21.15.78:0/4172578191 learned_addr learned my addr 172.21.15.78:0/4172578191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.451+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f0c104df0 0x7f9f0c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f115b2640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f0c109af0 0x7f9f0c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f10db1640 1 -- 172.21.15.78:0/4172578191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f0c104df0 msgr2=0x7f9f0c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f10db1640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f0c104df0 0x7f9f0c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f10db1640 1 -- 172.21.15.78:0/4172578191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f0c109af0 msgr2=0x7f9f0c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f10db1640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f0c109af0 0x7f9f0c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f10db1640 1 -- 172.21.15.78:0/4172578191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c075ee0 con 0x7f9f0c103bb0 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f10db1640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c103bb0 0x7f9f0c19a1e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8005d00 tx=0x7f9ef800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f09ffb640 1 -- 172.21.15.78:0/4172578191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8017070 con 0x7f9f0c103bb0 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c0761d0 con 0x7f9f0c103bb0 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f09ffb640 1 -- 172.21.15.78:0/4172578191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef8009c30 con 0x7f9f0c103bb0 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f09ffb640 1 -- 172.21.15.78:0/4172578191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800f620 con 0x7f9f0c103bb0 2024-04-25T14:00:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.452+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c076720 con 0x7f9f0c103bb0 2024-04-25T14:00:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.455+0000 7f9f19fa7740 1 Processor -- start 2024-04-25T14:00:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.455+0000 7f9f19fa7740 1 -- start start 2024-04-25T14:00:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.455+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb21c1b0 0x564acb5bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1f32f0 0x564acb5bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564acb522460 con 0x564acb21c1b0 2024-04-25T14:00:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564acb4dc590 con 0x564acb1f32f0 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564acb51e420 con 0x564acb1fb4d0 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb21c1b0 0x564acb5bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb21c1b0 0x564acb5bc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45518/0 (socket says 172.21.15.78:45518) 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 -- 172.21.15.78:0/1927300362 learned_addr learned my addr 172.21.15.78:0/1927300362 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 -- 172.21.15.78:0/1927300362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 msgr2=0x564acb5c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 -- 172.21.15.78:0/1927300362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1f32f0 msgr2=0x564acb5bcef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1f32f0 0x564acb5bcef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 -- 172.21.15.78:0/1927300362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564acb4e90d0 con 0x564acb21c1b0 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9f115b2640 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb21c1b0 0x564acb5bc970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9f04035000 tx=0x7f9f0402f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90254429bca94fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.456+0000 7f9ef6ffd640 1 -- 172.21.15.78:0/1927300362 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04031650 con 0x564acb21c1b0 2024-04-25T14:00:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.457+0000 7f9ef6ffd640 1 -- 172.21.15.78:0/1927300362 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f04036030 con 0x564acb21c1b0 2024-04-25T14:00:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.457+0000 7f9ef6ffd640 1 -- 172.21.15.78:0/1927300362 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04004800 con 0x564acb21c1b0 2024-04-25T14:00:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.457+0000 7f9f09ffb640 1 -- 172.21.15.78:0/4172578191 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9ef801a070 con 0x7f9f0c103bb0 2024-04-25T14:00:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.457+0000 7f9f09ffb640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9efc0afae0 0x7f9efc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f10db1640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9efc0afae0 0x7f9efc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1927300362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb21c1b0 msgr2=0x564acb5bc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb21c1b0 0x564acb5bc970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9f04035000 tx=0x7f9f0402f4c0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1927300362 shutdown_connections 2024-04-25T14:00:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1f32f0 0x564acb5bcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1927300362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb21c1b0 0x564acb5bc970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1927300362 >> 172.21.15.78:0/1927300362 conn(0x564acb210300 msgr2=0x564acb5b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1927300362 shutdown_connections 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f09ffb640 1 -- 172.21.15.78:0/4172578191 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9ef80de630 con 0x7f9f0c103bb0 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f10db1640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9efc0afae0 0x7f9efc0b1ff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800bfd0 tx=0x7f9ef809b4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.458+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1927300362 wait complete. 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 -- start start 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 Processor -- start 2024-04-25T14:00:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1fb4d0 0x564acb5cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb21c1b0 0x564acb5d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564acb5c4660 con 0x564acb1f32f0 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564acb4a9b60 con 0x564acb1fb4d0 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564acb5c47e0 con 0x564acb21c1b0 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f19fa7740 10 client.0 _ll_register_callbacks cb 0x564acb4ce2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f115b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f115b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45532/0 (socket says 172.21.15.78:45532) 2024-04-25T14:00:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.459+0000 7f9f115b2640 1 -- 172.21.15.78:0/1468783544 learned_addr learned my addr 172.21.15.78:0/1468783544 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1fb4d0 0x564acb5cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f115b2640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb21c1b0 0x564acb5d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f115b2640 1 -- 172.21.15.78:0/1468783544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb21c1b0 msgr2=0x564acb5d6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f115b2640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb21c1b0 0x564acb5d6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f115b2640 1 -- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1fb4d0 msgr2=0x564acb5cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f115b2640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1fb4d0 0x564acb5cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f115b2640 1 -- 172.21.15.78:0/1468783544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564acb234210 con 0x564acb1f32f0 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1fb4d0 0x564acb5cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f115b2640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5ceca0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9f04031b50 tx=0x7f9f040099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc9742c72b12a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9eebfff640 1 -- 172.21.15.78:0/1468783544 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04058060 con 0x564acb1f32f0 2024-04-25T14:00:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9eebfff640 1 -- 172.21.15.78:0/1468783544 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9f04003f40 con 0x564acb1f32f0 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.460+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f04036030 con 0x564acb1f32f0 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9eebfff640 1 -- 172.21.15.78:0/1468783544 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9f04004140 con 0x564acb1f32f0 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 10 client.15762 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564acb300620 con 0x564acb1f32f0 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9eebfff640 1 -- 172.21.15.78:0/1468783544 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f0402fb10 con 0x564acb1f32f0 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9eebfff640 1 -- 172.21.15.78:0/1468783544 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9f04043070 con 0x564acb1f32f0 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 10 client.15762 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 20 client.15762 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1dad80 0x564acb5dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 4 client.15762 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9f04036030 con 0x564acb1dad80 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.461+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1dad80 0x564acb5dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.462+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1dad80 0x564acb5dc680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd861773a6f7508 server_cookie=4e0d154ec1b0aad9 in_seq=0 out_seq=0 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.462+0000 7f9eebfff640 10 client.15762 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9eebfff640 1 -- 172.21.15.78:0/1468783544 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9f04036030 con 0x564acb1dad80 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9eebfff640 10 client.15762 handle_command_reply: tid=0 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 client.15762 shutdown 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 msgr2=0x564acb5ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5ceca0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9f04031b50 tx=0x7f9f040099f0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 shutdown_connections 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1dad80 0x564acb5dc680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb21c1b0 0x564acb5d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb1fb4d0 0x564acb5cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/1468783544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5ceca0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 >> 172.21.15.78:0/1468783544 conn(0x564acb210300 msgr2=0x564acb5b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9eebfff640 0 client.15762 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 shutdown_connections 2024-04-25T14:00:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 1 -- 172.21.15.78:0/1468783544 wait complete. 2024-04-25T14:00:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.463+0000 7f9f19fa7740 20 client.15762 trim_cache size 0 max 16384 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 Processor -- start 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 -- start start 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1f32f0 0x564acb5b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1fb4d0 0x564acb5ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ab830 con 0x564acb1fb4d0 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1a9100 con 0x564acb21c1b0 2024-04-25T14:00:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1a6f50 con 0x564acb1f32f0 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1fb4d0 0x564acb5ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1fb4d0 0x564acb5ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45534/0 (socket says 172.21.15.78:45534) 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f0bfff640 1 -- 172.21.15.78:0/2366154906 learned_addr learned my addr 172.21.15.78:0/2366154906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f115b2640 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1f32f0 0x564acb5b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f115b2640 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f0bfff640 1 -- 172.21.15.78:0/2366154906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1f32f0 msgr2=0x564acb5b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.493+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1f32f0 0x564acb5b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9f0bfff640 1 -- 172.21.15.78:0/2366154906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 msgr2=0x564acb5cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9f0bfff640 1 -- 172.21.15.78:0/2366154906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564acb5c1490 con 0x564acb1fb4d0 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9f115b2640 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1fb4d0 0x564acb5ce870 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x564acb5b9090 tx=0x7f9f00005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc7cbb4ad004ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9ef67fc640 1 -- 172.21.15.78:0/2366154906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00003e20 con 0x564acb1fb4d0 2024-04-25T14:00:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9ef67fc640 1 -- 172.21.15.78:0/2366154906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f000054d0 con 0x564acb1fb4d0 2024-04-25T14:00:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.494+0000 7f9ef67fc640 1 -- 172.21.15.78:0/2366154906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00004e60 con 0x564acb1fb4d0 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 -- 172.21.15.78:0/2366154906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1fb4d0 msgr2=0x564acb5ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1fb4d0 0x564acb5ce870 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x564acb5b9090 tx=0x7f9f00005f90 comp rx=0 tx=0).stop 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 -- 172.21.15.78:0/2366154906 shutdown_connections 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1fb4d0 0x564acb5ce870 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/2366154906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1f32f0 0x564acb5b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 -- 172.21.15.78:0/2366154906 >> 172.21.15.78:0/2366154906 conn(0x564acb210300 msgr2=0x564acb5b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 -- 172.21.15.78:0/2366154906 shutdown_connections 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 -- 172.21.15.78:0/2366154906 wait complete. 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 -- start start 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.495+0000 7f9f19fa7740 1 Processor -- start 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f19fa7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1a75c0 con 0x564acb1f32f0 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1a8a30 con 0x564acb21c1b0 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f19fa7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1a7c90 con 0x564acb1fb4d0 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45544/0 (socket says 172.21.15.78:45544) 2024-04-25T14:00:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 -- 172.21.15.78:0/3340271967 learned_addr learned my addr 172.21.15.78:0/3340271967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f19fa7740 10 client.0 _ll_register_callbacks cb 0x7f9f0402f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 -- 172.21.15.78:0/3340271967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 msgr2=0x564acb5d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 -- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 msgr2=0x564acb5d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5d4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f115b2640 1 -- 172.21.15.78:0/3340271967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564acb5d4a20 con 0x564acb1f32f0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.496+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.497+0000 7f9f115b2640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5d0fe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9f040094e0 tx=0x7f9f0403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3054e1d9685058f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.497+0000 7f9eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.497+0000 7f9eebfff640 1 -- 172.21.15.78:0/3340271967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04058070 con 0x564acb1f32f0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.497+0000 7f9eebfff640 1 -- 172.21.15.78:0/3340271967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9f040042e0 con 0x564acb1f32f0 2024-04-25T14:00:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.497+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f000054d0 con 0x564acb1f32f0 2024-04-25T14:00:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.497+0000 7f9eebfff640 1 -- 172.21.15.78:0/3340271967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9f040305c0 con 0x564acb1f32f0 2024-04-25T14:00:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.498+0000 7f9f19fa7740 10 client.15768 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.498+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564acb644a30 con 0x564acb1f32f0 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9eebfff640 1 -- 172.21.15.78:0/3340271967 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f04036040 con 0x564acb1f32f0 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9eebfff640 1 -- 172.21.15.78:0/3340271967 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9f04043070 con 0x564acb1f32f0 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9f19fa7740 10 client.15768 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9f19fa7740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9f19fa7740 20 client.15768 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1b01c0 0x564acb6472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9f19fa7740 4 client.15768 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9f000054d0 con 0x564acb1b01c0 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.499+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1b01c0 0x564acb6472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.500+0000 7f9f0bfff640 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1b01c0 0x564acb6472b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df4ad835d86e014 server_cookie=5aee7750cffe5449 in_seq=0 out_seq=0 2024-04-25T14:00:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.500+0000 7f9eebfff640 10 client.15768 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9eebfff640 1 -- 172.21.15.78:0/3340271967 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9f000054d0 con 0x564acb1b01c0 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9eebfff640 10 client.15768 handle_command_reply: tid=0 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 client.15768 shutdown 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 msgr2=0x564acb5d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5d0fe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9f040094e0 tx=0x7f9f0403e040 comp rx=0 tx=0).stop 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 shutdown_connections 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564acb21c1b0 0x564acb5d4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564acb1b01c0 0x564acb6472b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acb1fb4d0 0x564acb5d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9eebfff640 0 client.15768 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 --2- 172.21.15.78:0/3340271967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564acb1f32f0 0x564acb5d0fe0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 >> 172.21.15.78:0/3340271967 conn(0x564acb210300 msgr2=0x564acb5ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 shutdown_connections 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 1 -- 172.21.15.78:0/3340271967 wait complete. 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.504+0000 7f9f19fa7740 20 client.15768 trim_cache size 0 max 16384 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:00:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:00:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9efc0afae0 msgr2=0x7f9efc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9efc0afae0 0x7f9efc0b1ff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800bfd0 tx=0x7f9ef809b4f0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c103bb0 msgr2=0x7f9f0c19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c103bb0 0x7f9f0c19a1e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8005d00 tx=0x7f9ef800dde0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 shutdown_connections 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f0c109af0 0x7f9f0c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f0c104df0 0x7f9f0c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9efc0afae0 0x7f9efc0b1ff0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 --2- 172.21.15.78:0/4172578191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f0c103bb0 0x7f9f0c19a1e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 >> 172.21.15.78:0/4172578191 conn(0x7f9f0c0feda0 msgr2=0x7f9f0c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 shutdown_connections 2024-04-25T14:00:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:36.505+0000 7f9f13015640 1 -- 172.21.15.78:0/4172578191 wait complete. 2024-04-25T14:00:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:36 smithi078 ceph-mon[29163]: pgmap v381: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:36 smithi112 ceph-mon[32617]: pgmap v381: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:36 smithi129 ceph-mon[32569]: pgmap v381: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:38 smithi129 ceph-mon[32569]: pgmap v382: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:00:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:38 smithi078 ceph-mon[29163]: pgmap v382: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:00:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:38 smithi112 ceph-mon[32617]: pgmap v382: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T14:00:40.466 INFO:tasks.workunit.client.0.smithi078.stderr: git switch -c 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr:Or undo this operation with: 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr: git switch - 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T14:00:40.467 INFO:tasks.workunit.client.0.smithi078.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T14:00:40.472 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T14:00:40.538 INFO:tasks.workunit.client.0.smithi078.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T14:00:40.540 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T14:00:40.540 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T14:00:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:40 smithi129 ceph-mon[32569]: pgmap v383: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T14:00:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:40 smithi078 ceph-mon[29163]: pgmap v383: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T14:00:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:40 smithi112 ceph-mon[32617]: pgmap v383: 97 pgs: 97 active+clean; 665 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T14:00:41.119 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T14:00:41.175 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T14:00:41.215 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T14:00:41.216 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T14:00:41.217 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T14:00:41.256 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T14:00:41.259 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T14:00:41.259 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T14:00:41.314 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-25T14:00:41.315 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-25T14:00:41.315 DEBUG:teuthology.orchestra.run.smithi078:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T14:00:41.374 INFO:tasks.workunit.client.0.smithi078.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T14:00:41.374 INFO:tasks.workunit.client.0.smithi078.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-25T14:00:41.375 INFO:tasks.workunit.client.0.smithi078.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-25T14:00:42.666 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:42 smithi129 ceph-mon[32569]: pgmap v384: 97 pgs: 97 active+clean; 666 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:42 smithi078 ceph-mon[29163]: pgmap v384: 97 pgs: 97 active+clean; 666 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:42.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:42 smithi112 ceph-mon[32617]: pgmap v384: 97 pgs: 97 active+clean; 666 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T14:00:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:44 smithi129 ceph-mon[32569]: pgmap v385: 97 pgs: 97 active+clean; 666 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:44 smithi078 ceph-mon[29163]: pgmap v385: 97 pgs: 97 active+clean; 666 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:44 smithi112 ceph-mon[32617]: pgmap v385: 97 pgs: 97 active+clean; 666 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T14:00:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:46 smithi129 ceph-mon[32569]: pgmap v386: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T14:00:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:46 smithi078 ceph-mon[29163]: pgmap v386: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T14:00:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:46 smithi112 ceph-mon[32617]: pgmap v386: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T14:00:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:48 smithi129 ceph-mon[32569]: pgmap v387: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T14:00:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:48 smithi078 ceph-mon[29163]: pgmap v387: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T14:00:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:48 smithi112 ceph-mon[32617]: pgmap v387: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T14:00:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.608+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/4099478445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8102810 msgr2=0x7ff4f8102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.608+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/4099478445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8102810 0x7ff4f8102ce0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009900 tx=0x7ff4e802a410 comp rx=0 tx=0).stop 2024-04-25T14:00:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/4099478445 shutdown_connections 2024-04-25T14:00:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/4099478445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f8107980 0x7ff4f8107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/4099478445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8102810 0x7ff4f8102ce0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/4099478445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8101dd0 0x7ff4f8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/4099478445 >> 172.21.15.78:0/4099478445 conn(0x7ff4f80fd3c0 msgr2=0x7ff4f80ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/4099478445 shutdown_connections 2024-04-25T14:00:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/4099478445 wait complete. 2024-04-25T14:00:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 Processor -- start 2024-04-25T14:00:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 -- start start 2024-04-25T14:00:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8101dd0 0x7ff4f81a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f8102810 0x7ff4f81a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.609+0000 7ff4ffb8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8107980 0x7ff4f81a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4ffb8f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8115c50 con 0x7ff4f8107980 2024-04-25T14:00:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4ffb8f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8115ad0 con 0x7ff4f8101dd0 2024-04-25T14:00:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4ffb8f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8115dd0 con 0x7ff4f8102810 2024-04-25T14:00:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8107980 0x7ff4f81a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8107980 0x7ff4f81a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59876/0 (socket says 172.21.15.78:59876) 2024-04-25T14:00:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 -- 172.21.15.78:0/426393477 learned_addr learned my addr 172.21.15.78:0/426393477 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f8102810 0x7ff4f81a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8101dd0 0x7ff4f81a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 -- 172.21.15.78:0/426393477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f8102810 msgr2=0x7ff4f81a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f8102810 0x7ff4f81a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 -- 172.21.15.78:0/426393477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8101dd0 msgr2=0x7ff4f81a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8101dd0 0x7ff4f81a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 -- 172.21.15.78:0/426393477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f81a8320 con 0x7ff4f8107980 2024-04-25T14:00:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4fe12c640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8107980 0x7ff4f81a7cb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000eb10 tx=0x7ff4f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4eeffd640 1 -- 172.21.15.78:0/426393477 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0004b20 con 0x7ff4f8107980 2024-04-25T14:00:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4eeffd640 1 -- 172.21.15.78:0/426393477 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f0004cc0 con 0x7ff4f8107980 2024-04-25T14:00:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f81a8610 con 0x7ff4f8107980 2024-04-25T14:00:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4eeffd640 1 -- 172.21.15.78:0/426393477 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0010670 con 0x7ff4f8107980 2024-04-25T14:00:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.610+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a8bb0 con 0x7ff4f8107980 2024-04-25T14:00:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.613+0000 7ff4eeffd640 1 -- 172.21.15.78:0/426393477 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff4f0005020 con 0x7ff4f8107980 2024-04-25T14:00:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.613+0000 7ff506a44740 1 Processor -- start 2024-04-25T14:00:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.613+0000 7ff506a44740 1 -- start start 2024-04-25T14:00:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff506a44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58b9e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff506a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587d52f0 0x55fd58b9eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff506a44740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587dd4d0 0x55fd58ba5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff506a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fd58b04460 con 0x55fd587d52f0 2024-04-25T14:00:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff506a44740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd58abe590 con 0x55fd587dd4d0 2024-04-25T14:00:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff506a44740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fd58b00420 con 0x55fd587fe1b0 2024-04-25T14:00:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4eeffd640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff4e40afb60 0x7ff4e40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587d52f0 0x55fd58b9eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587d52f0 0x55fd58b9eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59884/0 (socket says 172.21.15.78:59884) 2024-04-25T14:00:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 -- 172.21.15.78:0/1723458057 learned_addr learned my addr 172.21.15.78:0/1723458057 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58b9e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fe12c640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff4e40afb60 0x7ff4e40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587dd4d0 0x55fd58ba5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4eeffd640 1 -- 172.21.15.78:0/426393477 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff4f00df1d0 con 0x7ff4f8107980 2024-04-25T14:00:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 -- 172.21.15.78:0/1723458057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 msgr2=0x55fd58b9e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58b9e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 -- 172.21.15.78:0/1723458057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587dd4d0 msgr2=0x55fd58ba5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587dd4d0 0x55fd58ba5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 -- 172.21.15.78:0/1723458057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd58acb0d0 con 0x55fd587d52f0 2024-04-25T14:00:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587d52f0 0x55fd58b9eef0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400cbf0 tx=0x7ff4f400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1fb67fb990afc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.614+0000 7ff4fe12c640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff4e40afb60 0x7ff4e40b2070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00a73a0 tx=0x7ff4f006db10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.615+0000 7ff4e3fff640 1 -- 172.21.15.78:0/1723458057 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4010630 con 0x55fd587d52f0 2024-04-25T14:00:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.615+0000 7ff4e3fff640 1 -- 172.21.15.78:0/1723458057 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f4010c80 con 0x55fd587d52f0 2024-04-25T14:00:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.615+0000 7ff4e3fff640 1 -- 172.21.15.78:0/1723458057 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f400fb20 con 0x55fd587d52f0 2024-04-25T14:00:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 -- 172.21.15.78:0/1723458057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587d52f0 msgr2=0x55fd58b9eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587d52f0 0x55fd58b9eef0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400cbf0 tx=0x7ff4f400e720 comp rx=0 tx=0).stop 2024-04-25T14:00:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 -- 172.21.15.78:0/1723458057 shutdown_connections 2024-04-25T14:00:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587dd4d0 0x55fd58ba5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587d52f0 0x55fd58b9eef0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 --2- 172.21.15.78:0/1723458057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58b9e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 -- 172.21.15.78:0/1723458057 >> 172.21.15.78:0/1723458057 conn(0x55fd587f2300 msgr2=0x55fd58b9a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 -- 172.21.15.78:0/1723458057 shutdown_connections 2024-04-25T14:00:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 -- 172.21.15.78:0/1723458057 wait complete. 2024-04-25T14:00:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 -- start start 2024-04-25T14:00:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.616+0000 7ff506a44740 1 Processor -- start 2024-04-25T14:00:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff506a44740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 0x55fd58ba85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff506a44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587dd4d0 0x55fd58ba8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff506a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587fe1b0 0x55fd58bafa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff506a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fd58ba6840 con 0x55fd587fe1b0 2024-04-25T14:00:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff506a44740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd58a8bb60 con 0x55fd587d52f0 2024-04-25T14:00:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff506a44740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fd58ba69c0 con 0x55fd587dd4d0 2024-04-25T14:00:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff506a44740 10 client.0 _ll_register_callbacks cb 0x55fd58ab02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587fe1b0 0x55fd58bafa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587fe1b0 0x55fd58bafa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59886/0 (socket says 172.21.15.78:59886) 2024-04-25T14:00:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 -- 172.21.15.78:0/2014125555 learned_addr learned my addr 172.21.15.78:0/2014125555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 0x55fd58ba85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587dd4d0 0x55fd58ba8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 -- 172.21.15.78:0/2014125555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587dd4d0 msgr2=0x55fd58ba8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587dd4d0 0x55fd58ba8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 -- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 msgr2=0x55fd58ba85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 0x55fd58ba85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 -- 172.21.15.78:0/2014125555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd58816210 con 0x55fd587fe1b0 2024-04-25T14:00:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587fe1b0 0x55fd58bafa80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8031500 tx=0x7ff4e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afbb0d9d8427369 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.617+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2014125555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8035040 con 0x55fd587fe1b0 2024-04-25T14:00:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2014125555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff4e8004480 con 0x55fd587fe1b0 2024-04-25T14:00:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f4010c80 con 0x55fd587fe1b0 2024-04-25T14:00:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2014125555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff4e80035f0 con 0x55fd587fe1b0 2024-04-25T14:00:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 10 client.15780 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fd588e2620 con 0x55fd587fe1b0 2024-04-25T14:00:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2014125555 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f4010c80 con 0x55fd587fe1b0 2024-04-25T14:00:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2014125555 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff4e8055050 con 0x55fd587fe1b0 2024-04-25T14:00:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 10 client.15780 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 20 client.15780 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587bcd80 0x55fd58bb6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 4 client.15780 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd58b12070 con 0x55fd587bcd80 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.618+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587bcd80 0x55fd58bb6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.619+0000 7ff4e17fa640 10 client.15780 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.619+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587bcd80 0x55fd58bb6290 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de1f0719159d8e34 server_cookie=bd593e8d66d14cd1 in_seq=0 out_seq=0 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2014125555 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fd58b12070 con 0x55fd587bcd80 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff4e17fa640 10 client.15780 handle_command_reply: tid=0 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 client.15780 shutdown 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587fe1b0 msgr2=0x55fd58bafa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587fe1b0 0x55fd58bafa80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8031500 tx=0x7ff4e8009260 comp rx=0 tx=0).stop 2024-04-25T14:00:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 shutdown_connections 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587fe1b0 0x55fd58bafa80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587bcd80 0x55fd58bb6290 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587dd4d0 0x55fd58ba8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 --2- 172.21.15.78:0/2014125555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 0x55fd58ba85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff4e17fa640 0 client.15780 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 >> 172.21.15.78:0/2014125555 conn(0x55fd587f2300 msgr2=0x55fd58b9d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 shutdown_connections 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 1 -- 172.21.15.78:0/2014125555 wait complete. 2024-04-25T14:00:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.620+0000 7ff506a44740 20 client.15780 trim_cache size 0 max 16384 2024-04-25T14:00:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 Processor -- start 2024-04-25T14:00:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 -- start start 2024-04-25T14:00:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 0x55fd58b9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58ba6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b41d0 con 0x55fd587dd4d0 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b1aa0 con 0x55fd587d52f0 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff506a44740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81af8f0 con 0x55fd587fe1b0 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bafa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59896/0 (socket says 172.21.15.78:59896) 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2992289087 learned_addr learned my addr 172.21.15.78:0/2992289087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58ba6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2992289087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 msgr2=0x55fd58ba6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58ba6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2992289087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 msgr2=0x55fd58b9a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 0x55fd58b9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2992289087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd58ba3480 con 0x55fd587dd4d0 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.635+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58ba6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bafa10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55fd58b9fd50 tx=0x7ff4f400c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a73777f3bc2e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff4e37fe640 1 -- 172.21.15.78:0/2992289087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f402c070 con 0x55fd587dd4d0 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff4e37fe640 1 -- 172.21.15.78:0/2992289087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f40054d0 con 0x55fd587dd4d0 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff4e37fe640 1 -- 172.21.15.78:0/2992289087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004b80 con 0x55fd587dd4d0 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff506a44740 1 -- 172.21.15.78:0/2992289087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 msgr2=0x55fd58bafa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff506a44740 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bafa10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55fd58b9fd50 tx=0x7ff4f400c4f0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff506a44740 1 -- 172.21.15.78:0/2992289087 shutdown_connections 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff506a44740 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587fe1b0 0x55fd58ba6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff506a44740 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bafa10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.636+0000 7ff506a44740 1 --2- 172.21.15.78:0/2992289087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587d52f0 0x55fd58b9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 -- 172.21.15.78:0/2992289087 >> 172.21.15.78:0/2992289087 conn(0x55fd587f2300 msgr2=0x55fd58b99cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 -- 172.21.15.78:0/2992289087 shutdown_connections 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 -- 172.21.15.78:0/2992289087 wait complete. 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 -- start start 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 Processor -- start 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587d52f0 0x55fd58bacc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587fe1b0 0x55fd58bae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81aff60 con 0x55fd587dd4d0 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b13d0 con 0x55fd587fe1b0 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81b0630 con 0x55fd587d52f0 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff506a44740 10 client.0 _ll_register_callbacks cb 0x7ff4e8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff4fd92b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff4fd92b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59902/0 (socket says 172.21.15.78:59902) 2024-04-25T14:00:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.637+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2945613706 learned_addr learned my addr 172.21.15.78:0/2945613706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587d52f0 0x55fd58bacc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587fe1b0 0x55fd58bae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2945613706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587d52f0 msgr2=0x55fd58bacc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587d52f0 0x55fd58bacc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587fe1b0 msgr2=0x55fd58bae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587fe1b0 0x55fd58bae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd92b640 1 -- 172.21.15.78:0/2945613706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd58bae640 con 0x55fd587dd4d0 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587fe1b0 0x55fd58bae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4fd92b640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bad150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40058c0 tx=0x7ff4f40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491e198b89c7166c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2945613706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4037070 con 0x55fd587dd4d0 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2945613706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff4f4004230 con 0x55fd587dd4d0 2024-04-25T14:00:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f40054d0 con 0x55fd587dd4d0 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2945613706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff4f4043050 con 0x55fd587dd4d0 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff506a44740 10 client.15786 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fd58baf840 con 0x55fd587dd4d0 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.638+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2945613706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f40054d0 con 0x55fd587dd4d0 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2945613706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff4f4003820 con 0x55fd587dd4d0 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff506a44740 10 client.15786 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff506a44740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff506a44740 20 client.15786 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff506a44740 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587921c0 0x55fd58c20710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff506a44740 4 client.15786 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd58b0bee0 con 0x55fd587921c0 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.639+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587921c0 0x55fd58c20710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.640+0000 7ff4e17fa640 10 client.15786 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.640+0000 7ff4fd12a640 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587921c0 0x55fd58c20710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aace5fe6fac768d server_cookie=37c63d3f90a5ef5d in_seq=0 out_seq=0 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff4e17fa640 1 -- 172.21.15.78:0/2945613706 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd58b0bee0 con 0x55fd587921c0 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff4e17fa640 10 client.15786 handle_command_reply: tid=0 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 client.15786 shutdown 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 msgr2=0x55fd58bad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bad150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40058c0 tx=0x7ff4f40056a0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 shutdown_connections 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd587fe1b0 0x55fd58bae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd587dd4d0 0x55fd58bad150 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fd587921c0 0x55fd58c20710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 --2- 172.21.15.78:0/2945613706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd587d52f0 0x55fd58bacc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff4e17fa640 0 client.15786 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 >> 172.21.15.78:0/2945613706 conn(0x55fd587f2300 msgr2=0x55fd58ba6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 shutdown_connections 2024-04-25T14:00:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 1 -- 172.21.15.78:0/2945613706 wait complete. 2024-04-25T14:00:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff506a44740 20 client.15786 trim_cache size 0 max 16384 2024-04-25T14:00:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:00:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:00:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:00:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:00:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff4e40afb60 msgr2=0x7ff4e40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff4e40afb60 0x7ff4e40b2070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00a73a0 tx=0x7ff4f006db10 comp rx=0 tx=0).stop 2024-04-25T14:00:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8107980 msgr2=0x7ff4f81a7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.649+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8107980 0x7ff4f81a7cb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000eb10 tx=0x7ff4f000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 shutdown_connections 2024-04-25T14:00:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff4e40afb60 0x7ff4e40b2070 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4f8107980 0x7ff4f81a7cb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f8102810 0x7ff4f81a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 --2- 172.21.15.78:0/426393477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f8101dd0 0x7ff4f81a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 >> 172.21.15.78:0/426393477 conn(0x7ff4f80fd3c0 msgr2=0x7ff4f8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 shutdown_connections 2024-04-25T14:00:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:00:49.650+0000 7ff4ffb8f640 1 -- 172.21.15.78:0/426393477 wait complete. 2024-04-25T14:00:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:50 smithi129 ceph-mon[32569]: pgmap v388: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-04-25T14:00:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:50 smithi078 ceph-mon[29163]: pgmap v388: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-04-25T14:00:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:50 smithi112 ceph-mon[32617]: pgmap v388: 97 pgs: 97 active+clean; 719 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-04-25T14:00:52.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:52 smithi112 ceph-mon[32617]: pgmap v389: 97 pgs: 97 active+clean; 777 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:00:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:52 smithi129 ceph-mon[32569]: pgmap v389: 97 pgs: 97 active+clean; 777 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:00:52.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:52.677 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:52.703 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:52 smithi078 ceph-mon[29163]: pgmap v389: 97 pgs: 97 active+clean; 777 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:00:52.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 -- 172.21.15.78:0/3550023473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501098b0 msgr2=0x7fb15010bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 --2- 172.21.15.78:0/3550023473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501098b0 0x7fb15010bcd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb148009ca0 tx=0x7fb14802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 -- 172.21.15.78:0/3550023473 shutdown_connections 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 --2- 172.21.15.78:0/3550023473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501098b0 0x7fb15010bcd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 --2- 172.21.15.78:0/3550023473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb150106f50 0x7fb150109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 --2- 172.21.15.78:0/3550023473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1501045f0 0x7fb150106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 -- 172.21.15.78:0/3550023473 >> 172.21.15.78:0/3550023473 conn(0x7fb1500fdbf0 msgr2=0x7fb1501000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 -- 172.21.15.78:0/3550023473 shutdown_connections 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.804+0000 7fb15657a640 1 -- 172.21.15.78:0/3550023473 wait complete. 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 Processor -- start 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 -- start start 2024-04-25T14:00:52.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501045f0 0x7fb15019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb150106f50 0x7fb15019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1501098b0 0x7fb1501a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb15010f630 con 0x7fb1501045f0 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb15010f4b0 con 0x7fb1501098b0 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb15657a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb15010f7b0 con 0x7fb150106f50 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.805+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501045f0 0x7fb15019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501045f0 0x7fb15019a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59926/0 (socket says 172.21.15.78:59926) 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 -- 172.21.15.78:0/2300498870 learned_addr learned my addr 172.21.15.78:0/2300498870 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb154b17640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1501098b0 0x7fb1501a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb150106f50 0x7fb15019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 -- 172.21.15.78:0/2300498870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb150106f50 msgr2=0x7fb15019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb150106f50 0x7fb15019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 -- 172.21.15.78:0/2300498870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1501098b0 msgr2=0x7fb1501a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1501098b0 0x7fb1501a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 -- 172.21.15.78:0/2300498870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1501a1d50 con 0x7fb1501045f0 2024-04-25T14:00:52.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14ffff640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501045f0 0x7fb15019a210 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9f0 tx=0x7fb14000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:52.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14d7fa640 1 -- 172.21.15.78:0/2300498870 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14000ccc0 con 0x7fb1501045f0 2024-04-25T14:00:52.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14d7fa640 1 -- 172.21.15.78:0/2300498870 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1400054d0 con 0x7fb1501045f0 2024-04-25T14:00:52.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb14d7fa640 1 -- 172.21.15.78:0/2300498870 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140010430 con 0x7fb1501045f0 2024-04-25T14:00:52.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1501a1fe0 con 0x7fb1501045f0 2024-04-25T14:00:52.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.806+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1501a2450 con 0x7fb1501045f0 2024-04-25T14:00:52.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb14d7fa640 1 -- 172.21.15.78:0/2300498870 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb1400046c0 con 0x7fb1501045f0 2024-04-25T14:00:52.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 Processor -- start 2024-04-25T14:00:52.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 -- start start 2024-04-25T14:00:52.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e7aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab04d0 0x557b79e7afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e7bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b79ddfd80 con 0x557b79ab86b0 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b79d998b0 con 0x557b79ad9390 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb15d420740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557b79ddbd40 con 0x557b79ab04d0 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb14f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e7bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb14f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e7bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59940/0 (socket says 172.21.15.78:59940) 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb154b17640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e7aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb14f7fe640 1 -- 172.21.15.78:0/1206944048 learned_addr learned my addr 172.21.15.78:0/1206944048 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.809+0000 7fb14ffff640 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab04d0 0x557b79e7afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14f7fe640 1 -- 172.21.15.78:0/1206944048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab04d0 msgr2=0x557b79e7afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab04d0 0x557b79e7afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14f7fe640 1 -- 172.21.15.78:0/1206944048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 msgr2=0x557b79e7aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e7aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14f7fe640 1 -- 172.21.15.78:0/1206944048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b79d6aa40 con 0x557b79ab86b0 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb154b17640 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e7aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14d7fa640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1380afaf0 0x7fb1380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e7bc70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb144009c20 tx=0x7fb1440117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3d172da087bb91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb154b17640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1380afaf0 0x7fb1380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb14d7fa640 1 -- 172.21.15.78:0/2300498870 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb1400df090 con 0x7fb1501045f0 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb1367fc640 1 -- 172.21.15.78:0/1206944048 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144012450 con 0x557b79ab86b0 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb1367fc640 1 -- 172.21.15.78:0/1206944048 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb144012aa0 con 0x557b79ab86b0 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.810+0000 7fb1367fc640 1 -- 172.21.15.78:0/1206944048 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400fa80 con 0x557b79ab86b0 2024-04-25T14:00:52.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 -- 172.21.15.78:0/1206944048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 msgr2=0x557b79e7bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb154b17640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1380afaf0 0x7fb1380b2000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb148004af0 tx=0x7fb14802f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e7bc70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb144009c20 tx=0x7fb1440117b0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 -- 172.21.15.78:0/1206944048 shutdown_connections 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e7bc70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab04d0 0x557b79e7afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 --2- 172.21.15.78:0/1206944048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e7aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 -- 172.21.15.78:0/1206944048 >> 172.21.15.78:0/1206944048 conn(0x557b79acd4e0 msgr2=0x557b79e76270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 -- 172.21.15.78:0/1206944048 shutdown_connections 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 -- 172.21.15.78:0/1206944048 wait complete. 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.811+0000 7fb15d420740 1 -- start start 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 1 Processor -- start 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e84870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab86b0 0x557b79e84db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e8bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b79e7c780 con 0x557b79ab04d0 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b79d66d10 con 0x557b79ad9390 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557b79e7c900 con 0x557b79ab86b0 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb15d420740 10 client.0 _ll_register_callbacks cb 0x557b79d93210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e84870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e84870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59942/0 (socket says 172.21.15.78:59942) 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 -- 172.21.15.78:0/3756393452 learned_addr learned my addr 172.21.15.78:0/3756393452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab86b0 0x557b79e84db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb154b17640 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e8bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 -- 172.21.15.78:0/3756393452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab86b0 msgr2=0x557b79e84db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab86b0 0x557b79e84db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 -- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 msgr2=0x557b79e8bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e8bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 -- 172.21.15.78:0/3756393452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b79bbb7d0 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb14ffff640 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e84870 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c6a0 tx=0x7fb1400a7b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e87892e6e613661 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb12f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.812+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3756393452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140005680 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3756393452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb1400afed0 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb144012aa0 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3756393452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb1400a7010 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 10 client.15798 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557b79af13f0 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3756393452 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb144012aa0 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3756393452 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb140005000 con 0x557b79ab04d0 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 10 client.15798 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 20 client.15798 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a97f60 0x557b79e920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 4 client.15798 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b79ded990 con 0x557b79a97f60 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.813+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a97f60 0x557b79e920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.814+0000 7fb12f7fe640 10 client.15798 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.814+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a97f60 0x557b79e920d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c9f76813fe0bb6 server_cookie=db717852ff62b293 in_seq=0 out_seq=0 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.814+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3756393452 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b79ded990 con 0x557b79a97f60 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.814+0000 7fb12f7fe640 10 client.15798 handle_command_reply: tid=0 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.814+0000 7fb15d420740 1 client.15798 shutdown 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 msgr2=0x557b79e84870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e84870 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c6a0 tx=0x7fb1400a7b20 comp rx=0 tx=0).stop 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 shutdown_connections 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ad9390 0x557b79e8bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a97f60 0x557b79e920d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ab86b0 0x557b79e84db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 --2- 172.21.15.78:0/3756393452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e84870 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb12f7fe640 0 client.15798 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 >> 172.21.15.78:0/3756393452 conn(0x557b79acd4e0 msgr2=0x557b79e79180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 shutdown_connections 2024-04-25T14:00:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 1 -- 172.21.15.78:0/3756393452 wait complete. 2024-04-25T14:00:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.815+0000 7fb15d420740 20 client.15798 trim_cache size 0 max 16384 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.832+0000 7fb15d420740 1 Processor -- start 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.832+0000 7fb15d420740 1 -- start start 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb15d420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb15d420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab86b0 0x557b79e8bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb15d420740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e84440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb15d420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501adaf0 con 0x557b79ab04d0 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb15d420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501ab3c0 con 0x557b79ab86b0 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb15d420740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501a9210 con 0x557b79ad9390 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e763b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59948/0 (socket says 172.21.15.78:59948) 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 -- 172.21.15.78:0/2696580727 learned_addr learned my addr 172.21.15.78:0/2696580727 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:52.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab86b0 0x557b79e8bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb154b17640 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e84440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 -- 172.21.15.78:0/2696580727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 msgr2=0x557b79e84440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e84440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 -- 172.21.15.78:0/2696580727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab86b0 msgr2=0x557b79e8bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab86b0 0x557b79e8bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 -- 172.21.15.78:0/2696580727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b79ab10f0 con 0x557b79ab04d0 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb154b17640 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e84440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb14ffff640 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e763b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9c0 tx=0x7fb1400a6f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb839f540e96d7d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb135ffb640 1 -- 172.21.15.78:0/2696580727 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140005680 con 0x557b79ab04d0 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb135ffb640 1 -- 172.21.15.78:0/2696580727 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1400a7c00 con 0x557b79ab04d0 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.833+0000 7fb135ffb640 1 -- 172.21.15.78:0/2696580727 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140005000 con 0x557b79ab04d0 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 -- 172.21.15.78:0/2696580727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 msgr2=0x557b79e763b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e763b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9c0 tx=0x7fb1400a6f10 comp rx=0 tx=0).stop 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 -- 172.21.15.78:0/2696580727 shutdown_connections 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e84440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab86b0 0x557b79e8bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 --2- 172.21.15.78:0/2696580727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab04d0 0x557b79e763b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 -- 172.21.15.78:0/2696580727 >> 172.21.15.78:0/2696580727 conn(0x557b79acd4e0 msgr2=0x557b79e75ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 -- 172.21.15.78:0/2696580727 shutdown_connections 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 -- 172.21.15.78:0/2696580727 wait complete. 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.834+0000 7fb15d420740 1 -- start start 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 1 Processor -- start 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab04d0 0x557b79e89310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e89850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e8a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501a9880 con 0x557b79ab86b0 2024-04-25T14:00:52.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501aacf0 con 0x557b79ab04d0 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1501a9f50 con 0x557b79ad9390 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb15d420740 10 client.0 _ll_register_callbacks cb 0x7fb14000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e89850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e89850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59960/0 (socket says 172.21.15.78:59960) 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 -- 172.21.15.78:0/3207693573 learned_addr learned my addr 172.21.15.78:0/3207693573 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14ffff640 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab04d0 0x557b79e89310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb154b17640 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e8a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 -- 172.21.15.78:0/3207693573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 msgr2=0x557b79e8a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e8a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 -- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab04d0 msgr2=0x557b79e89310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab04d0 0x557b79e89310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 -- 172.21.15.78:0/3207693573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b79da04b0 con 0x557b79ab86b0 2024-04-25T14:00:52.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb14f7fe640 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e89850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb144009c20 tx=0x7fb144005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ade9c50ae71697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.835+0000 7fb12f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3207693573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14402d250 con 0x557b79ab86b0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3207693573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb14402d3f0 con 0x557b79ab86b0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1400a7c00 con 0x557b79ab86b0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3207693573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb14402d5f0 con 0x557b79ab86b0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 10 client.15804 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557b79e8bd60 con 0x557b79ab86b0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3207693573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1400a7c00 con 0x557b79ab86b0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3207693573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb144036460 con 0x557b79ab86b0 2024-04-25T14:00:52.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 10 client.15804 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 20 client.15804 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a6d3a0 0x557b79efc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 4 client.15804 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b79de7800 con 0x557b79a6d3a0 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.836+0000 7fb14ffff640 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a6d3a0 0x557b79efc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb12f7fe640 10 client.15804 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb14ffff640 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a6d3a0 0x557b79efc7e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a4424577661b23 server_cookie=2cc22168c736d2ea in_seq=0 out_seq=0 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb12f7fe640 1 -- 172.21.15.78:0/3207693573 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557b79de7800 con 0x557b79a6d3a0 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb12f7fe640 10 client.15804 handle_command_reply: tid=0 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 client.15804 shutdown 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 msgr2=0x557b79e89850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e89850 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb144009c20 tx=0x7fb144005130 comp rx=0 tx=0).stop 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 shutdown_connections 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b79a6d3a0 0x557b79efc7e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b79ad9390 0x557b79e8a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb12f7fe640 0 client.15804 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b79ab86b0 0x557b79e89850 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 --2- 172.21.15.78:0/3207693573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b79ab04d0 0x557b79e89310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 >> 172.21.15.78:0/3207693573 conn(0x557b79acd4e0 msgr2=0x557b79e84440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 shutdown_connections 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.837+0000 7fb15d420740 1 -- 172.21.15.78:0/3207693573 wait complete. 2024-04-25T14:00:52.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15d420740 20 client.15804 trim_cache size 0 max 16384 2024-04-25T14:00:52.846 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:00:52.846 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:00:52.846 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:00:52.846 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1380afaf0 msgr2=0x7fb1380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1380afaf0 0x7fb1380b2000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb148004af0 tx=0x7fb14802f310 comp rx=0 tx=0).stop 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501045f0 msgr2=0x7fb15019a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501045f0 0x7fb15019a210 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb14000e9f0 tx=0x7fb14000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 shutdown_connections 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1501098b0 0x7fb1501a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb150106f50 0x7fb15019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1380afaf0 0x7fb1380b2000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 --2- 172.21.15.78:0/2300498870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1501045f0 0x7fb15019a210 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:52.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 >> 172.21.15.78:0/2300498870 conn(0x7fb1500fdbf0 msgr2=0x7fb1501050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:52.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 shutdown_connections 2024-04-25T14:00:52.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.838+0000 7fb15657a640 1 -- 172.21.15.78:0/2300498870 wait complete. 2024-04-25T14:00:52.849 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:00:52.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:00:52.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 -- 172.21.15.78:0/1097299248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81045f0 msgr2=0x7f30a8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:52.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 --2- 172.21.15.78:0/1097299248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81045f0 0x7f30a8106a10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f309c009900 tx=0x7f309c02a410 comp rx=0 tx=0).stop 2024-04-25T14:00:52.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 -- 172.21.15.78:0/1097299248 shutdown_connections 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 --2- 172.21.15.78:0/1097299248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a81098b0 0x7f30a810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 --2- 172.21.15.78:0/1097299248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a8106f50 0x7f30a8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 --2- 172.21.15.78:0/1097299248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81045f0 0x7f30a8106a10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 -- 172.21.15.78:0/1097299248 >> 172.21.15.78:0/1097299248 conn(0x7f30a80fdbf0 msgr2=0x7f30a81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 -- 172.21.15.78:0/1097299248 shutdown_connections 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.997+0000 7f30adf9d640 1 -- 172.21.15.78:0/1097299248 wait complete. 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 Processor -- start 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 -- start start 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a81045f0 0x7f30a819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a8106f50 0x7f30a819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81098b0 0x7f30a81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30a810f600 con 0x7f30a81098b0 2024-04-25T14:00:53.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30a810f480 con 0x7f30a8106f50 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30adf9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30a810f780 con 0x7f30a81045f0 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81098b0 0x7f30a81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81098b0 0x7f30a81a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59972/0 (socket says 172.21.15.78:59972) 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30a7fff640 1 -- 172.21.15.78:0/660511353 learned_addr learned my addr 172.21.15.78:0/660511353 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30a6ffd640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a8106f50 0x7f30a819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.998+0000 7f30a77fe640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a81045f0 0x7f30a819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a7fff640 1 -- 172.21.15.78:0/660511353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a81045f0 msgr2=0x7f30a819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a7fff640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a81045f0 0x7f30a819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a7fff640 1 -- 172.21.15.78:0/660511353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a8106f50 msgr2=0x7f30a819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a7fff640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a8106f50 0x7f30a819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a7fff640 1 -- 172.21.15.78:0/660511353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a81a1d40 con 0x7f30a81098b0 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a7fff640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81098b0 0x7f30a81a16d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f30a000eb40 tx=0x7f30a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a4ff9640 1 -- 172.21.15.78:0/660511353 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0004b20 con 0x7f30a81098b0 2024-04-25T14:00:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a4ff9640 1 -- 172.21.15.78:0/660511353 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30a0004cc0 con 0x7f30a81098b0 2024-04-25T14:00:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a81a1fd0 con 0x7f30a81098b0 2024-04-25T14:00:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30a4ff9640 1 -- 172.21.15.78:0/660511353 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0010720 con 0x7f30a81098b0 2024-04-25T14:00:53.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:52.999+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a81a24e0 con 0x7f30a81098b0 2024-04-25T14:00:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.000+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a80ff710 con 0x7f30a81098b0 2024-04-25T14:00:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.001+0000 7f30a4ff9640 1 -- 172.21.15.78:0/660511353 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f30a0005560 con 0x7f30a81098b0 2024-04-25T14:00:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.002+0000 7f30a4ff9640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f30900afb70 0x7f30900b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.002+0000 7f30a77fe640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f30900afb70 0x7f30900b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.002+0000 7f30a4ff9640 1 -- 172.21.15.78:0/660511353 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f30a00df280 con 0x7f30a81098b0 2024-04-25T14:00:53.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.002+0000 7f30a77fe640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f30900afb70 0x7f30900b2080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f309c009230 tx=0x7f309c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.005+0000 7f30a4ff9640 1 -- 172.21.15.78:0/660511353 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30a00a7e50 con 0x7f30a81098b0 2024-04-25T14:00:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.220+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30a802db50 con 0x7f30a81098b0 2024-04-25T14:00:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.221+0000 7f30a4ff9640 1 -- 172.21.15.78:0/660511353 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f30a00a7750 con 0x7f30a81098b0 2024-04-25T14:00:53.221 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:00:53.221 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:00:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f30900afb70 msgr2=0x7f30900b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f30900afb70 0x7f30900b2080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f309c009230 tx=0x7f309c031400 comp rx=0 tx=0).stop 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81098b0 msgr2=0x7f30a81a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81098b0 0x7f30a81a16d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f30a000eb40 tx=0x7f30a000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 shutdown_connections 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f30900afb70 0x7f30900b2080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30a81098b0 0x7f30a81a16d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a8106f50 0x7f30a819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 --2- 172.21.15.78:0/660511353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30a81045f0 0x7f30a819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 >> 172.21.15.78:0/660511353 conn(0x7f30a80fdbf0 msgr2=0x7f30a8105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 shutdown_connections 2024-04-25T14:00:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.223+0000 7f30adf9d640 1 -- 172.21.15.78:0/660511353 wait complete. 2024-04-25T14:00:53.235 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:00:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 -- 172.21.15.78:0/2267556101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e01046a0 msgr2=0x7f84e010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 --2- 172.21.15.78:0/2267556101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e01046a0 0x7f84e010adf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009ca0 tx=0x7f84d802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 -- 172.21.15.78:0/2267556101 shutdown_connections 2024-04-25T14:00:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 --2- 172.21.15.78:0/2267556101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e01046a0 0x7f84e010adf0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 --2- 172.21.15.78:0/2267556101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84e0103b40 0x7f84e0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 --2- 172.21.15.78:0/2267556101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e0102fe0 0x7f84e01033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 -- 172.21.15.78:0/2267556101 >> 172.21.15.78:0/2267556101 conn(0x7f84e00fd380 msgr2=0x7f84e00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 -- 172.21.15.78:0/2267556101 shutdown_connections 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.345+0000 7f84e769a640 1 -- 172.21.15.78:0/2267556101 wait complete. 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 Processor -- start 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 -- start start 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e0102fe0 0x7f84e0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e0103b40 0x7f84e019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84e019f4a0 0x7f84e01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84e010f870 con 0x7f84e0103b40 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84e010f6f0 con 0x7f84e0102fe0 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e769a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84e010f9f0 con 0x7f84e019f4a0 2024-04-25T14:00:53.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e4c35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e0103b40 0x7f84e019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e4c35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e0103b40 0x7f84e019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59984/0 (socket says 172.21.15.78:59984) 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e4c35640 1 -- 172.21.15.78:0/3726668142 learned_addr learned my addr 172.21.15.78:0/3726668142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e5436640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e0102fe0 0x7f84e0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.346+0000 7f84e5c37640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84e019f4a0 0x7f84e01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e4c35640 1 -- 172.21.15.78:0/3726668142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84e019f4a0 msgr2=0x7f84e01a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e4c35640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84e019f4a0 0x7f84e01a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e4c35640 1 -- 172.21.15.78:0/3726668142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e0102fe0 msgr2=0x7f84e0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e4c35640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e0102fe0 0x7f84e0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e4c35640 1 -- 172.21.15.78:0/3726668142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e01a1f30 con 0x7f84e0103b40 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e5436640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e0102fe0 0x7f84e0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e4c35640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e0103b40 0x7f84e019a520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84d000d950 tx=0x7f84d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84d67fc640 1 -- 172.21.15.78:0/3726668142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0017030 con 0x7f84e0103b40 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84d67fc640 1 -- 172.21.15.78:0/3726668142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d0004020 con 0x7f84e0103b40 2024-04-25T14:00:53.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e01a2220 con 0x7f84e0103b40 2024-04-25T14:00:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84d67fc640 1 -- 172.21.15.78:0/3726668142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00047a0 con 0x7f84e0103b40 2024-04-25T14:00:53.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.347+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01a27c0 con 0x7f84e0103b40 2024-04-25T14:00:53.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84d67fc640 1 -- 172.21.15.78:0/3726668142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f84d0009a30 con 0x7f84e0103b40 2024-04-25T14:00:53.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 Processor -- start 2024-04-25T14:00:53.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 -- start start 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aab4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aab4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6f16b0 0x56164aab5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56164aa18d80 con 0x56164a6e94d0 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56164a9d2a20 con 0x56164a712390 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84ee620740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56164aa14d40 con 0x56164a6f16b0 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84e5436640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aab4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84e5c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aab4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84e5c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aab4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:45116/0 (socket says 172.21.15.78:45116) 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84e5c37640 1 -- 172.21.15.78:0/4051470352 learned_addr learned my addr 172.21.15.78:0/4051470352 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84e4c35640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6f16b0 0x56164aab5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84e5436640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aab4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59986/0 (socket says 172.21.15.78:59986) 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.350+0000 7f84d67fc640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f84cc0afb70 0x7f84cc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 -- 172.21.15.78:0/4051470352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6f16b0 msgr2=0x56164aab5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 --2- 172.21.15.78:0/4051470352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6f16b0 0x56164aab5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 -- 172.21.15.78:0/4051470352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 msgr2=0x56164aab4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 --2- 172.21.15.78:0/4051470352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aab4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 -- 172.21.15.78:0/4051470352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164a9ddc30 con 0x56164a712390 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84d67fc640 1 -- 172.21.15.78:0/3726668142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f84d00de5d0 con 0x7f84e0103b40 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f84cc0afb70 0x7f84cc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 --2- 172.21.15.78:0/4051470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aab4810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84d802f460 tx=0x7f84d802f490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d581f4ad37d2ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84cb7fe640 1 -- 172.21.15.78:0/4051470352 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8003e40 con 0x56164a712390 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84e5c37640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f84cc0afb70 0x7f84cc0b2080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f84d804c000 tx=0x7f84d8049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84cb7fe640 1 -- 172.21.15.78:0/4051470352 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d8036040 con 0x56164a712390 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.351+0000 7f84cb7fe640 1 -- 172.21.15.78:0/4051470352 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d802f9c0 con 0x56164a712390 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 -- 172.21.15.78:0/4051470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 msgr2=0x56164aab4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 --2- 172.21.15.78:0/4051470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aab4810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84d802f460 tx=0x7f84d802f490 comp rx=0 tx=0).stop 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 -- 172.21.15.78:0/4051470352 shutdown_connections 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 --2- 172.21.15.78:0/4051470352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6f16b0 0x56164aab5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 --2- 172.21.15.78:0/4051470352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aab4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 --2- 172.21.15.78:0/4051470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aab4810 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 -- 172.21.15.78:0/4051470352 >> 172.21.15.78:0/4051470352 conn(0x56164a7064e0 msgr2=0x56164aab0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 -- 172.21.15.78:0/4051470352 shutdown_connections 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 -- 172.21.15.78:0/4051470352 wait complete. 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 -- start start 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.352+0000 7f84ee620740 1 Processor -- start 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84ee620740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6e94d0 0x56164aabe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84ee620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aabe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84ee620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aac58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84ee620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56164aab63b0 con 0x56164a6f16b0 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84ee620740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56164a99fdc0 con 0x56164a712390 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84ee620740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56164aab6530 con 0x56164a6e94d0 2024-04-25T14:00:53.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84ee620740 10 client.0 _ll_register_callbacks cb 0x56164a9bb600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aabe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aabe990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60000/0 (socket says 172.21.15.78:60000) 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e5c37640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aac58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 -- 172.21.15.78:0/756998404 learned_addr learned my addr 172.21.15.78:0/756998404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e5436640 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6e94d0 0x56164aabe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 -- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6e94d0 msgr2=0x56164aabe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6e94d0 0x56164aabe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 -- 172.21.15.78:0/756998404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 msgr2=0x56164aac58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aac58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 -- 172.21.15.78:0/756998404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164a72a3f0 con 0x56164a6f16b0 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e5c37640 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aac58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84e4c35640 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aabe990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f84d00a3d10 tx=0x7f84d000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdbe8025d4b0c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.353+0000 7f84c8ff9640 1 -- 172.21.15.78:0/756998404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00a63f0 con 0x56164a6f16b0 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84c8ff9640 1 -- 172.21.15.78:0/756998404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f84d00a2d30 con 0x56164a6f16b0 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d8036040 con 0x56164a6f16b0 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84c8ff9640 1 -- 172.21.15.78:0/756998404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f84d00a6210 con 0x56164a6f16b0 2024-04-25T14:00:53.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 10 client.15822 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56164a7f47d0 con 0x56164a6f16b0 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84c8ff9640 1 -- 172.21.15.78:0/756998404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d8036040 con 0x56164a6f16b0 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84c8ff9640 1 -- 172.21.15.78:0/756998404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f84d00a5d00 con 0x56164a6f16b0 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 10 client.15822 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 20 client.15822 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6d0f60 0x56164aacc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 4 client.15822 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164aa26990 con 0x56164a6d0f60 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.354+0000 7f84e5436640 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6d0f60 0x56164aacc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.355+0000 7f84c8ff9640 10 client.15822 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.355+0000 7f84e5436640 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6d0f60 0x56164aacc190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a55ac0163c897f29 server_cookie=39971514f5769571 in_seq=0 out_seq=0 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.355+0000 7f84c8ff9640 1 -- 172.21.15.78:0/756998404 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56164aa26990 con 0x56164a6d0f60 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84c8ff9640 10 client.15822 handle_command_reply: tid=0 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 client.15822 shutdown 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 msgr2=0x56164aabe990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aabe990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f84d00a3d10 tx=0x7f84d000b530 comp rx=0 tx=0).stop 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 shutdown_connections 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6d0f60 0x56164aacc190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a712390 0x56164aac58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aabe990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 --2- 172.21.15.78:0/756998404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a6e94d0 0x56164aabe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 >> 172.21.15.78:0/756998404 conn(0x56164a7064e0 msgr2=0x56164aaaf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84c8ff9640 0 client.15822 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:53.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 shutdown_connections 2024-04-25T14:00:53.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 1 -- 172.21.15.78:0/756998404 wait complete. 2024-04-25T14:00:53.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.356+0000 7f84ee620740 20 client.15822 trim_cache size 0 max 16384 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.369+0000 7f84ee620740 1 Processor -- start 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.369+0000 7f84ee620740 1 -- start start 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84ee620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aad0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84ee620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6f16b0 0x56164aac5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84ee620740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aad0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84ee620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01adb50 con 0x56164a6e94d0 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84ee620740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01ab420 con 0x56164a6f16b0 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84ee620740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01a9270 con 0x56164a712390 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aad0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aad0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60016/0 (socket says 172.21.15.78:60016) 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 -- 172.21.15.78:0/2636890375 learned_addr learned my addr 172.21.15.78:0/2636890375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e4c35640 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6f16b0 0x56164aac5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5c37640 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aad0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 -- 172.21.15.78:0/2636890375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 msgr2=0x56164aad0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aad0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 -- 172.21.15.78:0/2636890375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6f16b0 msgr2=0x56164aac5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6f16b0 0x56164aac5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 -- 172.21.15.78:0/2636890375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164aab9d60 con 0x56164a6e94d0 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e4c35640 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6f16b0 0x56164aac5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84e5436640 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aad0940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f84e0103a40 tx=0x7f84dc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34ced32b1a02d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84caffd640 1 -- 172.21.15.78:0/2636890375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004220 con 0x56164a6e94d0 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84caffd640 1 -- 172.21.15.78:0/2636890375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84dc025030 con 0x56164a6e94d0 2024-04-25T14:00:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.370+0000 7f84caffd640 1 -- 172.21.15.78:0/2636890375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0214e0 con 0x56164a6e94d0 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 -- 172.21.15.78:0/2636890375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 msgr2=0x56164aad0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aad0940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f84e0103a40 tx=0x7f84dc0096c0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 -- 172.21.15.78:0/2636890375 shutdown_connections 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aad0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6f16b0 0x56164aac5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 --2- 172.21.15.78:0/2636890375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6e94d0 0x56164aad0940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 -- 172.21.15.78:0/2636890375 >> 172.21.15.78:0/2636890375 conn(0x56164a7064e0 msgr2=0x56164aabca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 -- 172.21.15.78:0/2636890375 shutdown_connections 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 -- 172.21.15.78:0/2636890375 wait complete. 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.371+0000 7f84ee620740 1 -- start start 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 1 Processor -- start 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6e94d0 0x56164aac2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aac3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aac4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01a98e0 con 0x56164a6f16b0 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01aad50 con 0x56164a6e94d0 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01a9fb0 con 0x56164a712390 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aac3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84ee620740 10 client.0 _ll_register_callbacks cb 0x7f84d00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aac3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60026/0 (socket says 172.21.15.78:60026) 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 -- 172.21.15.78:0/274966017 learned_addr learned my addr 172.21.15.78:0/274966017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e5436640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6e94d0 0x56164aac2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e5c37640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aac4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 -- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 msgr2=0x56164aac4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aac4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 -- 172.21.15.78:0/274966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6e94d0 msgr2=0x56164aac2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6e94d0 0x56164aac2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e4c35640 1 -- 172.21.15.78:0/274966017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164aac46f0 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e5c37640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aac4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.372+0000 7f84e5436640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6e94d0 0x56164aac2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84e4c35640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aac3200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f84d00af900 tx=0x7f84d0004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efb8a6fcc8f0e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84c8ff9640 1 -- 172.21.15.78:0/274966017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00a63f0 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84c8ff9640 1 -- 172.21.15.78:0/274966017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f84d00c92b0 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84dc025030 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84c8ff9640 1 -- 172.21.15.78:0/274966017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f84d00aaea0 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 10 client.15828 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56164aac5970 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84c8ff9640 1 -- 172.21.15.78:0/274966017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84dc025030 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84c8ff9640 1 -- 172.21.15.78:0/274966017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f84d00a2410 con 0x56164a6f16b0 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 10 client.15828 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 10 client.15828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:00:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 20 client.15828 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6a63a0 0x56164ab36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 4 client.15828 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.373+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56164aa20800 con 0x56164a6a63a0 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84e5436640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6a63a0 0x56164ab36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84e5436640 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6a63a0 0x56164ab36b40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3298a416be43f server_cookie=3eb010c4b77a6b8c in_seq=0 out_seq=0 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84c8ff9640 10 client.15828 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84c8ff9640 1 -- 172.21.15.78:0/274966017 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56164aa20800 con 0x56164a6a63a0 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84c8ff9640 10 client.15828 handle_command_reply: tid=0 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 client.15828 shutdown 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 msgr2=0x56164aac3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aac3200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f84d00af900 tx=0x7f84d0004400 comp rx=0 tx=0).stop 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 shutdown_connections 2024-04-25T14:00:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56164a6a63a0 0x56164ab36b40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56164a712390 0x56164aac4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84c8ff9640 0 client.15828 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56164a6f16b0 0x56164aac3200 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 --2- 172.21.15.78:0/274966017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56164a6e94d0 0x56164aac2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.374+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 >> 172.21.15.78:0/274966017 conn(0x56164a7064e0 msgr2=0x56164aab2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 shutdown_connections 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84ee620740 1 -- 172.21.15.78:0/274966017 wait complete. 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84ee620740 20 client.15828 trim_cache size 0 max 16384 2024-04-25T14:00:53.383 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:00:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f84cc0afb70 msgr2=0x7f84cc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f84cc0afb70 0x7f84cc0b2080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f84d804c000 tx=0x7f84d8049000 comp rx=0 tx=0).stop 2024-04-25T14:00:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e0103b40 msgr2=0x7f84e019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e0103b40 0x7f84e019a520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84d000d950 tx=0x7f84d000de10 comp rx=0 tx=0).stop 2024-04-25T14:00:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 shutdown_connections 2024-04-25T14:00:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84e019f4a0 0x7f84e01a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f84cc0afb70 0x7f84cc0b2080 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84e0103b40 0x7f84e019a520 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 --2- 172.21.15.78:0/3726668142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84e0102fe0 0x7f84e0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 >> 172.21.15.78:0/3726668142 conn(0x7f84e00fd380 msgr2=0x7f84e0106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 shutdown_connections 2024-04-25T14:00:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.375+0000 7f84e769a640 1 -- 172.21.15.78:0/3726668142 wait complete. 2024-04-25T14:00:53.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 -- 172.21.15.78:0/4197349741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 msgr2=0x7f52e010ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 --2- 172.21.15.78:0/4197349741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e010ad50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009ca0 tx=0x7f52d802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 -- 172.21.15.78:0/4197349741 shutdown_connections 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 --2- 172.21.15.78:0/4197349741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e010ad50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 --2- 172.21.15.78:0/4197349741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e0103c30 0x7f52e0104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 --2- 172.21.15.78:0/4197349741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e01029f0 0x7f52e01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 -- 172.21.15.78:0/4197349741 >> 172.21.15.78:0/4197349741 conn(0x7f52e00fdbc0 msgr2=0x7f52e0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 -- 172.21.15.78:0/4197349741 shutdown_connections 2024-04-25T14:00:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.496+0000 7f52e6405640 1 -- 172.21.15.78:0/4197349741 wait complete. 2024-04-25T14:00:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 Processor -- start 2024-04-25T14:00:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 -- start start 2024-04-25T14:00:53.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e01029f0 0x7f52e019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e0103c30 0x7f52e019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52e010e520 con 0x7f52e0108930 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52e010e3a0 con 0x7f52e0103c30 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e6405640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52e010e6a0 con 0x7f52e01029f0 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.497+0000 7f52e49a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e01a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60048/0 (socket says 172.21.15.78:60048) 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 -- 172.21.15.78:0/266947289 learned_addr learned my addr 172.21.15.78:0/266947289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52df7fe640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e0103c30 0x7f52e019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52dffff640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e01029f0 0x7f52e019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 -- 172.21.15.78:0/266947289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e01029f0 msgr2=0x7f52e019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e01029f0 0x7f52e019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 -- 172.21.15.78:0/266947289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e0103c30 msgr2=0x7f52e019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e0103c30 0x7f52e019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 -- 172.21.15.78:0/266947289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e01a1d50 con 0x7f52e0108930 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e49a2640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e01a16e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f52d80097b0 tx=0x7f52d8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52dd7fa640 1 -- 172.21.15.78:0/266947289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8003bf0 con 0x7f52e0108930 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e01a1fe0 con 0x7f52e0108930 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52dd7fa640 1 -- 172.21.15.78:0/266947289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d8003d90 con 0x7f52e0108930 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52dd7fa640 1 -- 172.21.15.78:0/266947289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d802fc30 con 0x7f52e0108930 2024-04-25T14:00:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.498+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e01a2440 con 0x7f52e0108930 2024-04-25T14:00:53.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dd7fa640 1 -- 172.21.15.78:0/266947289 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f52d8004040 con 0x7f52e0108930 2024-04-25T14:00:53.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 Processor -- start 2024-04-25T14:00:53.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 -- start start 2024-04-25T14:00:53.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589355c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589355da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5558934c0d80 con 0x5558931914d0 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55589347aa20 con 0x5558931ba390 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52ed392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5558934bcd40 con 0x5558931996b0 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60052/0 (socket says 172.21.15.78:60052) 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 -- 172.21.15.78:0/3774385493 learned_addr learned my addr 172.21.15.78:0/3774385493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52df7fe640 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589355c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52df7fe640 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589355da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 -- 172.21.15.78:0/3774385493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 msgr2=0x55589355da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589355da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 -- 172.21.15.78:0/3774385493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 msgr2=0x55589355c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589355c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52dffff640 1 -- 172.21.15.78:0/3774385493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555893485c30 con 0x5558931914d0 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.501+0000 7f52df7fe640 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589355da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.502+0000 7f52dd7fa640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f52c80afaf0 0x7f52c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.502+0000 7f52dffff640 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355cd90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f52d000bc70 tx=0x7f52d00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6a5617eacd97f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.502+0000 7f52df7fe640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f52c80afaf0 0x7f52c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.502+0000 7f52dd7fa640 1 -- 172.21.15.78:0/266947289 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f52d81002b0 con 0x7f52e0108930 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.502+0000 7f52c67fc640 1 -- 172.21.15.78:0/3774385493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d001c680 con 0x5558931914d0 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.502+0000 7f52c67fc640 1 -- 172.21.15.78:0/3774385493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d000be20 con 0x5558931914d0 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.502+0000 7f52c67fc640 1 -- 172.21.15.78:0/3774385493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0011b20 con 0x5558931914d0 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 -- 172.21.15.78:0/3774385493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 msgr2=0x55589355cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355cd90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f52d000bc70 tx=0x7f52d00107a0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52df7fe640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f52c80afaf0 0x7f52c80b2000 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55589355d860 tx=0x7f52d4004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 -- 172.21.15.78:0/3774385493 shutdown_connections 2024-04-25T14:00:53.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589355da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355cd90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 --2- 172.21.15.78:0/3774385493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589355c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 -- 172.21.15.78:0/3774385493 >> 172.21.15.78:0/3774385493 conn(0x5558931ae4e0 msgr2=0x555893558020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 -- 172.21.15.78:0/3774385493 shutdown_connections 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 -- 172.21.15.78:0/3774385493 wait complete. 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 -- start start 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.503+0000 7f52ed392740 1 Processor -- start 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x5558935665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x555893566af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589356d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55589355e530 con 0x5558931914d0 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555893447dc0 con 0x5558931ba390 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55589355e6b0 con 0x5558931996b0 2024-04-25T14:00:53.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 10 client.0 _ll_register_callbacks cb 0x555893463600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x5558935665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x5558935665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60066/0 (socket says 172.21.15.78:60066) 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 -- 172.21.15.78:0/3181887335 learned_addr learned my addr 172.21.15.78:0/3181887335 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52e49a2640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589356d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52df7fe640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x555893566af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 -- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 msgr2=0x555893566af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x555893566af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 -- 172.21.15.78:0/3181887335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 msgr2=0x55589356d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589356d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 -- 172.21.15.78:0/3181887335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558931d23f0 con 0x5558931914d0 2024-04-25T14:00:53.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52df7fe640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x555893566af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52e49a2640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589356d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52dffff640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x5558935665b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f52d0011010 tx=0x7f52d0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7183615a0acceeb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52bf7fe640 1 -- 172.21.15.78:0/3181887335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0011a20 con 0x5558931914d0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52bf7fe640 1 -- 172.21.15.78:0/3181887335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f52d0011bc0 con 0x5558931914d0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.504+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d000be20 con 0x5558931914d0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52bf7fe640 1 -- 172.21.15.78:0/3181887335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f52d0011dc0 con 0x5558931914d0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 10 client.15840 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55589329c7d0 con 0x5558931914d0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52bf7fe640 1 -- 172.21.15.78:0/3181887335 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d000be20 con 0x5558931914d0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52bf7fe640 1 -- 172.21.15.78:0/3181887335 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f52d002e020 con 0x5558931914d0 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 10 client.15840 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 10 client.15840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:00:53.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 20 client.15840 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x555893178f60 0x555893573f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 4 client.15840 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558934ce990 con 0x555893178f60 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.505+0000 7f52df7fe640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x555893178f60 0x555893573f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.506+0000 7f52df7fe640 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x555893178f60 0x555893573f40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f02d53e29f26a18 server_cookie=f4254ebdaa1787d8 in_seq=0 out_seq=0 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.506+0000 7f52bf7fe640 10 client.15840 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52bf7fe640 1 -- 172.21.15.78:0/3181887335 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558934ce990 con 0x555893178f60 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52bf7fe640 10 client.15840 handle_command_reply: tid=0 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 client.15840 shutdown 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 msgr2=0x5558935665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x5558935665b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f52d0011010 tx=0x7f52d0005270 comp rx=0 tx=0).stop 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 shutdown_connections 2024-04-25T14:00:53.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x55589356d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x555893178f60 0x555893573f40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x555893566af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 --2- 172.21.15.78:0/3181887335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x5558935665b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52bf7fe640 0 client.15840 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 >> 172.21.15.78:0/3181887335 conn(0x5558931ae4e0 msgr2=0x55589355af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 shutdown_connections 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 1 -- 172.21.15.78:0/3181887335 wait complete. 2024-04-25T14:00:53.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.507+0000 7f52ed392740 20 client.15840 trim_cache size 0 max 16384 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.520+0000 7f52ed392740 1 Processor -- start 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.520+0000 7f52ed392740 1 -- start start 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52ed392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52ed392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589356d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52ed392740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x555893579670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52ed392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01adad0 con 0x5558931914d0 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52ed392740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01ab3a0 con 0x5558931ba390 2024-04-25T14:00:53.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52ed392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01a91f0 con 0x5558931996b0 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60076/0 (socket says 172.21.15.78:60076) 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 -- 172.21.15.78:0/1469210955 learned_addr learned my addr 172.21.15.78:0/1469210955 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52e49a2640 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x555893579670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 -- 172.21.15.78:0/1469210955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 msgr2=0x55589356d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589356d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 -- 172.21.15.78:0/1469210955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 msgr2=0x555893579670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x555893579670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 -- 172.21.15.78:0/1469210955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555893560dc0 con 0x5558931914d0 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52e49a2640 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x555893579670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52dffff640 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355b530 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f52d0001eb0 tx=0x7f52d000b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92a6bdc1b77be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52c5ffb640 1 -- 172.21.15.78:0/1469210955 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d000be20 con 0x5558931914d0 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52c5ffb640 1 -- 172.21.15.78:0/1469210955 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d0011a60 con 0x5558931914d0 2024-04-25T14:00:53.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.521+0000 7f52c5ffb640 1 -- 172.21.15.78:0/1469210955 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0004930 con 0x5558931914d0 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 -- 172.21.15.78:0/1469210955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 msgr2=0x55589355b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355b530 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f52d0001eb0 tx=0x7f52d000b9a0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 -- 172.21.15.78:0/1469210955 shutdown_connections 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931ba390 0x555893579670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931996b0 0x55589356d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 --2- 172.21.15.78:0/1469210955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931914d0 0x55589355b530 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 -- 172.21.15.78:0/1469210955 >> 172.21.15.78:0/1469210955 conn(0x5558931ae4e0 msgr2=0x55589355ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 -- 172.21.15.78:0/1469210955 shutdown_connections 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 -- 172.21.15.78:0/1469210955 wait complete. 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.522+0000 7f52ed392740 1 -- start start 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 1 Processor -- start 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931914d0 0x55589356cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931996b0 0x55589356d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931ba390 0x5558935da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01a9860 con 0x5558931ba390 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01aacd0 con 0x5558931996b0 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52e01a9f30 con 0x5558931914d0 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52ed392740 10 client.0 _ll_register_callbacks cb 0x7f52d4009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931ba390 0x5558935da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931ba390 0x5558935da430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60080/0 (socket says 172.21.15.78:60080) 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 -- 172.21.15.78:0/434604047 learned_addr learned my addr 172.21.15.78:0/434604047 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52df7fe640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931996b0 0x55589356d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52dffff640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931914d0 0x55589356cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 -- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931914d0 msgr2=0x55589356cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931914d0 0x55589356cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 -- 172.21.15.78:0/434604047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931996b0 msgr2=0x55589356d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931996b0 0x55589356d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52e49a2640 1 -- 172.21.15.78:0/434604047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589356de40 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.523+0000 7f52dffff640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931914d0 0x55589356cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52e49a2640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931ba390 0x5558935da430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f52d80c4800 tx=0x7f52d802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8ff8fc6e089408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52bf7fe640 1 -- 172.21.15.78:0/434604047 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80e2060 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52bf7fe640 1 -- 172.21.15.78:0/434604047 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f52d80046a0 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d0011a60 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52bf7fe640 1 -- 172.21.15.78:0/434604047 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f52d80e8070 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 10 client.15846 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5558935dbba0 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52bf7fe640 1 -- 172.21.15.78:0/434604047 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d0011a60 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52bf7fe640 1 -- 172.21.15.78:0/434604047 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f52d80c4a10 con 0x5558931ba390 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 10 client.15846 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 10 client.15846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:00:53.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 20 client.15846 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55589314e3a0 0x5558935de590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 4 client.15846 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558934c8800 con 0x55589314e3a0 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.524+0000 7f52dffff640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55589314e3a0 0x5558935de590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52dffff640 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55589314e3a0 0x5558935de590 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a13ebd86b7214ef7 server_cookie=ea6a195e0daa31c2 in_seq=0 out_seq=0 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52bf7fe640 10 client.15846 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52bf7fe640 1 -- 172.21.15.78:0/434604047 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558934c8800 con 0x55589314e3a0 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52bf7fe640 10 client.15846 handle_command_reply: tid=0 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 client.15846 shutdown 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931ba390 msgr2=0x5558935da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931ba390 0x5558935da430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f52d80c4800 tx=0x7f52d802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 shutdown_connections 2024-04-25T14:00:53.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5558931ba390 0x5558935da430 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558931996b0 0x55589356d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55589314e3a0 0x5558935de590 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 --2- 172.21.15.78:0/434604047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558931914d0 0x55589356cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52bf7fe640 0 client.15846 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.525+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 >> 172.21.15.78:0/434604047 conn(0x5558931ae4e0 msgr2=0x5558935660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 shutdown_connections 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52ed392740 1 -- 172.21.15.78:0/434604047 wait complete. 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52ed392740 20 client.15846 trim_cache size 0 max 16384 2024-04-25T14:00:53.532 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:00:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f52c80afaf0 msgr2=0x7f52c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f52c80afaf0 0x7f52c80b2000 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55589355d860 tx=0x7f52d4004730 comp rx=0 tx=0).stop 2024-04-25T14:00:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 msgr2=0x7f52e01a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e01a16e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f52d80097b0 tx=0x7f52d8031840 comp rx=0 tx=0).stop 2024-04-25T14:00:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 shutdown_connections 2024-04-25T14:00:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f52c80afaf0 0x7f52c80b2000 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52e0108930 0x7f52e01a16e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e0103c30 0x7f52e019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 --2- 172.21.15.78:0/266947289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52e01029f0 0x7f52e019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 >> 172.21.15.78:0/266947289 conn(0x7f52e00fdbc0 msgr2=0x7f52e0074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 shutdown_connections 2024-04-25T14:00:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.526+0000 7f52e6405640 1 -- 172.21.15.78:0/266947289 wait complete. 2024-04-25T14:00:53.536 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:53 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/660511353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:53.537 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:00:53.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 -- 172.21.15.78:0/1116446866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 msgr2=0x7f0a641036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1116446866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641036f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009ca0 tx=0x7f0a5402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 -- 172.21.15.78:0/1116446866 shutdown_connections 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1116446866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64108930 0x7f0a6410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1116446866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a64103c30 0x7f0a64104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1116446866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641036f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 -- 172.21.15.78:0/1116446866 >> 172.21.15.78:0/1116446866 conn(0x7f0a640fdbc0 msgr2=0x7f0a64100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 -- 172.21.15.78:0/1116446866 shutdown_connections 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.646+0000 7f0a6951f640 1 -- 172.21.15.78:0/1116446866 wait complete. 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 Processor -- start 2024-04-25T14:00:53.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 -- start start 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a64103c30 0x7f0a64199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64108930 0x7f0a641a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410e4f0 con 0x7f0a641029f0 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410e370 con 0x7f0a64103c30 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.647+0000 7f0a6951f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410e670 con 0x7f0a64108930 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60102/0 (socket says 172.21.15.78:60102) 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1210075384 learned_addr learned my addr 172.21.15.78:0/1210075384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a627fc640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a64103c30 0x7f0a64199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1210075384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64108930 msgr2=0x7f0a641a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64108930 0x7f0a641a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1210075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a64103c30 msgr2=0x7f0a64199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a64103c30 0x7f0a64199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1210075384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a641a0c90 con 0x7f0a641029f0 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641991c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0a5402ade0 tx=0x7f0a540316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a4bfff640 1 -- 172.21.15.78:0/1210075384 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54003b70 con 0x7f0a641029f0 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a4bfff640 1 -- 172.21.15.78:0/1210075384 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54003d10 con 0x7f0a641029f0 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a641a0f20 con 0x7f0a641029f0 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a4bfff640 1 -- 172.21.15.78:0/1210075384 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5402fb90 con 0x7f0a641029f0 2024-04-25T14:00:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.648+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a641a1380 con 0x7f0a641029f0 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.650+0000 7f0a4bfff640 1 -- 172.21.15.78:0/1210075384 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0a5402fd30 con 0x7f0a641029f0 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 Processor -- start 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 -- start start 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b776390 0x55999bb1aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb1afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b7556b0 0x55999bb1bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55999ba7cd80 con 0x55999b74d4d0 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55999ba36a20 con 0x55999b7556b0 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a703d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55999ba78d40 con 0x55999b776390 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb1afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb1afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60118/0 (socket says 172.21.15.78:60118) 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1634740157 learned_addr learned my addr 172.21.15.78:0/1634740157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a627fc640 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b7556b0 0x55999bb1bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a637fe640 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b776390 0x55999bb1aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1634740157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b776390 msgr2=0x55999bb1aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b776390 0x55999bb1aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1634740157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b7556b0 msgr2=0x55999bb1bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a4bfff640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a4c0afaf0 0x7f0a4c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b7556b0 0x55999bb1bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1634740157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999ba41c30 con 0x55999b74d4d0 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a637fe640 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b776390 0x55999bb1aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a637fe640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a4c0afaf0 0x7f0a4c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.651+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb1afa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0a540cb930 tx=0x7f0a540c4d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfb45247f046ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.652+0000 7f0a4bfff640 1 -- 172.21.15.78:0/1210075384 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0a54100120 con 0x7f0a641029f0 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.652+0000 7f0a497fa640 1 -- 172.21.15.78:0/1634740157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540c83c0 con 0x55999b74d4d0 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.652+0000 7f0a497fa640 1 -- 172.21.15.78:0/1634740157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54036020 con 0x55999b74d4d0 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.652+0000 7f0a497fa640 1 -- 172.21.15.78:0/1634740157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540cb9c0 con 0x55999b74d4d0 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.652+0000 7f0a637fe640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a4c0afaf0 0x7f0a4c0b2000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c005d00 tx=0x7f0a5c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.652+0000 7f0a703d1740 1 -- 172.21.15.78:0/1634740157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 msgr2=0x55999bb1afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb1afa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0a540cb930 tx=0x7f0a540c4d00 comp rx=0 tx=0).stop 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- 172.21.15.78:0/1634740157 shutdown_connections 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b7556b0 0x55999bb1bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb1afa0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1634740157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b776390 0x55999bb1aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- 172.21.15.78:0/1634740157 >> 172.21.15.78:0/1634740157 conn(0x55999b76a4e0 msgr2=0x55999bb14020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- 172.21.15.78:0/1634740157 shutdown_connections 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- 172.21.15.78:0/1634740157 wait complete. 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- start start 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 Processor -- start 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b74d4d0 0x55999bb242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b7556b0 0x55999bb24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55999ba7cd80 con 0x55999b7556b0 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55999ba03dc0 con 0x55999b776390 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55999bb1c500 con 0x55999b74d4d0 2024-04-25T14:00:53.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a703d1740 10 client.0 _ll_register_callbacks cb 0x55999ba1f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b7556b0 0x55999bb24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b7556b0 0x55999bb24800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60132/0 (socket says 172.21.15.78:60132) 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a627fc640 1 -- 172.21.15.78:0/3722285105 learned_addr learned my addr 172.21.15.78:0/3722285105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.653+0000 7f0a637fe640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b74d4d0 0x55999bb242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a627fc640 1 -- 172.21.15.78:0/3722285105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b74d4d0 msgr2=0x55999bb242c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a627fc640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b74d4d0 0x55999bb242c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a627fc640 1 -- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 msgr2=0x55999bb2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a627fc640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a627fc640 1 -- 172.21.15.78:0/3722285105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999b78e3f0 con 0x55999b7556b0 2024-04-25T14:00:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a637fe640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a627fc640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b7556b0 0x55999bb24800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0a58012720 tx=0x7f0a58012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5c987eb6a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a46ff5640 1 -- 172.21.15.78:0/3722285105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800f920 con 0x55999b7556b0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a46ff5640 1 -- 172.21.15.78:0/3722285105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0a5800fac0 con 0x55999b7556b0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a54036020 con 0x55999b7556b0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a46ff5640 1 -- 172.21.15.78:0/3722285105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0a5800fcc0 con 0x55999b7556b0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a703d1740 10 client.15858 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55999b8587d0 con 0x55999b7556b0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.654+0000 7f0a46ff5640 1 -- 172.21.15.78:0/3722285105 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54036020 con 0x55999b7556b0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a46ff5640 1 -- 172.21.15.78:0/3722285105 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0a58025050 con 0x55999b7556b0 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a703d1740 10 client.15858 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a703d1740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a703d1740 20 client.15858 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a703d1740 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b734f60 0x55999bb32030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a703d1740 4 client.15858 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55999ba8a990 con 0x55999b734f60 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b734f60 0x55999bb32030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a46ff5640 10 client.15858 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.655+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b734f60 0x55999bb32030 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb39fb4caa5e8d08 server_cookie=f33d8771cc1c4898 in_seq=0 out_seq=0 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a46ff5640 1 -- 172.21.15.78:0/3722285105 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55999ba8a990 con 0x55999b734f60 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a46ff5640 10 client.15858 handle_command_reply: tid=0 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 client.15858 shutdown 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b7556b0 msgr2=0x55999bb24800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b7556b0 0x55999bb24800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0a58012720 tx=0x7f0a58012be0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 shutdown_connections 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b7556b0 0x55999bb24800 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b734f60 0x55999bb32030 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 --2- 172.21.15.78:0/3722285105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b74d4d0 0x55999bb242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a46ff5640 0 client.15858 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 >> 172.21.15.78:0/3722285105 conn(0x55999b76a4e0 msgr2=0x55999bb13b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 shutdown_connections 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.656+0000 7f0a703d1740 1 -- 172.21.15.78:0/3722285105 wait complete. 2024-04-25T14:00:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.657+0000 7f0a703d1740 20 client.15858 trim_cache size 0 max 16384 2024-04-25T14:00:53.667 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:53 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/660511353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 Processor -- start 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 -- start start 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb1cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641ab990 con 0x55999b74d4d0 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641a9260 con 0x55999b776390 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a703d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641a70b0 con 0x55999b7556b0 2024-04-25T14:00:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb228e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60144/0 (socket says 172.21.15.78:60144) 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1714727305 learned_addr learned my addr 172.21.15.78:0/1714727305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a637fe640 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb1cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a627fc640 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1714727305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 msgr2=0x55999bb22e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb22e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1714727305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 msgr2=0x55999bb1cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb1cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a62ffd640 1 -- 172.21.15.78:0/1714727305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999bb1c640 con 0x55999b74d4d0 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.670+0000 7f0a627fc640 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb22e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb228e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a540094e0 tx=0x7f0a54103c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bd73ceb107203c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a48ff9640 1 -- 172.21.15.78:0/1714727305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540cbc20 con 0x55999b74d4d0 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a48ff9640 1 -- 172.21.15.78:0/1714727305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54036070 con 0x55999b74d4d0 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a48ff9640 1 -- 172.21.15.78:0/1714727305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540c7d60 con 0x55999b74d4d0 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a703d1740 1 -- 172.21.15.78:0/1714727305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 msgr2=0x55999bb228e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb228e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a540094e0 tx=0x7f0a54103c00 comp rx=0 tx=0).stop 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a703d1740 1 -- 172.21.15.78:0/1714727305 shutdown_connections 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b776390 0x55999bb1cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb22e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a703d1740 1 --2- 172.21.15.78:0/1714727305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b74d4d0 0x55999bb228e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.671+0000 7f0a703d1740 1 -- 172.21.15.78:0/1714727305 >> 172.21.15.78:0/1714727305 conn(0x55999b76a4e0 msgr2=0x55999bb1a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 -- 172.21.15.78:0/1714727305 shutdown_connections 2024-04-25T14:00:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 -- 172.21.15.78:0/1714727305 wait complete. 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 -- start start 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 Processor -- start 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b74d4d0 0x55999bb194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb19a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b776390 0x55999bb9a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641a7720 con 0x55999b776390 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641a8b90 con 0x55999b74d4d0 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641a7df0 con 0x55999b7556b0 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a703d1740 10 client.0 _ll_register_callbacks cb 0x7f0a58009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b776390 0x55999bb9a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b776390 0x55999bb9a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60154/0 (socket says 172.21.15.78:60154) 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a62ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b74d4d0 0x55999bb194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a637fe640 1 -- 172.21.15.78:0/2609391906 learned_addr learned my addr 172.21.15.78:0/2609391906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.672+0000 7f0a627fc640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb19a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a637fe640 1 -- 172.21.15.78:0/2609391906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 msgr2=0x55999bb19a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a637fe640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb19a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a637fe640 1 -- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b74d4d0 msgr2=0x55999bb194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a637fe640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b74d4d0 0x55999bb194f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a637fe640 1 -- 172.21.15.78:0/2609391906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999bb1fbd0 con 0x55999b776390 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a627fc640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb19a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b74d4d0 0x55999bb194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a637fe640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b776390 0x55999bb9a5b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c011d30 tx=0x7f0a5c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50828938123029d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:53.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a46ff5640 1 -- 172.21.15.78:0/2609391906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c029640 con 0x55999b776390 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a46ff5640 1 -- 172.21.15.78:0/2609391906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0a5c0283e0 con 0x55999b776390 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a54036070 con 0x55999b776390 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a46ff5640 1 -- 172.21.15.78:0/2609391906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0a5c011e20 con 0x55999b776390 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 10 client.15864 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55999bb9bd20 con 0x55999b776390 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a46ff5640 1 -- 172.21.15.78:0/2609391906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a54036070 con 0x55999b776390 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a46ff5640 1 -- 172.21.15.78:0/2609391906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0a5c032050 con 0x55999b776390 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 10 client.15864 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 20 client.15864 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b70a3a0 0x55999bb9e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 4 client.15864 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.673+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55999ba84800 con 0x55999b70a3a0 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b70a3a0 0x55999bb9e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a46ff5640 10 client.15864 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a62ffd640 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b70a3a0 0x55999bb9e740 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2556cbbcb5590396 server_cookie=2463ce54176f48e8 in_seq=0 out_seq=0 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a46ff5640 1 -- 172.21.15.78:0/2609391906 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55999ba84800 con 0x55999b70a3a0 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a46ff5640 10 client.15864 handle_command_reply: tid=0 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 client.15864 shutdown 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b776390 msgr2=0x55999bb9a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b776390 0x55999bb9a5b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c011d30 tx=0x7f0a5c005c90 comp rx=0 tx=0).stop 2024-04-25T14:00:53.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 shutdown_connections 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55999b776390 0x55999bb9a5b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55999b70a3a0 0x55999bb9e740 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55999b7556b0 0x55999bb19a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 --2- 172.21.15.78:0/2609391906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55999b74d4d0 0x55999bb194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a46ff5640 0 client.15864 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 >> 172.21.15.78:0/2609391906 conn(0x55999b76a4e0 msgr2=0x55999bb22810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 shutdown_connections 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 1 -- 172.21.15.78:0/2609391906 wait complete. 2024-04-25T14:00:53.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.674+0000 7f0a703d1740 20 client.15864 trim_cache size 0 max 16384 2024-04-25T14:00:53.684 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a4c0afaf0 msgr2=0x7f0a4c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a4c0afaf0 0x7f0a4c0b2000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c005d00 tx=0x7f0a5c005ef0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 msgr2=0x7f0a641991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641991c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0a5402ade0 tx=0x7f0a540316e0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 shutdown_connections 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64108930 0x7f0a641a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a64103c30 0x7f0a64199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a4c0afaf0 0x7f0a4c0b2000 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 --2- 172.21.15.78:0/1210075384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a641029f0 0x7f0a641991c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 >> 172.21.15.78:0/1210075384 conn(0x7f0a640fdbc0 msgr2=0x7f0a64104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:53.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 shutdown_connections 2024-04-25T14:00:53.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:53.675+0000 7f0a6951f640 1 -- 172.21.15.78:0/1210075384 wait complete. 2024-04-25T14:00:53.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:53 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/660511353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:00:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:54 smithi129 ceph-mon[32569]: pgmap v390: 97 pgs: 97 active+clean; 777 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:00:54.687 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:00:54.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:00:54.687 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:00:54.712 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:54 smithi078 ceph-mon[29163]: pgmap v390: 97 pgs: 97 active+clean; 777 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:00:54.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:54 smithi112 ceph-mon[32617]: pgmap v390: 97 pgs: 97 active+clean; 777 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/2274126019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64109a30 msgr2=0x7f2f6410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/2274126019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64109a30 0x7f2f6410be50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009ca0 tx=0x7f2f5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/2274126019 shutdown_connections 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/2274126019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64109a30 0x7f2f6410be50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/2274126019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f64104d30 0x7f2f64105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/2274126019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f64103af0 0x7f2f641047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/2274126019 >> 172.21.15.78:0/2274126019 conn(0x7f2f640fece0 msgr2=0x7f2f64101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:54.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/2274126019 shutdown_connections 2024-04-25T14:00:54.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/2274126019 wait complete. 2024-04-25T14:00:54.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 Processor -- start 2024-04-25T14:00:54.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.814+0000 7f2f6a4b6640 1 -- start start 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f6a4b6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f64103af0 0x7f2f6419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f6a4b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64104d30 0x7f2f6419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f6a4b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f64109a30 0x7f2f641a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f6a4b6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6410f620 con 0x7f2f64104d30 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f6a4b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6410f4a0 con 0x7f2f64109a30 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f6a4b6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6410f7a0 con 0x7f2f64103af0 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64104d30 0x7f2f6419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64104d30 0x7f2f6419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60166/0 (socket says 172.21.15.78:60166) 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 -- 172.21.15.78:0/68262683 learned_addr learned my addr 172.21.15.78:0/68262683 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f63fff640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f64103af0 0x7f2f6419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f68a53640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f64109a30 0x7f2f641a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 -- 172.21.15.78:0/68262683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f64103af0 msgr2=0x7f2f6419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f64103af0 0x7f2f6419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 -- 172.21.15.78:0/68262683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f64109a30 msgr2=0x7f2f641a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f64109a30 0x7f2f641a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 -- 172.21.15.78:0/68262683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f641a1d30 con 0x7f2f64104d30 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.815+0000 7f2f637fe640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64104d30 0x7f2f6419a730 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800d770 tx=0x7f2f5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.816+0000 7f2f617fa640 1 -- 172.21.15.78:0/68262683 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58017070 con 0x7f2f64104d30 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.816+0000 7f2f617fa640 1 -- 172.21.15.78:0/68262683 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f580054d0 con 0x7f2f64104d30 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.816+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f641a1fc0 con 0x7f2f64104d30 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.816+0000 7f2f617fa640 1 -- 172.21.15.78:0/68262683 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58004c30 con 0x7f2f64104d30 2024-04-25T14:00:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.816+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f641a2460 con 0x7f2f64104d30 2024-04-25T14:00:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.818+0000 7f2f7136a740 1 Processor -- start 2024-04-25T14:00:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.818+0000 7f2f617fa640 1 -- 172.21.15.78:0/68262683 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2f58004e10 con 0x7f2f64104d30 2024-04-25T14:00:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.818+0000 7f2f7136a740 1 -- start start 2024-04-25T14:00:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x5653445284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344528a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x56534452f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f7136a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56534448db00 con 0x565344187410 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f7136a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565344429920 con 0x56534415e550 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f7136a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565344489ac0 con 0x565344166730 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x5653445284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x5653445284a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60176/0 (socket says 172.21.15.78:60176) 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 -- 172.21.15.78:0/628887185 learned_addr learned my addr 172.21.15.78:0/628887185 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f637fe640 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x56534452f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f63fff640 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344528a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 -- 172.21.15.78:0/628887185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 msgr2=0x56534452f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x56534452f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 -- 172.21.15.78:0/628887185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 msgr2=0x565344528a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344528a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 -- 172.21.15.78:0/628887185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653442677e0 con 0x565344187410 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f637fe640 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x56534452f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f617fa640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2f4c0afaf0 0x7f2f4c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f68a53640 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x5653445284a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c02f440 tx=0x7f2f5c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873fd2f8c16a7b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.819+0000 7f2f63fff640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2f4c0afaf0 0x7f2f4c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.820+0000 7f2f617fa640 1 -- 172.21.15.78:0/68262683 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2f580de6f0 con 0x7f2f64104d30 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.820+0000 7f2f4a7fc640 1 -- 172.21.15.78:0/628887185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c031650 con 0x565344187410 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.820+0000 7f2f4a7fc640 1 -- 172.21.15.78:0/628887185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c036040 con 0x565344187410 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.820+0000 7f2f4a7fc640 1 -- 172.21.15.78:0/628887185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0046e0 con 0x565344187410 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 -- 172.21.15.78:0/628887185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 msgr2=0x5653445284a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f63fff640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2f4c0afaf0 0x7f2f4c0b2000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x565344529880 tx=0x7f2f5400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x5653445284a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c02f440 tx=0x7f2f5c02f470 comp rx=0 tx=0).stop 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 -- 172.21.15.78:0/628887185 shutdown_connections 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x56534452f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344528a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 --2- 172.21.15.78:0/628887185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x5653445284a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 -- 172.21.15.78:0/628887185 >> 172.21.15.78:0/628887185 conn(0x56534417b560 msgr2=0x565344523e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 -- 172.21.15.78:0/628887185 shutdown_connections 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 -- 172.21.15.78:0/628887185 wait complete. 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 -- start start 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 Processor -- start 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.821+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x5653445322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344166730 0x565344532800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344539760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565344530310 con 0x565344166730 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5653444275d0 con 0x56534415e550 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565344530490 con 0x565344187410 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 10 client.0 _ll_register_callbacks cb 0x565344415080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344166730 0x565344532800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344166730 0x565344532800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60178/0 (socket says 172.21.15.78:60178) 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 -- 172.21.15.78:0/2675344951 learned_addr learned my addr 172.21.15.78:0/2675344951 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f63fff640 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x5653445322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f68a53640 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344539760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 -- 172.21.15.78:0/2675344951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 msgr2=0x565344539760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344539760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 -- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 msgr2=0x5653445322c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x5653445322c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 -- 172.21.15.78:0/2675344951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653442c7f70 con 0x565344166730 2024-04-25T14:00:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f637fe640 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344166730 0x565344532800 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2f580a3e30 tx=0x7f2f580a65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfc5d3dceaa0e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f437fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f437fe640 1 -- 172.21.15.78:0/2675344951 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58020050 con 0x565344166730 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f437fe640 1 -- 172.21.15.78:0/2675344951 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2f580af590 con 0x565344166730 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f5c036040 con 0x565344166730 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f437fe640 1 -- 172.21.15.78:0/2675344951 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2f580a66d0 con 0x565344166730 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 10 client.15876 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.822+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56534415f170 con 0x565344166730 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f437fe640 1 -- 172.21.15.78:0/2675344951 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c036040 con 0x565344166730 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f437fe640 1 -- 172.21.15.78:0/2675344951 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2f580af040 con 0x565344166730 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f7136a740 10 client.15876 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f7136a740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f7136a740 20 client.15876 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5653441461f0 0x56534453fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f7136a740 4 client.15876 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534449b710 con 0x5653441461f0 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.823+0000 7f2f63fff640 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5653441461f0 0x56534453fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.824+0000 7f2f437fe640 10 client.15876 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.824+0000 7f2f63fff640 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5653441461f0 0x56534453fcc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572f7532fcb259c5 server_cookie=e17d149aafb6bd97 in_seq=0 out_seq=0 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.824+0000 7f2f437fe640 1 -- 172.21.15.78:0/2675344951 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56534449b710 con 0x5653441461f0 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.824+0000 7f2f437fe640 10 client.15876 handle_command_reply: tid=0 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.824+0000 7f2f7136a740 1 client.15876 shutdown 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344166730 msgr2=0x565344532800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344166730 0x565344532800 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2f580a3e30 tx=0x7f2f580a65d0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 shutdown_connections 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5653441461f0 0x56534453fcc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344539760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344166730 0x565344532800 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f437fe640 0 client.15876 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2675344951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x5653445322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 >> 172.21.15.78:0/2675344951 conn(0x56534417b560 msgr2=0x565344522ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 shutdown_connections 2024-04-25T14:00:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 1 -- 172.21.15.78:0/2675344951 wait complete. 2024-04-25T14:00:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.825+0000 7f2f7136a740 20 client.15876 trim_cache size 0 max 16384 2024-04-25T14:00:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 Processor -- start 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 -- start start 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344530f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x565344523620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x565344523b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641adb90 con 0x565344187410 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641ab460 con 0x56534415e550 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f7136a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641a92b0 con 0x565344166730 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x565344523b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x565344523b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60184/0 (socket says 172.21.15.78:60184) 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 -- 172.21.15.78:0/749752158 learned_addr learned my addr 172.21.15.78:0/749752158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f637fe640 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x565344523620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f63fff640 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344530f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 -- 172.21.15.78:0/749752158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 msgr2=0x565344523620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x565344523620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 -- 172.21.15.78:0/749752158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 msgr2=0x565344530f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344530f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.843+0000 7f2f68a53640 1 -- 172.21.15.78:0/749752158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534452d1a0 con 0x565344187410 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.844+0000 7f2f68a53640 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x565344523b60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c02ade0 tx=0x7f2f5c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efb1685111c4b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.844+0000 7f2f49ffb640 1 -- 172.21.15.78:0/749752158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c052070 con 0x565344187410 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.844+0000 7f2f49ffb640 1 -- 172.21.15.78:0/749752158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c036070 con 0x565344187410 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.844+0000 7f2f49ffb640 1 -- 172.21.15.78:0/749752158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c004460 con 0x565344187410 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.844+0000 7f2f7136a740 1 -- 172.21.15.78:0/749752158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 msgr2=0x565344523b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.844+0000 7f2f7136a740 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x565344523b60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c02ade0 tx=0x7f2f5c005fd0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- 172.21.15.78:0/749752158 shutdown_connections 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565344187410 0x565344523b60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344166730 0x565344523620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 --2- 172.21.15.78:0/749752158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56534415e550 0x565344530f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- 172.21.15.78:0/749752158 >> 172.21.15.78:0/749752158 conn(0x56534417b560 msgr2=0x565344530860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- 172.21.15.78:0/749752158 shutdown_connections 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- 172.21.15.78:0/749752158 wait complete. 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- start start 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 Processor -- start 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534415e550 0x565344536830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565344166730 0x565344536d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344537ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641a9920 con 0x56534415e550 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641aad90 con 0x565344166730 2024-04-25T14:00:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641a9ff0 con 0x565344187410 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.845+0000 7f2f7136a740 10 client.0 _ll_register_callbacks cb 0x7f2f580a2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534415e550 0x565344536830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534415e550 0x565344536830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60194/0 (socket says 172.21.15.78:60194) 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 -- 172.21.15.78:0/2863748803 learned_addr learned my addr 172.21.15.78:0/2863748803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f637fe640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565344166730 0x565344536d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f68a53640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344537ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 -- 172.21.15.78:0/2863748803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 msgr2=0x565344537ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344537ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 -- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565344166730 msgr2=0x565344536d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565344166730 0x565344536d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 -- 172.21.15.78:0/2863748803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565344538260 con 0x56534415e550 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f68a53640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344537ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f63fff640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534415e550 0x565344536830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005940 tx=0x7f2f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a537c0454983999b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f437fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f437fe640 1 -- 172.21.15.78:0/2863748803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54029710 con 0x56534415e550 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f437fe640 1 -- 172.21.15.78:0/2863748803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2f540298b0 con 0x56534415e550 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f5c036070 con 0x56534415e550 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f437fe640 1 -- 172.21.15.78:0/2863748803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2f54029ab0 con 0x56534415e550 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f7136a740 10 client.15882 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5653445a8340 con 0x56534415e550 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.846+0000 7f2f437fe640 1 -- 172.21.15.78:0/2863748803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c036070 con 0x56534415e550 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f437fe640 1 -- 172.21.15.78:0/2863748803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2f5403d410 con 0x56534415e550 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f7136a740 10 client.15882 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f7136a740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f7136a740 20 client.15882 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56534411b630 0x5653445aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f7136a740 4 client.15882 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565344495580 con 0x56534411b630 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f637fe640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56534411b630 0x5653445aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f637fe640 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56534411b630 0x5653445aad10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83dce7a8a8688302 server_cookie=5604e15adcbc1423 in_seq=0 out_seq=0 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.847+0000 7f2f437fe640 10 client.15882 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.848+0000 7f2f437fe640 1 -- 172.21.15.78:0/2863748803 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565344495580 con 0x56534411b630 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.848+0000 7f2f437fe640 10 client.15882 handle_command_reply: tid=0 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.848+0000 7f2f7136a740 1 client.15882 shutdown 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.848+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534415e550 msgr2=0x565344536830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.848+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534415e550 0x565344536830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005940 tx=0x7f2f540046c0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 shutdown_connections 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56534411b630 0x5653445aad10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565344187410 0x565344537ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565344166730 0x565344536d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 --2- 172.21.15.78:0/2863748803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534415e550 0x565344536830 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f437fe640 0 client.15882 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 >> 172.21.15.78:0/2863748803 conn(0x56534417b560 msgr2=0x565344526c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 shutdown_connections 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 1 -- 172.21.15.78:0/2863748803 wait complete. 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f7136a740 20 client.15882 trim_cache size 0 max 16384 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:00:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "237e6145-f6d5-4b12-90b3-0332b98b54f3", 2024-04-25T14:00:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:00:54.857 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:00:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2f4c0afaf0 msgr2=0x7f2f4c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2f4c0afaf0 0x7f2f4c0b2000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x565344529880 tx=0x7f2f5400b390 comp rx=0 tx=0).stop 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64104d30 msgr2=0x7f2f6419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64104d30 0x7f2f6419a730 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800d770 tx=0x7f2f5800dc30 comp rx=0 tx=0).stop 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 shutdown_connections 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f64109a30 0x7f2f641a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2f4c0afaf0 0x7f2f4c0b2000 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f64104d30 0x7f2f6419a730 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 --2- 172.21.15.78:0/68262683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f64103af0 0x7f2f6419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 >> 172.21.15.78:0/68262683 conn(0x7f2f640fece0 msgr2=0x7f2f64105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 shutdown_connections 2024-04-25T14:00:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:54.849+0000 7f2f6a4b6640 1 -- 172.21.15.78:0/68262683 wait complete. 2024-04-25T14:00:54.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:00:54.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:00:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 -- 172.21.15.78:0/887082888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a241029e0 msgr2=0x7f6a241036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/887082888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a241029e0 0x7f6a241036e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009900 tx=0x7f6a1402a410 comp rx=0 tx=0).stop 2024-04-25T14:00:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 -- 172.21.15.78:0/887082888 shutdown_connections 2024-04-25T14:00:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/887082888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a24108920 0x7f6a2410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/887082888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103c20 0x7f6a24104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/887082888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a241029e0 0x7f6a241036e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 -- 172.21.15.78:0/887082888 >> 172.21.15.78:0/887082888 conn(0x7f6a240fdbd0 msgr2=0x7f6a24100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 -- 172.21.15.78:0/887082888 shutdown_connections 2024-04-25T14:00:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 -- 172.21.15.78:0/887082888 wait complete. 2024-04-25T14:00:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.011+0000 7f6a29ce7640 1 Processor -- start 2024-04-25T14:00:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a29ce7640 1 -- start start 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a29ce7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a241029e0 0x7f6a2419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a29ce7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24103c20 0x7f6a2419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a29ce7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24108920 0x7f6a241a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a29ce7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e4e0 con 0x7f6a24103c20 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a29ce7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e360 con 0x7f6a24108920 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a29ce7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e660 con 0x7f6a241029e0 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24103c20 0x7f6a2419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24103c20 0x7f6a2419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60220/0 (socket says 172.21.15.78:60220) 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 -- 172.21.15.78:0/1289548213 learned_addr learned my addr 172.21.15.78:0/1289548213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a237fe640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a241029e0 0x7f6a2419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 -- 172.21.15.78:0/1289548213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a241029e0 msgr2=0x7f6a2419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a241029e0 0x7f6a2419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 -- 172.21.15.78:0/1289548213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24108920 msgr2=0x7f6a241a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24108920 0x7f6a241a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 -- 172.21.15.78:0/1289548213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a241a1fc0 con 0x7f6a24103c20 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a237fe640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a241029e0 0x7f6a2419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.012+0000 7f6a1affd640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24103c20 0x7f6a2419aa70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00c920 tx=0x7f6a0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.013+0000 7f6a217fa640 1 -- 172.21.15.78:0/1289548213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c007c70 con 0x7f6a24103c20 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.013+0000 7f6a217fa640 1 -- 172.21.15.78:0/1289548213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c007e10 con 0x7f6a24103c20 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.013+0000 7f6a217fa640 1 -- 172.21.15.78:0/1289548213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c018820 con 0x7f6a24103c20 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.013+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241a22b0 con 0x7f6a24103c20 2024-04-25T14:00:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.013+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a2850 con 0x7f6a24103c20 2024-04-25T14:00:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.015+0000 7f6a217fa640 1 -- 172.21.15.78:0/1289548213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6a0c00f450 con 0x7f6a24103c20 2024-04-25T14:00:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.015+0000 7f6a30b96740 1 Processor -- start 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.015+0000 7f6a30b96740 1 -- start start 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.015+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46c0390 0x55d2f4a65e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46974d0 0x55d2f4a663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f469f6b0 0x55d2f4a67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a30b96740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d2f49c6d80 con 0x55d2f469f6b0 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a30b96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d2f49808b0 con 0x55d2f46c0390 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a30b96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d2f49c2d40 con 0x55d2f46974d0 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f469f6b0 0x55d2f4a67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f469f6b0 0x55d2f4a67080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60228/0 (socket says 172.21.15.78:60228) 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 -- 172.21.15.78:0/3632021670 learned_addr learned my addr 172.21.15.78:0/3632021670 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a237fe640 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46974d0 0x55d2f4a663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a23fff640 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46c0390 0x55d2f4a65e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a217fa640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a100afb70 0x7f6a100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 -- 172.21.15.78:0/3632021670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46974d0 msgr2=0x55d2f4a663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46974d0 0x55d2f4a663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 -- 172.21.15.78:0/3632021670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46c0390 msgr2=0x55d2f4a65e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46c0390 0x55d2f4a65e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 -- 172.21.15.78:0/3632021670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f4951a40 con 0x55d2f469f6b0 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a237fe640 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46974d0 0x55d2f4a663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a23fff640 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46c0390 0x55d2f4a65e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a23fff640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a100afb70 0x7f6a100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a217fa640 1 -- 172.21.15.78:0/1289548213 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6a0c0dedb0 con 0x7f6a24103c20 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.016+0000 7f6a1affd640 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f469f6b0 0x55d2f4a67080 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c000c00 tx=0x7f6a0c09bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c5de95761982c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a19ffb640 1 -- 172.21.15.78:0/3632021670 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0a6bd0 con 0x55d2f469f6b0 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a23fff640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a100afb70 0x7f6a100b2080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009270 tx=0x7f6a1c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a19ffb640 1 -- 172.21.15.78:0/3632021670 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c0a6d90 con 0x55d2f469f6b0 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a19ffb640 1 -- 172.21.15.78:0/3632021670 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c0e25a0 con 0x55d2f469f6b0 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 -- 172.21.15.78:0/3632021670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f469f6b0 msgr2=0x55d2f4a67080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f469f6b0 0x55d2f4a67080 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c000c00 tx=0x7f6a0c09bbc0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 -- 172.21.15.78:0/3632021670 shutdown_connections 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f469f6b0 0x55d2f4a67080 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46974d0 0x55d2f4a663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 --2- 172.21.15.78:0/3632021670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46c0390 0x55d2f4a65e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 -- 172.21.15.78:0/3632021670 >> 172.21.15.78:0/3632021670 conn(0x55d2f46b44e0 msgr2=0x55d2f4a5d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.017+0000 7f6a30b96740 1 -- 172.21.15.78:0/3632021670 shutdown_connections 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 -- 172.21.15.78:0/3632021670 wait complete. 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 -- start start 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 Processor -- start 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a70230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a77190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d2f4a67b30 con 0x55d2f46c0390 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d2f494dd10 con 0x55d2f46974d0 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d2f4a67cb0 con 0x55d2f469f6b0 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a77190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a30b96740 10 client.0 _ll_register_callbacks cb 0x55d2f497a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.018+0000 7f6a23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a77190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60236/0 (socket says 172.21.15.78:60236) 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a23fff640 1 -- 172.21.15.78:0/193289619 learned_addr learned my addr 172.21.15.78:0/193289619 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a237fe640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a6fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a1affd640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a70230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a23fff640 1 -- 172.21.15.78:0/193289619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 msgr2=0x55d2f4a70230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a23fff640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a70230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a23fff640 1 -- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 msgr2=0x55d2f4a6fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a23fff640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a6fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a23fff640 1 -- 172.21.15.78:0/193289619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f47a27d0 con 0x55d2f46c0390 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a237fe640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a23fff640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a77190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01d740 tx=0x7f6a1c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd102457e97aa8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a02ffd640 1 -- 172.21.15.78:0/193289619 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c012a40 con 0x55d2f46c0390 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a02ffd640 1 -- 172.21.15.78:0/193289619 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6a1c012be0 con 0x55d2f46c0390 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0c0a6d90 con 0x55d2f46c0390 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a02ffd640 1 -- 172.21.15.78:0/193289619 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c012de0 con 0x55d2f46c0390 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a30b96740 10 client.15894 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d2f46d83f0 con 0x55d2f46c0390 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.019+0000 7f6a02ffd640 1 -- 172.21.15.78:0/193289619 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a0c0a6d90 con 0x55d2f46c0390 2024-04-25T14:00:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a02ffd640 1 -- 172.21.15.78:0/193289619 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6a1c02f050 con 0x55d2f46c0390 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a30b96740 10 client.15894 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a30b96740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a30b96740 20 client.15894 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a30b96740 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f467ef60 0x55d2f4a7d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a30b96740 4 client.15894 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2f49d4990 con 0x55d2f467ef60 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a237fe640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f467ef60 0x55d2f4a7d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a237fe640 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f467ef60 0x55d2f4a7d6c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152660e140abbd08 server_cookie=6c079bfa9f05fcf7 in_seq=0 out_seq=0 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.020+0000 7f6a02ffd640 10 client.15894 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a02ffd640 1 -- 172.21.15.78:0/193289619 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d2f49d4990 con 0x55d2f467ef60 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a02ffd640 10 client.15894 handle_command_reply: tid=0 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 client.15894 shutdown 2024-04-25T14:00:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 msgr2=0x55d2f4a77190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a77190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01d740 tx=0x7f6a1c009550 comp rx=0 tx=0).stop 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 shutdown_connections 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a77190 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f467ef60 0x55d2f4a7d6c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a70230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 --2- 172.21.15.78:0/193289619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a02ffd640 0 client.15894 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 >> 172.21.15.78:0/193289619 conn(0x55d2f46b44e0 msgr2=0x55d2f4a60180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 shutdown_connections 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 1 -- 172.21.15.78:0/193289619 wait complete. 2024-04-25T14:00:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.021+0000 7f6a30b96740 20 client.15894 trim_cache size 0 max 16384 2024-04-25T14:00:55.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 Processor -- start 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 -- start start 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46974d0 0x55d2f4a6b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f469f6b0 0x55d2f4a770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46c0390 0x55d2f4a5d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241ab8f0 con 0x55d2f46974d0 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241a91c0 con 0x55d2f469f6b0 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.039+0000 7f6a30b96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241a7010 con 0x55d2f46c0390 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46974d0 0x55d2f4a6b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46974d0 0x55d2f4a6b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60244/0 (socket says 172.21.15.78:60244) 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 -- 172.21.15.78:0/184350507 learned_addr learned my addr 172.21.15.78:0/184350507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:55.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a1affd640 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f469f6b0 0x55d2f4a770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 -- 172.21.15.78:0/184350507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46c0390 msgr2=0x55d2f4a5d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a23fff640 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46c0390 0x55d2f4a5d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46c0390 0x55d2f4a5d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 -- 172.21.15.78:0/184350507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f469f6b0 msgr2=0x55d2f4a770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f469f6b0 0x55d2f4a770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 -- 172.21.15.78:0/184350507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f46980f0 con 0x55d2f46974d0 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a1affd640 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f469f6b0 0x55d2f4a770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a237fe640 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46974d0 0x55d2f4a6b0c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6a14000c00 tx=0x7f6a14005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a01b3a0236ad34a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a197fa640 1 -- 172.21.15.78:0/184350507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14035040 con 0x55d2f46974d0 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a197fa640 1 -- 172.21.15.78:0/184350507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a14047070 con 0x55d2f46974d0 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.040+0000 7f6a197fa640 1 -- 172.21.15.78:0/184350507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14004920 con 0x55d2f46974d0 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 -- 172.21.15.78:0/184350507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46974d0 msgr2=0x55d2f4a6b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46974d0 0x55d2f4a6b0c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6a14000c00 tx=0x7f6a14005a20 comp rx=0 tx=0).stop 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 -- 172.21.15.78:0/184350507 shutdown_connections 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f46c0390 0x55d2f4a5d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f469f6b0 0x55d2f4a770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 --2- 172.21.15.78:0/184350507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46974d0 0x55d2f4a6b0c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 -- 172.21.15.78:0/184350507 >> 172.21.15.78:0/184350507 conn(0x55d2f46b44e0 msgr2=0x55d2f4a5cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 -- 172.21.15.78:0/184350507 shutdown_connections 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 -- 172.21.15.78:0/184350507 wait complete. 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.041+0000 7f6a30b96740 1 -- start start 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 1 Processor -- start 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a74f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241a7680 con 0x55d2f46c0390 2024-04-25T14:00:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241a8af0 con 0x55d2f46974d0 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241a7d50 con 0x55d2f469f6b0 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a30b96740 10 client.0 _ll_register_callbacks cb 0x7f6a1c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a75e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60254/0 (socket says 172.21.15.78:60254) 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 -- 172.21.15.78:0/1037121660 learned_addr learned my addr 172.21.15.78:0/1037121660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a1affd640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a74f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a237fe640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 -- 172.21.15.78:0/1037121660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 msgr2=0x55d2f4a74f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a74f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 -- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 msgr2=0x55d2f4a749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 -- 172.21.15.78:0/1037121660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f49874b0 con 0x55d2f46c0390 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a1affd640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a74f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:00:55.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a23fff640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a75e60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01d490 tx=0x7f6a1c012840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c896d5b2aaf77b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.042+0000 7f6a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a02ffd640 1 -- 172.21.15.78:0/1037121660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c012c40 con 0x55d2f46c0390 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a02ffd640 1 -- 172.21.15.78:0/1037121660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6a1c026b40 con 0x55d2f46c0390 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a14047070 con 0x55d2f46c0390 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a02ffd640 1 -- 172.21.15.78:0/1037121660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c026d40 con 0x55d2f46c0390 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 10 client.15900 fetch_fsmap learned FSMap version 22 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d2f4ae5a30 con 0x55d2f46c0390 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a02ffd640 1 -- 172.21.15.78:0/1037121660 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a14047070 con 0x55d2f46c0390 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a02ffd640 1 -- 172.21.15.78:0/1037121660 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6a1c02f070 con 0x55d2f46c0390 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 10 client.15900 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 20 client.15900 populate_metadata read hostname 'smithi078' 2024-04-25T14:00:55.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f46543a0 0x55d2f4ae83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 4 client.15900 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2f49ce800 con 0x55d2f46543a0 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.043+0000 7f6a237fe640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f46543a0 0x55d2f4ae83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a02ffd640 10 client.15900 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a237fe640 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f46543a0 0x55d2f4ae83a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d8230e2a6e5c01 server_cookie=e2bcef2eb905a9a7 in_seq=0 out_seq=0 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a02ffd640 1 -- 172.21.15.78:0/1037121660 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d2f49ce800 con 0x55d2f46543a0 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a02ffd640 10 client.15900 handle_command_reply: tid=0 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 client.15900 shutdown 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 msgr2=0x55d2f4a75e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a75e60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01d490 tx=0x7f6a1c012840 comp rx=0 tx=0).stop 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 shutdown_connections 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d2f46c0390 0x55d2f4a75e60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d2f46543a0 0x55d2f4ae83a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d2f469f6b0 0x55d2f4a74f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 --2- 172.21.15.78:0/1037121660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d2f46974d0 0x55d2f4a749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 >> 172.21.15.78:0/1037121660 conn(0x55d2f46b44e0 msgr2=0x55d2f4a5b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a02ffd640 0 client.15900 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 shutdown_connections 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 1 -- 172.21.15.78:0/1037121660 wait complete. 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.044+0000 7f6a30b96740 20 client.15900 trim_cache size 0 max 16384 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:00:55.051 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:00:55.052 INFO:teuthology.orchestra.run.smithi078.stdout: "237e6145-f6d5-4b12-90b3-0332b98b54f3": { 2024-04-25T14:00:55.052 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:00:55.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "237e6145-f6d5-4b12-90b3-0332b98b54f3", 2024-04-25T14:00:55.052 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:00:55.052 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:00:55.052 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:00:55.052 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a100afb70 msgr2=0x7f6a100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a100afb70 0x7f6a100b2080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009270 tx=0x7f6a1c004a40 comp rx=0 tx=0).stop 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24103c20 msgr2=0x7f6a2419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24103c20 0x7f6a2419aa70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00c920 tx=0x7f6a0c00cde0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 shutdown_connections 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24108920 0x7f6a241a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a100afb70 0x7f6a100b2080 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24103c20 0x7f6a2419aa70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 --2- 172.21.15.78:0/1289548213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a241029e0 0x7f6a2419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 >> 172.21.15.78:0/1289548213 conn(0x7f6a240fdbd0 msgr2=0x7f6a24069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:00:55.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 shutdown_connections 2024-04-25T14:00:55.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:00:55.045+0000 7f6a29ce7640 1 -- 172.21.15.78:0/1289548213 wait complete. 2024-04-25T14:00:55.056 INFO:tasks.cephfs.filesystem:scrub status for tag:237e6145-f6d5-4b12-90b3-0332b98b54f3 - {'path': '/', 'tag': '237e6145-f6d5-4b12-90b3-0332b98b54f3', 'options': 'recursive,force'} 2024-04-25T14:00:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:55 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:00:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:55 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:55 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:00:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:55 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:55.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:55 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:00:55.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:55 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:55.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:55 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:00:55.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:55 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:55.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:55 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:00:55.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:55 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:55.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:55 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:00:55.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:55 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:00:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:56 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:00:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:56 smithi129 ceph-mon[32569]: pgmap v391: 97 pgs: 97 active+clean; 834 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T14:00:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:56 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:00:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:56 smithi078 ceph-mon[29163]: pgmap v391: 97 pgs: 97 active+clean; 834 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T14:00:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:56 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:00:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:56 smithi112 ceph-mon[32617]: pgmap v391: 97 pgs: 97 active+clean; 834 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T14:00:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:00:58 smithi129 ceph-mon[32569]: pgmap v392: 97 pgs: 97 active+clean; 834 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:00:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:00:58 smithi078 ceph-mon[29163]: pgmap v392: 97 pgs: 97 active+clean; 834 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:00:58.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:00:58 smithi112 ceph-mon[32617]: pgmap v392: 97 pgs: 97 active+clean; 834 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:00 smithi129 ceph-mon[32569]: pgmap v393: 97 pgs: 97 active+clean; 834 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:01:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:00 smithi078 ceph-mon[29163]: pgmap v393: 97 pgs: 97 active+clean; 834 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:01:00.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:00 smithi112 ceph-mon[32617]: pgmap v393: 97 pgs: 97 active+clean; 834 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:01:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:02 smithi129 ceph-mon[32569]: pgmap v394: 97 pgs: 97 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T14:01:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:02 smithi078 ceph-mon[29163]: pgmap v394: 97 pgs: 97 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T14:01:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 -- 172.21.15.78:0/3368303645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 msgr2=0x7f65c8103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/3368303645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c8103720 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009900 tx=0x7f65b802a410 comp rx=0 tx=0).stop 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 -- 172.21.15.78:0/3368303645 shutdown_connections 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/3368303645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8108960 0x7f65c810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/3368303645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65c8103c60 0x7f65c8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/3368303645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c8103720 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 -- 172.21.15.78:0/3368303645 >> 172.21.15.78:0/3368303645 conn(0x7f65c80fdc10 msgr2=0x7f65c81000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 -- 172.21.15.78:0/3368303645 shutdown_connections 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.756+0000 7f65cd9e3640 1 -- 172.21.15.78:0/3368303645 wait complete. 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 Processor -- start 2024-04-25T14:01:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 -- start start 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c806ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8103c60 0x7f65c819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65c8108960 0x7f65c81a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810e570 con 0x7f65c8102a20 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810e3f0 con 0x7f65c8103c60 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65cd9e3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810e6f0 con 0x7f65c8108960 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c806ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c806ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35900/0 (socket says 172.21.15.78:35900) 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 -- 172.21.15.78:0/1806559170 learned_addr learned my addr 172.21.15.78:0/1806559170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c77fe640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65c8108960 0x7f65c81a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65be7fc640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8103c60 0x7f65c819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 -- 172.21.15.78:0/1806559170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65c8108960 msgr2=0x7f65c81a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65c8108960 0x7f65c81a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 -- 172.21.15.78:0/1806559170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8103c60 msgr2=0x7f65c819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8103c60 0x7f65c819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65c6ffd640 1 -- 172.21.15.78:0/1806559170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c81a1d90 con 0x7f65c8102a20 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.757+0000 7f65be7fc640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8103c60 0x7f65c819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.758+0000 7f65c6ffd640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c806ae90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f65b8031440 tx=0x7f65b8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.758+0000 7f65cc9e1640 1 -- 172.21.15.78:0/1806559170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b803f070 con 0x7f65c8102a20 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.758+0000 7f65cc9e1640 1 -- 172.21.15.78:0/1806559170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b803b460 con 0x7f65c8102a20 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.758+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c81a2020 con 0x7f65c8102a20 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.758+0000 7f65cc9e1640 1 -- 172.21.15.78:0/1806559170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8035030 con 0x7f65c8102a20 2024-04-25T14:01:02.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.758+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a2560 con 0x7f65c8102a20 2024-04-25T14:01:02.768 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:02 smithi112 ceph-mon[32617]: pgmap v394: 97 pgs: 97 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T14:01:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65cc9e1640 1 -- 172.21.15.78:0/1806559170 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f65b8042070 con 0x7f65c8102a20 2024-04-25T14:01:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 Processor -- start 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 -- start start 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697d71b0 0x55af69b77970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b77ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697b64d0 0x55af69b7edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af69add460 con 0x55af697d71b0 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af69a97590 con 0x55af697b64d0 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65d496d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55af69ad9420 con 0x55af697ae2f0 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697d71b0 0x55af69b77970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697d71b0 0x55af69b77970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35902/0 (socket says 172.21.15.78:35902) 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 -- 172.21.15.78:0/2085543586 learned_addr learned my addr 172.21.15.78:0/2085543586 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c6ffd640 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b77ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65be7fc640 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697b64d0 0x55af69b7edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 -- 172.21.15.78:0/2085543586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 msgr2=0x55af69b77ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b77ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 -- 172.21.15.78:0/2085543586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697b64d0 msgr2=0x55af69b7edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697b64d0 0x55af69b7edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65c77fe640 1 -- 172.21.15.78:0/2085543586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af69aa40d0 con 0x55af697d71b0 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.761+0000 7f65cc9e1640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65b40afb60 0x7f65b40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.762+0000 7f65c77fe640 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697d71b0 0x55af69b77970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009ee0 tx=0x7f65c0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f43826ab540e610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.762+0000 7f65c6ffd640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65b40afb60 0x7f65b40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.762+0000 7f65bdffb640 1 -- 172.21.15.78:0/2085543586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0016070 con 0x55af697d71b0 2024-04-25T14:01:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.762+0000 7f65bdffb640 1 -- 172.21.15.78:0/2085543586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c0004d20 con 0x55af697d71b0 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.762+0000 7f65bdffb640 1 -- 172.21.15.78:0/2085543586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00126b0 con 0x55af697d71b0 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.762+0000 7f65cc9e1640 1 -- 172.21.15.78:0/1806559170 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f65b80ff540 con 0x7f65c8102a20 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 -- 172.21.15.78:0/2085543586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697d71b0 msgr2=0x55af69b77970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697d71b0 0x55af69b77970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009ee0 tx=0x7f65c0010740 comp rx=0 tx=0).stop 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65c6ffd640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65b40afb60 0x7f65b40b2070 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55af69b78d50 tx=0x7f65b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 -- 172.21.15.78:0/2085543586 shutdown_connections 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697b64d0 0x55af69b7edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b77ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 --2- 172.21.15.78:0/2085543586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697d71b0 0x55af69b77970 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 -- 172.21.15.78:0/2085543586 >> 172.21.15.78:0/2085543586 conn(0x55af697cb300 msgr2=0x55af69b73300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 -- 172.21.15.78:0/2085543586 shutdown_connections 2024-04-25T14:01:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 -- 172.21.15.78:0/2085543586 wait complete. 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 -- start start 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.763+0000 7f65d496d740 1 Processor -- start 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 0x55af69b88b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af69b7f840 con 0x55af697b64d0 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af69a64b60 con 0x55af697ae2f0 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55af69b7f9c0 con 0x55af697d71b0 2024-04-25T14:01:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 10 client.0 _ll_register_callbacks cb 0x55af69a892b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b81bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35916/0 (socket says 172.21.15.78:35916) 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 -- 172.21.15.78:0/1541212634 learned_addr learned my addr 172.21.15.78:0/1541212634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65c77fe640 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 0x55af69b88b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65c77fe640 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 -- 172.21.15.78:0/1541212634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 msgr2=0x55af69b88b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 0x55af69b88b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 -- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 msgr2=0x55af69b816b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b816b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 -- 172.21.15.78:0/1541212634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af697ef210 con 0x55af697b64d0 2024-04-25T14:01:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65be7fc640 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b81bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f65b000e6e0 tx=0x7f65b000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1007d64ea8cedc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65a6ffd640 1 -- 172.21.15.78:0/1541212634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b001c060 con 0x55af697b64d0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65a6ffd640 1 -- 172.21.15.78:0/1541212634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f65b0009a80 con 0x55af697b64d0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.764+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c0004d20 con 0x55af697b64d0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65a6ffd640 1 -- 172.21.15.78:0/1541212634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f65b0009c80 con 0x55af697b64d0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 10 client.15912 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55af698bb620 con 0x55af697b64d0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65a6ffd640 1 -- 172.21.15.78:0/1541212634 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c0004d20 con 0x55af697b64d0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65a6ffd640 1 -- 172.21.15.78:0/1541212634 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f65b00223f0 con 0x55af697b64d0 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 10 client.15912 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 20 client.15912 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af69795d80 0x55af69b8f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 4 client.15912 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af69aeb070 con 0x55af69795d80 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.765+0000 7f65c77fe640 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af69795d80 0x55af69b8f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.766+0000 7f65a6ffd640 10 client.15912 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.766+0000 7f65c77fe640 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af69795d80 0x55af69b8f450 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a68bb44ba1bd448 server_cookie=4c5cc2c60a7cf429 in_seq=0 out_seq=0 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65a6ffd640 1 -- 172.21.15.78:0/1541212634 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af69aeb070 con 0x55af69795d80 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65a6ffd640 10 client.15912 handle_command_reply: tid=0 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 client.15912 shutdown 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 msgr2=0x55af69b81bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b81bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f65b000e6e0 tx=0x7f65b000eba0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 shutdown_connections 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af69795d80 0x55af69b8f450 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 0x55af69b88b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b81bf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 --2- 172.21.15.78:0/1541212634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65a6ffd640 0 client.15912 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 >> 172.21.15.78:0/1541212634 conn(0x55af697cb300 msgr2=0x55af69b76210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 shutdown_connections 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 1 -- 172.21.15.78:0/1541212634 wait complete. 2024-04-25T14:01:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.767+0000 7f65d496d740 20 client.15912 trim_cache size 0 max 16384 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 Processor -- start 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 -- start start 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 0x55af69b73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81adb50 con 0x55af697b64d0 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81ab420 con 0x55af697ae2f0 2024-04-25T14:01:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65d496d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81a9270 con 0x55af697d71b0 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b88a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35928/0 (socket says 172.21.15.78:35928) 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 -- 172.21.15.78:0/861143330 learned_addr learned my addr 172.21.15.78:0/861143330 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65c77fe640 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 -- 172.21.15.78:0/861143330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 msgr2=0x55af69b73190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 0x55af69b73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 -- 172.21.15.78:0/861143330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 msgr2=0x55af69b72c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b72c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65be7fc640 1 -- 172.21.15.78:0/861143330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af69b88fb0 con 0x55af697b64d0 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.782+0000 7f65c77fe640 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b72c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.783+0000 7f65be7fc640 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b88a70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f65b0001eb0 tx=0x7f65b000c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb7caf4edfc6afb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.783+0000 7f65bd7fa640 1 -- 172.21.15.78:0/861143330 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0009e20 con 0x55af697b64d0 2024-04-25T14:01:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.783+0000 7f65bd7fa640 1 -- 172.21.15.78:0/861143330 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b0022e50 con 0x55af697b64d0 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.783+0000 7f65bd7fa640 1 -- 172.21.15.78:0/861143330 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0009880 con 0x55af697b64d0 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- 172.21.15.78:0/861143330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 msgr2=0x55af69b88a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b88a70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f65b0001eb0 tx=0x7f65b000c630 comp rx=0 tx=0).stop 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- 172.21.15.78:0/861143330 shutdown_connections 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697d71b0 0x55af69b73190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b88a70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 --2- 172.21.15.78:0/861143330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697ae2f0 0x55af69b72c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- 172.21.15.78:0/861143330 >> 172.21.15.78:0/861143330 conn(0x55af697cb300 msgr2=0x55af69b7bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- 172.21.15.78:0/861143330 shutdown_connections 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- 172.21.15.78:0/861143330 wait complete. 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- start start 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 Processor -- start 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697d71b0 0x55af69b87730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81a98e0 con 0x55af697b64d0 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81aad50 con 0x55af697d71b0 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81a9fb0 con 0x55af697ae2f0 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65d496d740 10 client.0 _ll_register_callbacks cb 0x7f65b0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.784+0000 7f65be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b86800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35936/0 (socket says 172.21.15.78:35936) 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 -- 172.21.15.78:0/3837815048 learned_addr learned my addr 172.21.15.78:0/3837815048 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65c77fe640 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65c77fe640 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697d71b0 0x55af69b87730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 -- 172.21.15.78:0/3837815048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 msgr2=0x55af69b862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b862c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 -- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697d71b0 msgr2=0x55af69b87730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697d71b0 0x55af69b87730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 -- 172.21.15.78:0/3837815048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af69b87cf0 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65be7fc640 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b86800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f65b000e6e0 tx=0x7f65b000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e6784b1a2fdd04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65a6ffd640 1 -- 172.21.15.78:0/3837815048 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b001c040 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65a6ffd640 1 -- 172.21.15.78:0/3837815048 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f65b002a470 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b0022e50 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65a6ffd640 1 -- 172.21.15.78:0/3837815048 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f65b002a6d0 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65d496d740 10 client.15918 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55af69bf7800 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.785+0000 7f65a6ffd640 1 -- 172.21.15.78:0/3837815048 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b0022e50 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65a6ffd640 1 -- 172.21.15.78:0/3837815048 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f65b003b050 con 0x55af697b64d0 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65d496d740 10 client.15918 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65d496d740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65d496d740 20 client.15918 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65d496d740 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af6976b1c0 0x55af69bfa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65d496d740 4 client.15918 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af69ae4ee0 con 0x55af6976b1c0 2024-04-25T14:01:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65c77fe640 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af6976b1c0 0x55af69bfa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65a6ffd640 10 client.15918 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.786+0000 7f65c77fe640 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af6976b1c0 0x55af69bfa100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84abbb723a91d895 server_cookie=42aa15a84e43996 in_seq=0 out_seq=0 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.798+0000 7f65a6ffd640 1 -- 172.21.15.78:0/3837815048 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af69ae4ee0 con 0x55af6976b1c0 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.798+0000 7f65a6ffd640 10 client.15918 handle_command_reply: tid=0 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.798+0000 7f65d496d740 1 client.15918 shutdown 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 msgr2=0x55af69b86800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b86800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f65b000e6e0 tx=0x7f65b000ce70 comp rx=0 tx=0).stop 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 shutdown_connections 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af697d71b0 0x55af69b87730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af697b64d0 0x55af69b86800 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55af6976b1c0 0x55af69bfa100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 --2- 172.21.15.78:0/3837815048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55af697ae2f0 0x55af69b862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65a6ffd640 0 client.15918 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 >> 172.21.15.78:0/3837815048 conn(0x55af697cb300 msgr2=0x55af69b76100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 shutdown_connections 2024-04-25T14:01:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 1 -- 172.21.15.78:0/3837815048 wait complete. 2024-04-25T14:01:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65d496d740 20 client.15918 trim_cache size 0 max 16384 2024-04-25T14:01:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:01:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:01:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:01:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65b40afb60 msgr2=0x7f65b40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65b40afb60 0x7f65b40b2070 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55af69b78d50 tx=0x7f65b8009260 comp rx=0 tx=0).stop 2024-04-25T14:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 msgr2=0x7f65c806ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c806ae90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f65b8031440 tx=0x7f65b8003d50 comp rx=0 tx=0).stop 2024-04-25T14:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 shutdown_connections 2024-04-25T14:01:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65c8108960 0x7f65c81a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8103c60 0x7f65c819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f65b40afb60 0x7f65b40b2070 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 --2- 172.21.15.78:0/1806559170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8102a20 0x7f65c806ae90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.799+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 >> 172.21.15.78:0/1806559170 conn(0x7f65c80fdc10 msgr2=0x7f65c8104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.800+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 shutdown_connections 2024-04-25T14:01:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:02.800+0000 7f65cd9e3640 1 -- 172.21.15.78:0/1806559170 wait complete. 2024-04-25T14:01:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:04 smithi129 ceph-mon[32569]: pgmap v395: 97 pgs: 97 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-25T14:01:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:04 smithi078 ceph-mon[29163]: pgmap v395: 97 pgs: 97 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-25T14:01:04.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:04 smithi112 ceph-mon[32617]: pgmap v395: 97 pgs: 97 active+clean; 895 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-25T14:01:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:06 smithi129 ceph-mon[32569]: pgmap v396: 97 pgs: 97 active+clean; 965 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:01:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:06 smithi078 ceph-mon[29163]: pgmap v396: 97 pgs: 97 active+clean; 965 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:01:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:06 smithi112 ceph-mon[32617]: pgmap v396: 97 pgs: 97 active+clean; 965 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:01:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:08 smithi129 ceph-mon[32569]: pgmap v397: 97 pgs: 97 active+clean; 965 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:01:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:08 smithi078 ceph-mon[29163]: pgmap v397: 97 pgs: 97 active+clean; 965 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:01:08.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:08 smithi112 ceph-mon[32617]: pgmap v397: 97 pgs: 97 active+clean; 965 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:01:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:10 smithi129 ceph-mon[32569]: pgmap v398: 97 pgs: 97 active+clean; 965 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:10 smithi078 ceph-mon[29163]: pgmap v398: 97 pgs: 97 active+clean; 965 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:10.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:10 smithi112 ceph-mon[32617]: pgmap v398: 97 pgs: 97 active+clean; 965 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:12.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:12 smithi129 ceph-mon[32569]: pgmap v399: 97 pgs: 97 active+clean; 1022 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:01:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:12 smithi078 ceph-mon[29163]: pgmap v399: 97 pgs: 97 active+clean; 1022 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:01:12.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:12 smithi112 ceph-mon[32617]: pgmap v399: 97 pgs: 97 active+clean; 1022 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:01:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:14 smithi129 ceph-mon[32569]: pgmap v400: 97 pgs: 97 active+clean; 1022 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:01:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:14 smithi078 ceph-mon[29163]: pgmap v400: 97 pgs: 97 active+clean; 1022 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:01:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:14 smithi112 ceph-mon[32617]: pgmap v400: 97 pgs: 97 active+clean; 1022 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:01:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 -- 172.21.15.78:0/3723895739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c109a80 msgr2=0x7f9b9c10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/3723895739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c109a80 0x7f9b9c10bea0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009ca0 tx=0x7f9b9402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 -- 172.21.15.78:0/3723895739 shutdown_connections 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/3723895739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c109a80 0x7f9b9c10bea0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/3723895739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b9c104d80 0x7f9b9c105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/3723895739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9c103b40 0x7f9b9c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 -- 172.21.15.78:0/3723895739 >> 172.21.15.78:0/3723895739 conn(0x7f9b9c0fed10 msgr2=0x7f9b9c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 -- 172.21.15.78:0/3723895739 shutdown_connections 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.904+0000 7f9ba46a8640 1 -- 172.21.15.78:0/3723895739 wait complete. 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 Processor -- start 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 -- start start 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c103b40 0x7f9b9c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9c104d80 0x7f9b9c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b9c109a80 0x7f9b9c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c10f650 con 0x7f9b9c103b40 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c10f4d0 con 0x7f9b9c104d80 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba46a8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c10f7d0 con 0x7f9b9c109a80 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c103b40 0x7f9b9c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c103b40 0x7f9b9c19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57812/0 (socket says 172.21.15.78:57812) 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.905+0000 7f9ba2444640 1 -- 172.21.15.78:0/1428903577 learned_addr learned my addr 172.21.15.78:0/1428903577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9c104d80 0x7f9b9c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba2c45640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b9c109a80 0x7f9b9c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba2444640 1 -- 172.21.15.78:0/1428903577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b9c109a80 msgr2=0x7f9b9c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba2444640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b9c109a80 0x7f9b9c1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba2444640 1 -- 172.21.15.78:0/1428903577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9c104d80 msgr2=0x7f9b9c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba2444640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9c104d80 0x7f9b9c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba2444640 1 -- 172.21.15.78:0/1428903577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c1a1d00 con 0x7f9b9c103b40 2024-04-25T14:01:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba2444640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c103b40 0x7f9b9c19a1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d740 tx=0x7f9b8c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9b937fe640 1 -- 172.21.15.78:0/1428903577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c017070 con 0x7f9b9c103b40 2024-04-25T14:01:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9b937fe640 1 -- 172.21.15.78:0/1428903577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b8c00de50 con 0x7f9b9c103b40 2024-04-25T14:01:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9b937fe640 1 -- 172.21.15.78:0/1428903577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c003c20 con 0x7f9b9c103b40 2024-04-25T14:01:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c1a1ff0 con 0x7f9b9c103b40 2024-04-25T14:01:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.906+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c1a2400 con 0x7f9b9c103b40 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9b937fe640 1 -- 172.21.15.78:0/1428903577 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9b8c003dc0 con 0x7f9b9c103b40 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 Processor -- start 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 -- start start 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 0x564a238d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238d4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a2383a460 con 0x564a2350b2f0 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564a237f4590 con 0x564a235341b0 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9bab636740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564a23836420 con 0x564a235134d0 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.909+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238d4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238d4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57816/0 (socket says 172.21.15.78:57816) 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 -- 172.21.15.78:0/2383424285 learned_addr learned my addr 172.21.15.78:0/2383424285 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2c45640 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 0x564a238d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9b937fe640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9b880afb60 0x7f9b880b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 -- 172.21.15.78:0/2383424285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 msgr2=0x564a238dbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238dbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 -- 172.21.15.78:0/2383424285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 msgr2=0x564a238d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 0x564a238d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 -- 172.21.15.78:0/2383424285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a238010d0 con 0x564a2350b2f0 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2c45640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9b880afb60 0x7f9b880b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9b937fe640 1 -- 172.21.15.78:0/1428903577 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9b8c0de5c0 con 0x7f9b9c103b40 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2444640 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238d4ef0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c070760 tx=0x7f9b8c06cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf466a40f2f5c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9ba2c45640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9b880afb60 0x7f9b880b2070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9b94004af0 tx=0x7f9b940095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.910+0000 7f9b90bfe640 1 -- 172.21.15.78:0/2383424285 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0a6860 con 0x564a2350b2f0 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9b90bfe640 1 -- 172.21.15.78:0/2383424285 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b8c0a65a0 con 0x564a2350b2f0 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9b90bfe640 1 -- 172.21.15.78:0/2383424285 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c070800 con 0x564a2350b2f0 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9bab636740 1 -- 172.21.15.78:0/2383424285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 msgr2=0x564a238d4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9bab636740 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238d4ef0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c070760 tx=0x7f9b8c06cfb0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9bab636740 1 -- 172.21.15.78:0/2383424285 shutdown_connections 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9bab636740 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9bab636740 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238d4ef0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9bab636740 1 --2- 172.21.15.78:0/2383424285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 0x564a238d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.911+0000 7f9bab636740 1 -- 172.21.15.78:0/2383424285 >> 172.21.15.78:0/2383424285 conn(0x564a23528300 msgr2=0x564a238d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 -- 172.21.15.78:0/2383424285 shutdown_connections 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 -- 172.21.15.78:0/2383424285 wait complete. 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 -- start start 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 Processor -- start 2024-04-25T14:01:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a238dc7e0 con 0x564a2350b2f0 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564a237c1b60 con 0x564a235134d0 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564a238dc960 con 0x564a235341b0 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9bab636740 10 client.0 _ll_register_callbacks cb 0x564a237e62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57824/0 (socket says 172.21.15.78:57824) 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9ba2444640 1 -- 172.21.15.78:0/2708534735 learned_addr learned my addr 172.21.15.78:0/2708534735 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.912+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba2c45640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba2444640 1 -- 172.21.15.78:0/2708534735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 msgr2=0x564a238e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba2444640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba2444640 1 -- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 msgr2=0x564a238deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba2444640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238deac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba2444640 1 -- 172.21.15.78:0/2708534735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2354c210 con 0x564a2350b2f0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9ba2444640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238de580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0ac190 tx=0x7f9b8c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ae5f824b4a7ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9b85ffb640 1 -- 172.21.15.78:0/2708534735 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0e1e60 con 0x564a2350b2f0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9b85ffb640 1 -- 172.21.15.78:0/2708534735 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9b8c0ade90 con 0x564a2350b2f0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c0a65a0 con 0x564a2350b2f0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9b85ffb640 1 -- 172.21.15.78:0/2708534735 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9b8c020070 con 0x564a2350b2f0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9bab636740 10 client.15930 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564a23618620 con 0x564a2350b2f0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9b85ffb640 1 -- 172.21.15.78:0/2708534735 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b8c0a65a0 con 0x564a2350b2f0 2024-04-25T14:01:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.913+0000 7f9b85ffb640 1 -- 172.21.15.78:0/2708534735 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9b8c0a2d20 con 0x564a2350b2f0 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9bab636740 10 client.15930 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9bab636740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9bab636740 20 client.15930 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9bab636740 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234f2d80 0x564a238ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9bab636740 4 client.15930 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a23848070 con 0x564a234f2d80 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234f2d80 0x564a238ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9b85ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.914+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234f2d80 0x564a238ebe70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe3998e6f6120b6 server_cookie=e7beddb328a26e97 in_seq=0 out_seq=0 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9b85ffb640 1 -- 172.21.15.78:0/2708534735 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a23848070 con 0x564a234f2d80 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9b85ffb640 10 client.15930 handle_command_reply: tid=0 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 client.15930 shutdown 2024-04-25T14:01:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 msgr2=0x564a238de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238de580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0ac190 tx=0x7f9b8c009cb0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 shutdown_connections 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234f2d80 0x564a238ebe70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 --2- 172.21.15.78:0/2708534735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238de580 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9b85ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 >> 172.21.15.78:0/2708534735 conn(0x564a23528300 msgr2=0x564a238d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 shutdown_connections 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 1 -- 172.21.15.78:0/2708534735 wait complete. 2024-04-25T14:01:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.915+0000 7f9bab636740 20 client.15930 trim_cache size 0 max 16384 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.930+0000 7f9bab636740 1 Processor -- start 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.930+0000 7f9bab636740 1 -- start start 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9bab636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9bab636740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9bab636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 0x564a238d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9bab636740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c1adc80 con 0x564a2350b2f0 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9bab636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c1ab550 con 0x564a235341b0 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9bab636740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c1a93a0 con 0x564a235134d0 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57826/0 (socket says 172.21.15.78:57826) 2024-04-25T14:01:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 -- 172.21.15.78:0/3875038462 learned_addr learned my addr 172.21.15.78:0/3875038462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 -- 172.21.15.78:0/3875038462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 msgr2=0x564a238e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238e59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 -- 172.21.15.78:0/3875038462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 msgr2=0x564a238d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 0x564a238d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 -- 172.21.15.78:0/3875038462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a238dc1a0 con 0x564a2350b2f0 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9ba2444640 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238cfbd0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000c00 tx=0x7f9b8c0a2350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499636fea3917325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9b87fff640 1 -- 172.21.15.78:0/3875038462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0e1e60 con 0x564a2350b2f0 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9b87fff640 1 -- 172.21.15.78:0/3875038462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b8c0a7190 con 0x564a2350b2f0 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.931+0000 7f9b87fff640 1 -- 172.21.15.78:0/3875038462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0abfd0 con 0x564a2350b2f0 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 -- 172.21.15.78:0/3875038462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 msgr2=0x564a238cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238cfbd0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000c00 tx=0x7f9b8c0a2350 comp rx=0 tx=0).stop 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 -- 172.21.15.78:0/3875038462 shutdown_connections 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235341b0 0x564a238d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235134d0 0x564a238e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 --2- 172.21.15.78:0/3875038462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238cfbd0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 -- 172.21.15.78:0/3875038462 >> 172.21.15.78:0/3875038462 conn(0x564a23528300 msgr2=0x564a238d8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 -- 172.21.15.78:0/3875038462 shutdown_connections 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 -- 172.21.15.78:0/3875038462 wait complete. 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 -- start start 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.932+0000 7f9bab636740 1 Processor -- start 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c1a9a10 con 0x564a2350b2f0 2024-04-25T14:01:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c1aae80 con 0x564a235134d0 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c1aa0e0 con 0x564a235341b0 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 10 client.0 _ll_register_callbacks cb 0x7f9b94009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238e2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57838/0 (socket says 172.21.15.78:57838) 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 -- 172.21.15.78:0/662324257 learned_addr learned my addr 172.21.15.78:0/662324257 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2c45640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 -- 172.21.15.78:0/662324257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 msgr2=0x564a238e3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 -- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 msgr2=0x564a238e30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238e30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 -- 172.21.15.78:0/662324257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a238e4550 con 0x564a2350b2f0 2024-04-25T14:01:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9ba2444640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238e2b90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0a6a20 tx=0x7f9b8c0a2fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5051323ac42e0f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9b85ffb640 1 -- 172.21.15.78:0/662324257 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0e1e60 con 0x564a2350b2f0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9b85ffb640 1 -- 172.21.15.78:0/662324257 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9b8c0bb350 con 0x564a2350b2f0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.933+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b8c0a7190 con 0x564a2350b2f0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9b85ffb640 1 -- 172.21.15.78:0/662324257 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9b8c0ae1c0 con 0x564a2350b2f0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 10 client.15936 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564a238e5750 con 0x564a2350b2f0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9b85ffb640 1 -- 172.21.15.78:0/662324257 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b8c0a7190 con 0x564a2350b2f0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9b85ffb640 1 -- 172.21.15.78:0/662324257 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9b8c0a74b0 con 0x564a2350b2f0 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 10 client.15936 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 20 client.15936 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234c81c0 0x564a23956110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 4 client.15936 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a23841ee0 con 0x564a234c81c0 2024-04-25T14:01:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.934+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234c81c0 0x564a23956110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.935+0000 7f9b85ffb640 10 client.15936 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.935+0000 7f9ba1c43640 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234c81c0 0x564a23956110 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd328df18f593e5 server_cookie=cbc1197c35df3355 in_seq=0 out_seq=0 2024-04-25T14:01:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9b85ffb640 1 -- 172.21.15.78:0/662324257 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a23841ee0 con 0x564a234c81c0 2024-04-25T14:01:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9b85ffb640 10 client.15936 handle_command_reply: tid=0 2024-04-25T14:01:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 client.15936 shutdown 2024-04-25T14:01:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 msgr2=0x564a238e2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238e2b90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0a6a20 tx=0x7f9b8c0a2fc0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 shutdown_connections 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564a234c81c0 0x564a23956110 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a235341b0 0x564a238e3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564a235134d0 0x564a238e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 --2- 172.21.15.78:0/662324257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a2350b2f0 0x564a238e2b90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9b85ffb640 0 client.15936 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 >> 172.21.15.78:0/662324257 conn(0x564a23528300 msgr2=0x564a238de520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 shutdown_connections 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 1 -- 172.21.15.78:0/662324257 wait complete. 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.947+0000 7f9bab636740 20 client.15936 trim_cache size 0 max 16384 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:01:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:01:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:01:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:01:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9b880afb60 msgr2=0x7f9b880b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9b880afb60 0x7f9b880b2070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9b94004af0 tx=0x7f9b940095c0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c103b40 msgr2=0x7f9b9c19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c103b40 0x7f9b9c19a1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d740 tx=0x7f9b8c00dc00 comp rx=0 tx=0).stop 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 shutdown_connections 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b9c109a80 0x7f9b9c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b9c104d80 0x7f9b9c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9b880afb60 0x7f9b880b2070 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 --2- 172.21.15.78:0/1428903577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b9c103b40 0x7f9b9c19a1c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 >> 172.21.15.78:0/1428903577 conn(0x7f9b9c0fed10 msgr2=0x7f9b9c105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 shutdown_connections 2024-04-25T14:01:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:15.948+0000 7f9ba46a8640 1 -- 172.21.15.78:0/1428903577 wait complete. 2024-04-25T14:01:16.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:16 smithi129 ceph-mon[32569]: pgmap v401: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:16.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:16 smithi078 ceph-mon[29163]: pgmap v401: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:16.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:16 smithi112 ceph-mon[32617]: pgmap v401: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:18 smithi129 ceph-mon[32569]: pgmap v402: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:01:18.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:18 smithi078 ceph-mon[29163]: pgmap v402: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:01:18.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:18 smithi112 ceph-mon[32617]: pgmap v402: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:01:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:20 smithi129 ceph-mon[32569]: pgmap v403: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:01:20.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:20 smithi078 ceph-mon[29163]: pgmap v403: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:01:20.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:20 smithi112 ceph-mon[32617]: pgmap v403: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:01:22.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:22 smithi112 ceph-mon[32617]: pgmap v404: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:22 smithi129 ceph-mon[32569]: pgmap v404: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:22 smithi078 ceph-mon[29163]: pgmap v404: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:24 smithi129 ceph-mon[32569]: pgmap v405: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:24 smithi078 ceph-mon[29163]: pgmap v405: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:24.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:24 smithi112 ceph-mon[32617]: pgmap v405: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:25.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:01:25.057 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 -- 172.21.15.78:0/1150723507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c104d30 msgr2=0x7fac2c107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 --2- 172.21.15.78:0/1150723507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c104d30 0x7fac2c107150 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009900 tx=0x7fac1c02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 -- 172.21.15.78:0/1150723507 shutdown_connections 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 --2- 172.21.15.78:0/1150723507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac2c107690 0x7fac2c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 --2- 172.21.15.78:0/1150723507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c104d30 0x7fac2c107150 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 --2- 172.21.15.78:0/1150723507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c1023d0 0x7fac2c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 -- 172.21.15.78:0/1150723507 >> 172.21.15.78:0/1150723507 conn(0x7fac2c0fdbc0 msgr2=0x7fac2c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 -- 172.21.15.78:0/1150723507 shutdown_connections 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 -- 172.21.15.78:0/1150723507 wait complete. 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 Processor -- start 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.186+0000 7fac33c13640 1 -- start start 2024-04-25T14:01:25.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c1023d0 0x7fac2c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac2c104d30 0x7fac2c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c107690 0x7fac2c1a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c115c10 con 0x7fac2c1023d0 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c115a90 con 0x7fac2c104d30 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c115d90 con 0x7fac2c107690 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c1023d0 0x7fac2c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c1023d0 0x7fac2c1a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40198/0 (socket says 172.21.15.78:40198) 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 -- 172.21.15.78:0/2045424559 learned_addr learned my addr 172.21.15.78:0/2045424559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac311ae640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac2c104d30 0x7fac2c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac321b0640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c107690 0x7fac2c1a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 -- 172.21.15.78:0/2045424559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c107690 msgr2=0x7fac2c1a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c107690 0x7fac2c1a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 -- 172.21.15.78:0/2045424559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac2c104d30 msgr2=0x7fac2c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac2c104d30 0x7fac2c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 -- 172.21.15.78:0/2045424559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c1a8390 con 0x7fac2c1023d0 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac319af640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c1023d0 0x7fac2c1a0850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fac2800e9c0 tx=0x7fac2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac22ffd640 1 -- 172.21.15.78:0/2045424559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2800cd80 con 0x7fac2c1023d0 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac22ffd640 1 -- 172.21.15.78:0/2045424559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac280054d0 con 0x7fac2c1023d0 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c1a8680 con 0x7fac2c1023d0 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac22ffd640 1 -- 172.21.15.78:0/2045424559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28010430 con 0x7fac2c1023d0 2024-04-25T14:01:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.187+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1a8bd0 con 0x7fac2c1023d0 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.190+0000 7fac3ab9e740 1 Processor -- start 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac3ab9e740 1 -- start start 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac22ffd640 1 -- 172.21.15.78:0/2045424559 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fac28004fd0 con 0x7fac2c1023d0 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d04e390 0x55735d3efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d0254d0 0x55735d3effe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55735d354d80 con 0x55735d04e390 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55735d30e8b0 con 0x55735d0254d0 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55735d350d40 con 0x55735d02d6b0 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d04e390 0x55735d3efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d04e390 0x55735d3efa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40204/0 (socket says 172.21.15.78:40204) 2024-04-25T14:01:25.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 -- 172.21.15.78:0/3951284100 learned_addr learned my addr 172.21.15.78:0/3951284100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac311ae640 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac319af640 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d0254d0 0x55735d3effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 -- 172.21.15.78:0/3951284100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 msgr2=0x55735d3f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 -- 172.21.15.78:0/3951284100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d0254d0 msgr2=0x55735d3effe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d0254d0 0x55735d3effe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 -- 172.21.15.78:0/3951284100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735d2dfa40 con 0x55735d04e390 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac311ae640 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac319af640 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d0254d0 0x55735d3effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac22ffd640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac180afaf0 0x7fac180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.192+0000 7fac319af640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac180afaf0 0x7fac180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.191+0000 7fac321b0640 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d04e390 0x55735d3efa60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ea0 tx=0x7fac24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f632e9e1b86a1001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.192+0000 7fac22ffd640 1 -- 172.21.15.78:0/2045424559 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fac280df170 con 0x7fac2c1023d0 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.192+0000 7fac17fff640 1 -- 172.21.15.78:0/3951284100 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24011640 con 0x55735d04e390 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.192+0000 7fac17fff640 1 -- 172.21.15.78:0/3951284100 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac24011c90 con 0x55735d04e390 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.192+0000 7fac17fff640 1 -- 172.21.15.78:0/3951284100 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2401bd10 con 0x55735d04e390 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 -- 172.21.15.78:0/3951284100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d04e390 msgr2=0x55735d3efa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d04e390 0x55735d3efa60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ea0 tx=0x7fac24009f90 comp rx=0 tx=0).stop 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac319af640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac180afaf0 0x7fac180b2000 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55735d3ef780 tx=0x7fac280049b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 -- 172.21.15.78:0/3951284100 shutdown_connections 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d0254d0 0x55735d3effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/3951284100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d04e390 0x55735d3efa60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 -- 172.21.15.78:0/3951284100 >> 172.21.15.78:0/3951284100 conn(0x55735d0424e0 msgr2=0x55735d3eb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 -- 172.21.15.78:0/3951284100 shutdown_connections 2024-04-25T14:01:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 -- 172.21.15.78:0/3951284100 wait complete. 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 -- start start 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.193+0000 7fac3ab9e740 1 Processor -- start 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3f9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d400db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55735d3f1780 con 0x55735d0254d0 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55735d2dbd10 con 0x55735d04e390 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55735d3f1900 con 0x55735d02d6b0 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac3ab9e740 10 client.0 _ll_register_callbacks cb 0x55735d308210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3f9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40212/0 (socket says 172.21.15.78:40212) 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 -- 172.21.15.78:0/4194897234 learned_addr learned my addr 172.21.15.78:0/4194897234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac311ae640 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 -- 172.21.15.78:0/4194897234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 msgr2=0x55735d3f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 -- 172.21.15.78:0/4194897234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 msgr2=0x55735d400db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d400db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 -- 172.21.15.78:0/4194897234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735d1307d0 con 0x55735d0254d0 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac321b0640 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3f9910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac240099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64b3231ac6fe73a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.194+0000 7fac157fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac157fa640 1 -- 172.21.15.78:0/4194897234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2401ba50 con 0x55735d0254d0 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac157fa640 1 -- 172.21.15.78:0/4194897234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fac2401bbf0 con 0x55735d0254d0 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac24011c90 con 0x55735d0254d0 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac157fa640 1 -- 172.21.15.78:0/4194897234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fac2401bdf0 con 0x55735d0254d0 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 10 client.15948 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55735d0663f0 con 0x55735d0254d0 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac157fa640 1 -- 172.21.15.78:0/4194897234 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac24011c90 con 0x55735d0254d0 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac157fa640 1 -- 172.21.15.78:0/4194897234 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fac24004040 con 0x55735d0254d0 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 10 client.15948 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 20 client.15948 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735d00cf60 0x55735d406ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 4 client.15948 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.195+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55735d362990 con 0x55735d00cf60 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.196+0000 7fac311ae640 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735d00cf60 0x55735d406ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.196+0000 7fac311ae640 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735d00cf60 0x55735d406ed0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e589e4a369df7f59 server_cookie=68fd7eaf111bdf86 in_seq=0 out_seq=0 2024-04-25T14:01:25.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.196+0000 7fac157fa640 10 client.15948 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:25.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac157fa640 1 -- 172.21.15.78:0/4194897234 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55735d362990 con 0x55735d00cf60 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac157fa640 10 client.15948 handle_command_reply: tid=0 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 client.15948 shutdown 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 msgr2=0x55735d3f9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3f9910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fac24006010 tx=0x7fac240099f0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 shutdown_connections 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d400db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735d00cf60 0x55735d406ed0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac157fa640 0 client.15948 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/4194897234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3f9910 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 >> 172.21.15.78:0/4194897234 conn(0x55735d0424e0 msgr2=0x55735d3ea3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 shutdown_connections 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 1 -- 172.21.15.78:0/4194897234 wait complete. 2024-04-25T14:01:25.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.197+0000 7fac3ab9e740 20 client.15948 trim_cache size 0 max 16384 2024-04-25T14:01:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.215+0000 7fac3ab9e740 1 Processor -- start 2024-04-25T14:01:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.215+0000 7fac3ab9e740 1 -- start start 2024-04-25T14:01:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d0254d0 0x55735d3f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d02d6b0 0x55735d3f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d3ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1b4240 con 0x55735d02d6b0 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1b1b10 con 0x55735d04e390 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1af960 con 0x55735d0254d0 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d02d6b0 0x55735d3f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d02d6b0 0x55735d3f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40218/0 (socket says 172.21.15.78:40218) 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 -- 172.21.15.78:0/2253145020 learned_addr learned my addr 172.21.15.78:0/2253145020 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 -- 172.21.15.78:0/2253145020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d0254d0 msgr2=0x55735d3f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac321b0640 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d0254d0 0x55735d3f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d0254d0 0x55735d3f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 -- 172.21.15.78:0/2253145020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 msgr2=0x55735d3ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d3ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 -- 172.21.15.78:0/2253145020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735d0260f0 con 0x55735d02d6b0 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac321b0640 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d0254d0 0x55735d3f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac311ae640 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d02d6b0 0x55735d3f4870 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fac1c031440 tx=0x7fac1c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab02d917e42a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac177fe640 1 -- 172.21.15.78:0/2253145020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c048070 con 0x55735d02d6b0 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac177fe640 1 -- 172.21.15.78:0/2253145020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0054f0 con 0x55735d02d6b0 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.216+0000 7fac177fe640 1 -- 172.21.15.78:0/2253145020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c035030 con 0x55735d02d6b0 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2253145020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d02d6b0 msgr2=0x55735d3f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d02d6b0 0x55735d3f4870 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fac1c031440 tx=0x7fac1c005d50 comp rx=0 tx=0).stop 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2253145020 shutdown_connections 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d3ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d02d6b0 0x55735d3f4870 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2253145020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d0254d0 0x55735d3f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2253145020 >> 172.21.15.78:0/2253145020 conn(0x55735d0424e0 msgr2=0x55735d3ee0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2253145020 shutdown_connections 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2253145020 wait complete. 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.217+0000 7fac3ab9e740 1 -- start start 2024-04-25T14:01:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 1 Processor -- start 2024-04-25T14:01:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d3ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1affd0 con 0x55735d0254d0 2024-04-25T14:01:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1b1440 con 0x55735d04e390 2024-04-25T14:01:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1b06a0 con 0x55735d02d6b0 2024-04-25T14:01:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac3ab9e740 10 client.0 _ll_register_callbacks cb 0x7fac24015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3fe6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40222/0 (socket says 172.21.15.78:40222) 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 -- 172.21.15.78:0/2868276762 learned_addr learned my addr 172.21.15.78:0/2868276762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac311ae640 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d3ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 -- 172.21.15.78:0/2868276762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 msgr2=0x55735d3fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3fec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 -- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 msgr2=0x55735d3ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d3ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.218+0000 7fac321b0640 1 -- 172.21.15.78:0/2868276762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735d3154b0 con 0x55735d0254d0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac321b0640 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3fe6c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fac24002230 tx=0x7fac24011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88c813bdec26d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac157fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac157fa640 1 -- 172.21.15.78:0/2868276762 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24036070 con 0x55735d0254d0 2024-04-25T14:01:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac157fa640 1 -- 172.21.15.78:0/2868276762 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fac24004600 con 0x55735d0254d0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1c0054f0 con 0x55735d0254d0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac157fa640 1 -- 172.21.15.78:0/2868276762 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fac24003b80 con 0x55735d0254d0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 10 client.15954 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55735d401120 con 0x55735d0254d0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac157fa640 1 -- 172.21.15.78:0/2868276762 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0054f0 con 0x55735d0254d0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac157fa640 1 -- 172.21.15.78:0/2868276762 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fac2403b030 con 0x55735d0254d0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 10 client.15954 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 20 client.15954 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735cfe23a0 0x55735d471120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 4 client.15954 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.219+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55735d35c800 con 0x55735cfe23a0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac311ae640 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735cfe23a0 0x55735d471120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac157fa640 10 client.15954 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac311ae640 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735cfe23a0 0x55735d471120 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13766bcab9645f4c server_cookie=4036edaaab803b40 in_seq=0 out_seq=0 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac157fa640 1 -- 172.21.15.78:0/2868276762 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55735d35c800 con 0x55735cfe23a0 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac157fa640 10 client.15954 handle_command_reply: tid=0 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 client.15954 shutdown 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 msgr2=0x55735d3fe6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3fe6c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fac24002230 tx=0x7fac24011440 comp rx=0 tx=0).stop 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 shutdown_connections 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55735d04e390 0x55735d3ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55735cfe23a0 0x55735d471120 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55735d02d6b0 0x55735d3fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 --2- 172.21.15.78:0/2868276762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55735d0254d0 0x55735d3fe6c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac157fa640 0 client.15954 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 >> 172.21.15.78:0/2868276762 conn(0x55735d0424e0 msgr2=0x55735d3f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 shutdown_connections 2024-04-25T14:01:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 1 -- 172.21.15.78:0/2868276762 wait complete. 2024-04-25T14:01:25.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.220+0000 7fac3ab9e740 20 client.15954 trim_cache size 0 max 16384 2024-04-25T14:01:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:01:25.227 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:01:25.227 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:01:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:01:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac180afaf0 msgr2=0x7fac180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac180afaf0 0x7fac180b2000 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55735d3ef780 tx=0x7fac280049b0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c1023d0 msgr2=0x7fac2c1a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c1023d0 0x7fac2c1a0850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fac2800e9c0 tx=0x7fac2800ee80 comp rx=0 tx=0).stop 2024-04-25T14:01:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 shutdown_connections 2024-04-25T14:01:25.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c107690 0x7fac2c1a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac2c104d30 0x7fac2c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fac180afaf0 0x7fac180b2000 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 --2- 172.21.15.78:0/2045424559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2c1023d0 0x7fac2c1a0850 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 >> 172.21.15.78:0/2045424559 conn(0x7fac2c0fdbc0 msgr2=0x7fac2c103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 shutdown_connections 2024-04-25T14:01:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.221+0000 7fac33c13640 1 -- 172.21.15.78:0/2045424559 wait complete. 2024-04-25T14:01:25.232 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:01:25.232 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:01:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 -- 172.21.15.78:0/4178342453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec102180 msgr2=0x7fbeec102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 --2- 172.21.15.78:0/4178342453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec102180 0x7fbeec102e80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009900 tx=0x7fbee402a410 comp rx=0 tx=0).stop 2024-04-25T14:01:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 -- 172.21.15.78:0/4178342453 shutdown_connections 2024-04-25T14:01:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 --2- 172.21.15.78:0/4178342453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec1080c0 0x7fbeec10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 --2- 172.21.15.78:0/4178342453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeec1033c0 0x7fbeec1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 --2- 172.21.15.78:0/4178342453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec102180 0x7fbeec102e80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 -- 172.21.15.78:0/4178342453 >> 172.21.15.78:0/4178342453 conn(0x7fbeec0fd370 msgr2=0x7fbeec0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 -- 172.21.15.78:0/4178342453 shutdown_connections 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.341+0000 7fbef3415640 1 -- 172.21.15.78:0/4178342453 wait complete. 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 Processor -- start 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 -- start start 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeec102180 0x7fbeec19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec1033c0 0x7fbeec19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec1080c0 0x7fbeec1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10d410 con 0x7fbeec1080c0 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10d290 con 0x7fbeec102180 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef3415640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec10d590 con 0x7fbeec1033c0 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec1080c0 0x7fbeec1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec1080c0 0x7fbeec1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40244/0 (socket says 172.21.15.78:40244) 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 -- 172.21.15.78:0/897281364 learned_addr learned my addr 172.21.15.78:0/897281364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef09b0640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec1033c0 0x7fbeec19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef11b1640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeec102180 0x7fbeec19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 -- 172.21.15.78:0/897281364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec1033c0 msgr2=0x7fbeec19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec1033c0 0x7fbeec19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 -- 172.21.15.78:0/897281364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeec102180 msgr2=0x7fbeec19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeec102180 0x7fbeec19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 -- 172.21.15.78:0/897281364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec1a1c60 con 0x7fbeec1080c0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef11b1640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeec102180 0x7fbeec19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.342+0000 7fbef19b2640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec1080c0 0x7fbeec1a16a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00db10 tx=0x7fbedc00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.343+0000 7fbeea7fc640 1 -- 172.21.15.78:0/897281364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00ae60 con 0x7fbeec1080c0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.343+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1a1f50 con 0x7fbeec1080c0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.343+0000 7fbeea7fc640 1 -- 172.21.15.78:0/897281364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbedc0054d0 con 0x7fbeec1080c0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.343+0000 7fbeea7fc640 1 -- 172.21.15.78:0/897281364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00e6b0 con 0x7fbeec1080c0 2024-04-25T14:01:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.343+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1a23b0 con 0x7fbeec1080c0 2024-04-25T14:01:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.344+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec06a1c0 con 0x7fbeec1080c0 2024-04-25T14:01:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.345+0000 7fbeea7fc640 1 -- 172.21.15.78:0/897281364 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbedc004f70 con 0x7fbeec1080c0 2024-04-25T14:01:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.346+0000 7fbeea7fc640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbed40afaf0 0x7fbed40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.346+0000 7fbef11b1640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbed40afaf0 0x7fbed40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.346+0000 7fbeea7fc640 1 -- 172.21.15.78:0/897281364 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbedc0df040 con 0x7fbeec1080c0 2024-04-25T14:01:25.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.346+0000 7fbef11b1640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbed40afaf0 0x7fbed40b2000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbee4005f70 tx=0x7fbee4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.349+0000 7fbeea7fc640 1 -- 172.21.15.78:0/897281364 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbeec06a1c0 con 0x7fbeec1080c0 2024-04-25T14:01:25.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.565+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeec02dc10 con 0x7fbeec1080c0 2024-04-25T14:01:25.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.566+0000 7fbeea7fc640 1 -- 172.21.15.78:0/897281364 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fbedc0a7c10 con 0x7fbeec1080c0 2024-04-25T14:01:25.566 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:01:25.566 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:01:25.566 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbed40afaf0 msgr2=0x7fbed40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbed40afaf0 0x7fbed40b2000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbee4005f70 tx=0x7fbee4000bc0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec1080c0 msgr2=0x7fbeec1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec1080c0 0x7fbeec1a16a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00db10 tx=0x7fbedc00a250 comp rx=0 tx=0).stop 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 shutdown_connections 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbed40afaf0 0x7fbed40b2000 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec1080c0 0x7fbeec1a16a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeec1033c0 0x7fbeec19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 --2- 172.21.15.78:0/897281364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeec102180 0x7fbeec19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 >> 172.21.15.78:0/897281364 conn(0x7fbeec0fd370 msgr2=0x7fbeec1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 shutdown_connections 2024-04-25T14:01:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.568+0000 7fbef3415640 1 -- 172.21.15.78:0/897281364 wait complete. 2024-04-25T14:01:25.579 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 -- 172.21.15.78:0/2815353455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f70101760 msgr2=0x7f5f70101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2815353455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f70101760 0x7f5f70101bd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009900 tx=0x7f5f5c02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 -- 172.21.15.78:0/2815353455 shutdown_connections 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2815353455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f70102260 0x7f5f7010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2815353455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f70101760 0x7f5f70101bd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2815353455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f701050b0 0x7f5f701054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 -- 172.21.15.78:0/2815353455 >> 172.21.15.78:0/2815353455 conn(0x7f5f700fd380 msgr2=0x7f5f700ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 -- 172.21.15.78:0/2815353455 shutdown_connections 2024-04-25T14:01:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 -- 172.21.15.78:0/2815353455 wait complete. 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 Processor -- start 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.691+0000 7f5f770ee640 1 -- start start 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f770ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f70101760 0x7f5f7019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f770ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f70102260 0x7f5f7019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f770ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f701050b0 0x7f5f701a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f770ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7010f620 con 0x7f5f701050b0 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f770ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7010f4a0 con 0x7f5f70101760 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f770ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7010f7a0 con 0x7f5f70102260 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f701050b0 0x7f5f701a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f701050b0 0x7f5f701a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40274/0 (socket says 172.21.15.78:40274) 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 -- 172.21.15.78:0/2363122262 learned_addr learned my addr 172.21.15.78:0/2363122262 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 -- 172.21.15.78:0/2363122262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f70102260 msgr2=0x7f5f7019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f70101760 0x7f5f7019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f70102260 0x7f5f7019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f70102260 0x7f5f7019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 -- 172.21.15.78:0/2363122262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f70101760 msgr2=0x7f5f7019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f70101760 0x7f5f7019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 -- 172.21.15.78:0/2363122262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f701a1d10 con 0x7f5f701050b0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f70101760 0x7f5f7019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f7568b640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f701050b0 0x7f5f701a1750 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800cce0 tx=0x7f5f6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f6dffb640 1 -- 172.21.15.78:0/2363122262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800f450 con 0x7f5f701050b0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f6dffb640 1 -- 172.21.15.78:0/2363122262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6800faa0 con 0x7f5f701050b0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f701a1fa0 con 0x7f5f701050b0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.692+0000 7f5f6dffb640 1 -- 172.21.15.78:0/2363122262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68018b70 con 0x7f5f701050b0 2024-04-25T14:01:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.693+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f701a2410 con 0x7f5f701050b0 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f6dffb640 1 -- 172.21.15.78:0/2363122262 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5f68018d10 con 0x7f5f701050b0 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 Processor -- start 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 -- start start 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dad6390 0x55f62de78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dab56b0 0x55f62de79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f62dddcd80 con 0x55f62dab56b0 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f62dd96a20 con 0x55f62dad6390 2024-04-25T14:01:25.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f62ddd8d40 con 0x55f62daad4d0 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.695+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dab56b0 0x55f62de79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dab56b0 0x55f62de79a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40276/0 (socket says 172.21.15.78:40276) 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 -- 172.21.15.78:0/1992044327 learned_addr learned my addr 172.21.15.78:0/1992044327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6dffb640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f600afaf0 0x7f5f600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dad6390 0x55f62de78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 -- 172.21.15.78:0/1992044327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 msgr2=0x55f62de78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de78d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 -- 172.21.15.78:0/1992044327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dad6390 msgr2=0x55f62de78810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f7568b640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f600afaf0 0x7f5f600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dad6390 0x55f62de78810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 -- 172.21.15.78:0/1992044327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62dda1c30 con 0x55f62dab56b0 2024-04-25T14:01:25.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6dffb640 1 -- 172.21.15.78:0/2363122262 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5f680df130 con 0x7f5f701050b0 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dab56b0 0x55f62de79a20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c005940 tx=0x7f5f5c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb96849d9f62b1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f5affd640 1 -- 172.21.15.78:0/1992044327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c045070 con 0x55f62dab56b0 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f5affd640 1 -- 172.21.15.78:0/1992044327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c0035f0 con 0x55f62dab56b0 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f5affd640 1 -- 172.21.15.78:0/1992044327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c035040 con 0x55f62dab56b0 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.696+0000 7f5f7568b640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f600afaf0 0x7f5f600b2000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f680a4870 tx=0x7f5f68090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 -- 172.21.15.78:0/1992044327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dab56b0 msgr2=0x55f62de79a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dab56b0 0x55f62de79a20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c005940 tx=0x7f5f5c031400 comp rx=0 tx=0).stop 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 -- 172.21.15.78:0/1992044327 shutdown_connections 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dab56b0 0x55f62de79a20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/1992044327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dad6390 0x55f62de78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 -- 172.21.15.78:0/1992044327 >> 172.21.15.78:0/1992044327 conn(0x55f62daca4e0 msgr2=0x55f62de74020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 -- 172.21.15.78:0/1992044327 shutdown_connections 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 -- 172.21.15.78:0/1992044327 wait complete. 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.697+0000 7f5f7df9d740 1 -- start start 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 1 Processor -- start 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62daad4d0 0x55f62de825a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 0x55f62de82ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dad6390 0x55f62de89a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f62de7a4d0 con 0x55f62daad4d0 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f62dd63dc0 con 0x55f62dab56b0 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f62de7a650 con 0x55f62dad6390 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f7df9d740 10 client.0 _ll_register_callbacks cb 0x55f62dd7f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62daad4d0 0x55f62de825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62daad4d0 0x55f62de825a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40292/0 (socket says 172.21.15.78:40292) 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 -- 172.21.15.78:0/39540521 learned_addr learned my addr 172.21.15.78:0/39540521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 0x55f62de82ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 -- 172.21.15.78:0/39540521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dad6390 msgr2=0x55f62de89a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dad6390 0x55f62de89a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 -- 172.21.15.78:0/39540521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 msgr2=0x55f62de82ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 0x55f62de82ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.698+0000 7f5f6ffff640 1 -- 172.21.15.78:0/39540521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62daee3f0 con 0x55f62daad4d0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62daad4d0 0x55f62de825a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c043040 tx=0x7f5f5c031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf1f80ff9e558c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f4ffff640 1 -- 172.21.15.78:0/39540521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c04d020 con 0x55f62daad4d0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f4ffff640 1 -- 172.21.15.78:0/39540521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5f5c004000 con 0x55f62daad4d0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f5c0035f0 con 0x55f62daad4d0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f4ffff640 1 -- 172.21.15.78:0/39540521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5f5c035040 con 0x55f62daad4d0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 10 client.15972 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f62dbb87d0 con 0x55f62daad4d0 2024-04-25T14:01:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f4ffff640 1 -- 172.21.15.78:0/39540521 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c0035f0 con 0x55f62daad4d0 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f4ffff640 1 -- 172.21.15.78:0/39540521 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5f5c03a070 con 0x55f62daad4d0 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 10 client.15972 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 20 client.15972 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da94f60 0x55f62de8fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 4 client.15972 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.699+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f62ddea990 con 0x55f62da94f60 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.700+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da94f60 0x55f62de8fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.700+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da94f60 0x55f62de8fe90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58057d0bc094aa5b server_cookie=5b6b40c6cf27f2e5 in_seq=0 out_seq=0 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.700+0000 7f5f4ffff640 10 client.15972 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f4ffff640 1 -- 172.21.15.78:0/39540521 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f62ddea990 con 0x55f62da94f60 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f4ffff640 10 client.15972 handle_command_reply: tid=0 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 client.15972 shutdown 2024-04-25T14:01:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62daad4d0 msgr2=0x55f62de825a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62daad4d0 0x55f62de825a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c043040 tx=0x7f5f5c031a10 comp rx=0 tx=0).stop 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 shutdown_connections 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da94f60 0x55f62de8fe90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dad6390 0x55f62de89a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f4ffff640 0 client.15972 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 0x55f62de82ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/39540521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62daad4d0 0x55f62de825a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 >> 172.21.15.78:0/39540521 conn(0x55f62daca4e0 msgr2=0x55f62de76f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 shutdown_connections 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 1 -- 172.21.15.78:0/39540521 wait complete. 2024-04-25T14:01:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.701+0000 7f5f7df9d740 20 client.15972 trim_cache size 0 max 16384 2024-04-25T14:01:25.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 Processor -- start 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 -- start start 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 0x55f62de73f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de76e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701ada10 con 0x55f62dad6390 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701ab2e0 con 0x55f62dab56b0 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701a9130 con 0x55f62daad4d0 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de76e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de76e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40302/0 (socket says 172.21.15.78:40302) 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 -- 172.21.15.78:0/3288849661 learned_addr learned my addr 172.21.15.78:0/3288849661 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 -- 172.21.15.78:0/3288849661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 msgr2=0x55f62de73a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/3288849661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 -- 172.21.15.78:0/3288849661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 msgr2=0x55f62de73f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/3288849661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 0x55f62de73f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.715+0000 7f5f6ffff640 1 -- 172.21.15.78:0/3288849661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62de7a410 con 0x55f62dad6390 2024-04-25T14:01:25.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/3288849661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de76e20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0048f0 tx=0x7f5f5c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874198ba2a1faa0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f5a7fc640 1 -- 172.21.15.78:0/3288849661 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c04d040 con 0x55f62dad6390 2024-04-25T14:01:25.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f5a7fc640 1 -- 172.21.15.78:0/3288849661 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c041030 con 0x55f62dad6390 2024-04-25T14:01:25.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f5a7fc640 1 -- 172.21.15.78:0/3288849661 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c045040 con 0x55f62dad6390 2024-04-25T14:01:25.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f7df9d740 1 -- 172.21.15.78:0/3288849661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 msgr2=0x55f62de76e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/3288849661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de76e20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0048f0 tx=0x7f5f5c003ba0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f7df9d740 1 -- 172.21.15.78:0/3288849661 shutdown_connections 2024-04-25T14:01:25.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/3288849661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de76e20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/3288849661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62dab56b0 0x55f62de73f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/3288849661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62daad4d0 0x55f62de73a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.716+0000 7f5f7df9d740 1 -- 172.21.15.78:0/3288849661 >> 172.21.15.78:0/3288849661 conn(0x55f62daca4e0 msgr2=0x55f62de7d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 -- 172.21.15.78:0/3288849661 shutdown_connections 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 -- 172.21.15.78:0/3288849661 wait complete. 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 -- start start 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 Processor -- start 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62daad4d0 0x55f62de86550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dab56b0 0x55f62de86a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701a97a0 con 0x55f62dad6390 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701aac10 con 0x55f62daad4d0 2024-04-25T14:01:25.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701a9e70 con 0x55f62dab56b0 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f7df9d740 10 client.0 _ll_register_callbacks cb 0x7f5f5c043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de879c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40304/0 (socket says 172.21.15.78:40304) 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.717+0000 7f5f6ffff640 1 -- 172.21.15.78:0/514795140 learned_addr learned my addr 172.21.15.78:0/514795140 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dab56b0 0x55f62de86a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62daad4d0 0x55f62de86550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f6ffff640 1 -- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dab56b0 msgr2=0x55f62de86a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dab56b0 0x55f62de86a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f6ffff640 1 -- 172.21.15.78:0/514795140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62daad4d0 msgr2=0x55f62de86550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62daad4d0 0x55f62de86550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f6ffff640 1 -- 172.21.15.78:0/514795140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62de87f80 con 0x55f62dad6390 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f6ffff640 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de879c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c004a10 tx=0x7f5f5c003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35f4ab09cc7b44b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:25.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f4ffff640 1 -- 172.21.15.78:0/514795140 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c045070 con 0x55f62dad6390 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f4ffff640 1 -- 172.21.15.78:0/514795140 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5f5c0319f0 con 0x55f62dad6390 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f5c041030 con 0x55f62dad6390 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f4ffff640 1 -- 172.21.15.78:0/514795140 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5f5c04d040 con 0x55f62dad6390 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 10 client.15978 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f62de89200 con 0x55f62dad6390 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f4ffff640 1 -- 172.21.15.78:0/514795140 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c041030 con 0x55f62dad6390 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f4ffff640 1 -- 172.21.15.78:0/514795140 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5f5c03a070 con 0x55f62dad6390 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 10 client.15978 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 10 client.15978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 20 client.15978 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da6a3a0 0x55f62defa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 4 client.15978 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.718+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f62dde4800 con 0x55f62da6a3a0 2024-04-25T14:01:25.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.719+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da6a3a0 0x55f62defa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.719+0000 7f5f4ffff640 10 client.15978 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.719+0000 7f5f74e8a640 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da6a3a0 0x55f62defa5f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7430d55b134cfcbb server_cookie=823395bd888b9260 in_seq=0 out_seq=0 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.719+0000 7f5f4ffff640 1 -- 172.21.15.78:0/514795140 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f62dde4800 con 0x55f62da6a3a0 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.719+0000 7f5f4ffff640 10 client.15978 handle_command_reply: tid=0 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.719+0000 7f5f7df9d740 1 client.15978 shutdown 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 msgr2=0x55f62de879c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de879c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c004a10 tx=0x7f5f5c003ad0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 shutdown_connections 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55f62da6a3a0 0x55f62defa5f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f62dad6390 0x55f62de879c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f4ffff640 0 client.15978 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f62dab56b0 0x55f62de86a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 --2- 172.21.15.78:0/514795140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f62daad4d0 0x55f62de86550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 >> 172.21.15.78:0/514795140 conn(0x55f62daca4e0 msgr2=0x55f62de76e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 shutdown_connections 2024-04-25T14:01:25.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 1 -- 172.21.15.78:0/514795140 wait complete. 2024-04-25T14:01:25.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f7df9d740 20 client.15978 trim_cache size 0 max 16384 2024-04-25T14:01:25.726 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f600afaf0 msgr2=0x7f5f600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f600afaf0 0x7f5f600b2000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f680a4870 tx=0x7f5f68090040 comp rx=0 tx=0).stop 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f701050b0 msgr2=0x7f5f701a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f701050b0 0x7f5f701a1750 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800cce0 tx=0x7f5f6800a620 comp rx=0 tx=0).stop 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 shutdown_connections 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5f600afaf0 0x7f5f600b2000 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f701050b0 0x7f5f701a1750 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f70102260 0x7f5f7019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 --2- 172.21.15.78:0/2363122262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f70101760 0x7f5f7019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.720+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 >> 172.21.15.78:0/2363122262 conn(0x7f5f700fd380 msgr2=0x7f5f70105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.721+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 shutdown_connections 2024-04-25T14:01:25.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.721+0000 7f5f770ee640 1 -- 172.21.15.78:0/2363122262 wait complete. 2024-04-25T14:01:25.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:01:25.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 -- 172.21.15.78:0/4000527714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c1046a0 msgr2=0x7f5c5c10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 --2- 172.21.15.78:0/4000527714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c1046a0 0x7f5c5c10adf0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009ca0 tx=0x7f5c5402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 -- 172.21.15.78:0/4000527714 shutdown_connections 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 --2- 172.21.15.78:0/4000527714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c1046a0 0x7f5c5c10adf0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 --2- 172.21.15.78:0/4000527714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c5c103b40 0x7f5c5c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 --2- 172.21.15.78:0/4000527714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5c102fe0 0x7f5c5c1033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 -- 172.21.15.78:0/4000527714 >> 172.21.15.78:0/4000527714 conn(0x7f5c5c0fd380 msgr2=0x7f5c5c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 -- 172.21.15.78:0/4000527714 shutdown_connections 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.842+0000 7f5c60d06640 1 -- 172.21.15.78:0/4000527714 wait complete. 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 Processor -- start 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 -- start start 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5c102fe0 0x7f5c5c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c5c103b40 0x7f5c5c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c19f550 0x7f5c5c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10f8e0 con 0x7f5c5c19f550 2024-04-25T14:01:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10f760 con 0x7f5c5c103b40 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c60d06640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10fa60 con 0x7f5c5c102fe0 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c5ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c19f550 0x7f5c5c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c5ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c19f550 0x7f5c5c1a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40314/0 (socket says 172.21.15.78:40314) 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c5ad9d640 1 -- 172.21.15.78:0/2571492675 learned_addr learned my addr 172.21.15.78:0/2571492675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.843+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c5c103b40 0x7f5c5c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5c102fe0 0x7f5c5c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c5ad9d640 1 -- 172.21.15.78:0/2571492675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5c102fe0 msgr2=0x7f5c5c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c5ad9d640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5c102fe0 0x7f5c5c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c5ad9d640 1 -- 172.21.15.78:0/2571492675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c5c103b40 msgr2=0x7f5c5c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c5ad9d640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c5c103b40 0x7f5c5c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c5ad9d640 1 -- 172.21.15.78:0/2571492675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c1a1fe0 con 0x7f5c5c19f550 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c5c103b40 0x7f5c5c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:25.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c5ad9d640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c19f550 0x7f5c5c1a1970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097b0 tx=0x7f5c54031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c4b7fe640 1 -- 172.21.15.78:0/2571492675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54003bf0 con 0x7f5c5c19f550 2024-04-25T14:01:25.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c1a2270 con 0x7f5c5c19f550 2024-04-25T14:01:25.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c4b7fe640 1 -- 172.21.15.78:0/2571492675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c54003d90 con 0x7f5c5c19f550 2024-04-25T14:01:25.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c4b7fe640 1 -- 172.21.15.78:0/2571492675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5402fc30 con 0x7f5c5c19f550 2024-04-25T14:01:25.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.844+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1a2840 con 0x7f5c5c19f550 2024-04-25T14:01:25.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 Processor -- start 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c4b7fe640 1 -- 172.21.15.78:0/2571492675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5c54004040 con 0x7f5c5c19f550 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 -- start start 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 0x55d6bf8d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf5064d0 0x55d6bf8d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf835d80 con 0x55d6bf50e6b0 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf7efa20 con 0x55d6bf5064d0 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf831d40 con 0x55d6bf52f390 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8d2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40318/0 (socket says 172.21.15.78:40318) 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c59d9b640 1 -- 172.21.15.78:0/3480532501 learned_addr learned my addr 172.21.15.78:0/3480532501 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 0x55d6bf8d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c4b7fe640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c440afaf0 0x7f5c440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.847+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf5064d0 0x55d6bf8d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c5ad9d640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c440afaf0 0x7f5c440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c59d9b640 1 -- 172.21.15.78:0/3480532501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 msgr2=0x55d6bf8d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 0x55d6bf8d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c59d9b640 1 -- 172.21.15.78:0/3480532501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf5064d0 msgr2=0x55d6bf8d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf5064d0 0x55d6bf8d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c59d9b640 1 -- 172.21.15.78:0/3480532501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6bf7fac30 con 0x55d6bf50e6b0 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c4b7fe640 1 -- 172.21.15.78:0/2571492675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5c541003f0 con 0x7f5c5c19f550 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8d2a20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000afd0 tx=0x7f5c5000a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e2b652c0d7a3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c48ff9640 1 -- 172.21.15.78:0/3480532501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50018440 con 0x55d6bf50e6b0 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c48ff9640 1 -- 172.21.15.78:0/3480532501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50017890 con 0x55d6bf50e6b0 2024-04-25T14:01:25.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.848+0000 7f5c48ff9640 1 -- 172.21.15.78:0/3480532501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50012600 con 0x55d6bf50e6b0 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 -- 172.21.15.78:0/3480532501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf50e6b0 msgr2=0x55d6bf8d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8d2a20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000afd0 tx=0x7f5c5000a360 comp rx=0 tx=0).stop 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c5ad9d640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c440afaf0 0x7f5c440b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5c540095e0 tx=0x7f5c54004970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 -- 172.21.15.78:0/3480532501 shutdown_connections 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8d2a20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf5064d0 0x55d6bf8d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/3480532501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 0x55d6bf8d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 -- 172.21.15.78:0/3480532501 >> 172.21.15.78:0/3480532501 conn(0x55d6bf5234e0 msgr2=0x55d6bf8cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 -- 172.21.15.78:0/3480532501 shutdown_connections 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 -- 172.21.15.78:0/3480532501 wait complete. 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 -- start start 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.849+0000 7f5c67bb9740 1 Processor -- start 2024-04-25T14:01:25.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8db570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf8d34d0 con 0x55d6bf52f390 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf7bcdc0 con 0x55d6bf50e6b0 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf8d3650 con 0x55d6bf5064d0 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 10 client.0 _ll_register_callbacks cb 0x55d6bf7d8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40334/0 (socket says 172.21.15.78:40334) 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 -- 172.21.15.78:0/1061522599 learned_addr learned my addr 172.21.15.78:0/1061522599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 -- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 msgr2=0x55d6bf8db570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8db570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 -- 172.21.15.78:0/1061522599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 msgr2=0x55d6bf8dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 -- 172.21.15.78:0/1061522599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6bf5473f0 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8e2a10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000a300 tx=0x7f5c50006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f5a4561a19d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c41ffb640 1 -- 172.21.15.78:0/1061522599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50027460 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c41ffb640 1 -- 172.21.15.78:0/1061522599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5c50027600 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c50017890 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c41ffb640 1 -- 172.21.15.78:0/1061522599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5c50027800 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 10 client.15990 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.850+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d6bf6117d0 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c41ffb640 1 -- 172.21.15.78:0/1061522599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50017890 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c41ffb640 1 -- 172.21.15.78:0/1061522599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5c50012070 con 0x55d6bf52f390 2024-04-25T14:01:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c67bb9740 10 client.15990 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c67bb9740 10 client.15990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c67bb9740 20 client.15990 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4edf60 0x55d6bf8e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c67bb9740 4 client.15990 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6bf843990 con 0x55d6bf4edf60 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.851+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4edf60 0x55d6bf8e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.852+0000 7f5c41ffb640 10 client.15990 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.852+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4edf60 0x55d6bf8e8f70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40a83a50c67c27e server_cookie=243ed29e9ff0a5cb in_seq=0 out_seq=0 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.852+0000 7f5c41ffb640 1 -- 172.21.15.78:0/1061522599 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6bf843990 con 0x55d6bf4edf60 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.852+0000 7f5c41ffb640 10 client.15990 handle_command_reply: tid=0 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.852+0000 7f5c67bb9740 1 client.15990 shutdown 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 msgr2=0x55d6bf8e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8e2a10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000a300 tx=0x7f5c50006f90 comp rx=0 tx=0).stop 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 shutdown_connections 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8e2a10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4edf60 0x55d6bf8e8f70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/1061522599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8db570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 >> 172.21.15.78:0/1061522599 conn(0x55d6bf5234e0 msgr2=0x55d6bf8cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c41ffb640 0 client.15990 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 shutdown_connections 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 1 -- 172.21.15.78:0/1061522599 wait complete. 2024-04-25T14:01:25.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.853+0000 7f5c67bb9740 20 client.15990 trim_cache size 0 max 16384 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 Processor -- start 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 -- start start 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1adc20 con 0x55d6bf52f390 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1ab4f0 con 0x55d6bf50e6b0 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1a9340 con 0x55d6bf5064d0 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40348/0 (socket says 172.21.15.78:40348) 2024-04-25T14:01:25.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.866+0000 7f5c59d9b640 1 -- 172.21.15.78:0/264238226 learned_addr learned my addr 172.21.15.78:0/264238226 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c59d9b640 1 -- 172.21.15.78:0/264238226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 msgr2=0x55d6bf8cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c59d9b640 1 -- 172.21.15.78:0/264238226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 msgr2=0x55d6bf8e2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c59d9b640 1 -- 172.21.15.78:0/264238226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6bf8d5dc0 con 0x55d6bf52f390 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8ccf90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000a2d0 tx=0x7f5c5000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c197b1011570045b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c43fff640 1 -- 172.21.15.78:0/264238226 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50017560 con 0x55d6bf52f390 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c43fff640 1 -- 172.21.15.78:0/264238226 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c500237e0 con 0x55d6bf52f390 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.867+0000 7f5c43fff640 1 -- 172.21.15.78:0/264238226 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50017040 con 0x55d6bf52f390 2024-04-25T14:01:25.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- 172.21.15.78:0/264238226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 msgr2=0x55d6bf8ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8ccf90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000a2d0 tx=0x7f5c5000a330 comp rx=0 tx=0).stop 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- 172.21.15.78:0/264238226 shutdown_connections 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf52f390 0x55d6bf8ccf90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/264238226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf5064d0 0x55d6bf8cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- 172.21.15.78:0/264238226 >> 172.21.15.78:0/264238226 conn(0x55d6bf5234e0 msgr2=0x55d6bf8d62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- 172.21.15.78:0/264238226 shutdown_connections 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- 172.21.15.78:0/264238226 wait complete. 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- start start 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 Processor -- start 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf5064d0 0x55d6bf8dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 0x55d6bf8e11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1a99b0 con 0x55d6bf5064d0 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1aae20 con 0x55d6bf50e6b0 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1aa080 con 0x55d6bf52f390 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c67bb9740 10 client.0 _ll_register_callbacks cb 0x7f5c4c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf5064d0 0x55d6bf8dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c59d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf5064d0 0x55d6bf8dfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40364/0 (socket says 172.21.15.78:40364) 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.868+0000 7f5c59d9b640 1 -- 172.21.15.78:0/2442480664 learned_addr learned my addr 172.21.15.78:0/2442480664 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c59d9b640 1 -- 172.21.15.78:0/2442480664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 msgr2=0x55d6bf8e11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 0x55d6bf8e11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c59d9b640 1 -- 172.21.15.78:0/2442480664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 msgr2=0x55d6bf8e02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c59d9b640 1 -- 172.21.15.78:0/2442480664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6bf8e2e70 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c59d9b640 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf5064d0 0x55d6bf8dfd80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5c50004810 tx=0x7f5c5000a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c8f565adac9508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c41ffb640 1 -- 172.21.15.78:0/2442480664 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50023d80 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c41ffb640 1 -- 172.21.15.78:0/2442480664 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5c50016ea0 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c500237e0 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c41ffb640 1 -- 172.21.15.78:0/2442480664 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5c500170a0 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c67bb9740 10 client.15996 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d6bf8e27d0 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c41ffb640 1 -- 172.21.15.78:0/2442480664 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c500237e0 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.869+0000 7f5c41ffb640 1 -- 172.21.15.78:0/2442480664 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5c50012070 con 0x55d6bf5064d0 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c67bb9740 10 client.15996 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c67bb9740 10 client.15996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c67bb9740 20 client.15996 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:25.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4c33a0 0x55d6bf953360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c67bb9740 4 client.15996 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6bf83d800 con 0x55d6bf4c33a0 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4c33a0 0x55d6bf953360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c41ffb640 10 client.15996 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.870+0000 7f5c5a59c640 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4c33a0 0x55d6bf953360 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffd93b409e213c5d server_cookie=ea2b53e6dc79381c in_seq=0 out_seq=0 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c41ffb640 1 -- 172.21.15.78:0/2442480664 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6bf83d800 con 0x55d6bf4c33a0 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c41ffb640 10 client.15996 handle_command_reply: tid=0 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 client.15996 shutdown 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf5064d0 msgr2=0x55d6bf8dfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf5064d0 0x55d6bf8dfd80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5c50004810 tx=0x7f5c5000a880 comp rx=0 tx=0).stop 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 shutdown_connections 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d6bf4c33a0 0x55d6bf953360 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6bf52f390 0x55d6bf8e11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6bf50e6b0 0x55d6bf8e02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 --2- 172.21.15.78:0/2442480664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6bf5064d0 0x55d6bf8dfd80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c41ffb640 0 client.15996 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 >> 172.21.15.78:0/2442480664 conn(0x55d6bf5234e0 msgr2=0x55d6bf8cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 shutdown_connections 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 1 -- 172.21.15.78:0/2442480664 wait complete. 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c67bb9740 20 client.15996 trim_cache size 0 max 16384 2024-04-25T14:01:25.878 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:01:25.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c440afaf0 msgr2=0x7f5c440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c440afaf0 0x7f5c440b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5c540095e0 tx=0x7f5c54004970 comp rx=0 tx=0).stop 2024-04-25T14:01:25.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c19f550 msgr2=0x7f5c5c1a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c19f550 0x7f5c5c1a1970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097b0 tx=0x7f5c54031840 comp rx=0 tx=0).stop 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 shutdown_connections 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c440afaf0 0x7f5c440b2000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c5c19f550 0x7f5c5c1a1970 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c5c103b40 0x7f5c5c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 --2- 172.21.15.78:0/2571492675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5c102fe0 0x7f5c5c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 >> 172.21.15.78:0/2571492675 conn(0x7f5c5c0fd380 msgr2=0x7f5c5c106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.871+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 shutdown_connections 2024-04-25T14:01:25.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.872+0000 7f5c60d06640 1 -- 172.21.15.78:0/2571492675 wait complete. 2024-04-25T14:01:25.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:25 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/897281364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:25.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:01:25.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:25 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/897281364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 -- 172.21.15.78:0/2595504113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a0108920 msgr2=0x7fc8a010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 --2- 172.21.15.78:0/2595504113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a0108920 0x7fc8a010ad40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc898009ca0 tx=0x7fc89802a8e0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 -- 172.21.15.78:0/2595504113 shutdown_connections 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 --2- 172.21.15.78:0/2595504113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a0108920 0x7fc8a010ad40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 --2- 172.21.15.78:0/2595504113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8a0103c20 0x7fc8a0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 --2- 172.21.15.78:0/2595504113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a01029e0 0x7fc8a01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 -- 172.21.15.78:0/2595504113 >> 172.21.15.78:0/2595504113 conn(0x7fc8a00fdbd0 msgr2=0x7fc8a0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:25.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 -- 172.21.15.78:0/2595504113 shutdown_connections 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 -- 172.21.15.78:0/2595504113 wait complete. 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.992+0000 7fc8a8051640 1 Processor -- start 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 -- start start 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a01029e0 0x7fc8a006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8a0103c20 0x7fc8a019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a0108920 0x7fc8a01a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a010e560 con 0x7fc8a01029e0 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a010e3e0 con 0x7fc8a0108920 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a010e6e0 con 0x7fc8a0103c20 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a01029e0 0x7fc8a006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a01029e0 0x7fc8a006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40376/0 (socket says 172.21.15.78:40376) 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 -- 172.21.15.78:0/4259371507 learned_addr learned my addr 172.21.15.78:0/4259371507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:25.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8a0103c20 0x7fc8a019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 -- 172.21.15.78:0/4259371507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8a0103c20 msgr2=0x7fc8a019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8a0103c20 0x7fc8a019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 -- 172.21.15.78:0/4259371507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a0108920 msgr2=0x7fc8a01a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a0108920 0x7fc8a01a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 -- 172.21.15.78:0/4259371507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a01a1fc0 con 0x7fc8a01029e0 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8a0103c20 0x7fc8a019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a01029e0 0x7fc8a006a920 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc89000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc896ffd640 1 -- 172.21.15.78:0/4259371507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890011830 con 0x7fc8a01029e0 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a01a22b0 con 0x7fc8a01029e0 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc896ffd640 1 -- 172.21.15.78:0/4259371507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc890016070 con 0x7fc8a01029e0 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.993+0000 7fc896ffd640 1 -- 172.21.15.78:0/4259371507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890010e00 con 0x7fc8a01029e0 2024-04-25T14:01:25.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.994+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a01a2880 con 0x7fc8a01029e0 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 Processor -- start 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 -- start start 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc896ffd640 1 -- 172.21.15.78:0/4259371507 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc89001b460 con 0x7fc8a01029e0 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e08a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564db3d6bd80 con 0x564db3a446b0 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564db3d25a20 con 0x564db3a65390 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564db3d67d40 con 0x564db3a3c4d0 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8a55ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e08a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8a55ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e08a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40384/0 (socket says 172.21.15.78:40384) 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.996+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3492491368 learned_addr learned my addr 172.21.15.78:0/3492491368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3492491368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 msgr2=0x564db3e07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3492491368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 msgr2=0x564db3e07810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e07810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3492491368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db3d30c30 con 0x564db3a446b0 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e08a20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc89c008c30 tx=0x7fc89c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd98382f72c40005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc896ffd640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc88c0afaf0 0x7fc88c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc88bfff640 1 -- 172.21.15.78:0/3492491368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c014070 con 0x564db3a446b0 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc88bfff640 1 -- 172.21.15.78:0/3492491368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c0047a0 con 0x564db3a446b0 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc88bfff640 1 -- 172.21.15.78:0/3492491368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c010690 con 0x564db3a446b0 2024-04-25T14:01:26.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc88c0afaf0 0x7fc88c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.997+0000 7fc896ffd640 1 -- 172.21.15.78:0/4259371507 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc8900de340 con 0x7fc8a01029e0 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3492491368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 msgr2=0x564db3e08a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc88c0afaf0 0x7fc88c0b2000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc898009a60 tx=0x7fc898031830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e08a20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc89c008c30 tx=0x7fc89c00e720 comp rx=0 tx=0).stop 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3492491368 shutdown_connections 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e08a20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3492491368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e07810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3492491368 >> 172.21.15.78:0/3492491368 conn(0x564db3a594e0 msgr2=0x564db3e03020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3492491368 shutdown_connections 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3492491368 wait complete. 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.998+0000 7fc8aefe7740 1 -- start start 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 1 Processor -- start 2024-04-25T14:01:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e11570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e11ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564db3e09530 con 0x564db3a446b0 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564db3cf2dc0 con 0x564db3a65390 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564db3e096b0 con 0x564db3a3c4d0 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8aefe7740 10 client.0 _ll_register_callbacks cb 0x564db3d0e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e11ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e11ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40394/0 (socket says 172.21.15.78:40394) 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a5ded640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e11570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 -- 172.21.15.78:0/2248270489 learned_addr learned my addr 172.21.15.78:0/2248270489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 -- 172.21.15.78:0/2248270489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 msgr2=0x564db3e11570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e11570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 -- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 msgr2=0x564db3e18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e18a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:25.999+0000 7fc8a55ec640 1 -- 172.21.15.78:0/2248270489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db3a7d3f0 con 0x564db3a446b0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e11ab0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000c00 tx=0x7fc89c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3766e5faab6ea7af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8897fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8897fa640 1 -- 172.21.15.78:0/2248270489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c014030 con 0x564db3a446b0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8897fa640 1 -- 172.21.15.78:0/2248270489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc89c004050 con 0x564db3a446b0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c0047a0 con 0x564db3a446b0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8897fa640 1 -- 172.21.15.78:0/2248270489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc89c004250 con 0x564db3a446b0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 10 client.16008 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564db3b477d0 con 0x564db3a446b0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8897fa640 1 -- 172.21.15.78:0/2248270489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c0047a0 con 0x564db3a446b0 2024-04-25T14:01:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8897fa640 1 -- 172.21.15.78:0/2248270489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc89c02c050 con 0x564db3a446b0 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 10 client.16008 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 20 client.16008 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db3a23f60 0x564db3e1efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 4 client.16008 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.000+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564db3d79990 con 0x564db3a23f60 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.001+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db3a23f60 0x564db3e1efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.001+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db3a23f60 0x564db3e1efe0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa0f3c8a146c59c server_cookie=9d1434b3bed1c4b3 in_seq=0 out_seq=0 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.001+0000 7fc8897fa640 10 client.16008 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8897fa640 1 -- 172.21.15.78:0/2248270489 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc890081e50 con 0x564db3a23f60 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8897fa640 10 client.16008 handle_command_reply: tid=0 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 client.16008 shutdown 2024-04-25T14:01:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 msgr2=0x564db3e11ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e11ab0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000c00 tx=0x7fc89c005230 comp rx=0 tx=0).stop 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 shutdown_connections 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a65390 0x564db3e18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e11ab0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db3a23f60 0x564db3e1efe0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/2248270489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e11570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8897fa640 0 client.16008 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 >> 172.21.15.78:0/2248270489 conn(0x564db3a594e0 msgr2=0x564db3e05f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 shutdown_connections 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 1 -- 172.21.15.78:0/2248270489 wait complete. 2024-04-25T14:01:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.002+0000 7fc8aefe7740 20 client.16008 trim_cache size 0 max 16384 2024-04-25T14:01:26.021 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:25 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/897281364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 Processor -- start 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 -- start start 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a3c4d0 0x564db3e247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e24ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a65390 0x564db3e05ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01adc70 con 0x564db3a446b0 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01ab540 con 0x564db3a3c4d0 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01a9390 con 0x564db3a65390 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e24ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e24ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40410/0 (socket says 172.21.15.78:40410) 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3262880593 learned_addr learned my addr 172.21.15.78:0/3262880593 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:26.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a3c4d0 0x564db3e247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a65390 0x564db3e05ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3262880593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a65390 msgr2=0x564db3e05ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a65390 0x564db3e05ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3262880593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a3c4d0 msgr2=0x564db3e247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a3c4d0 0x564db3e247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 -- 172.21.15.78:0/3262880593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db3e0bdc0 con 0x564db3a446b0 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a3c4d0 0x564db3e247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a65390 0x564db3e05ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.016+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e24ce0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc89c003ec0 tx=0x7fc89c008f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de8410a3e6cf5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc88b7fe640 1 -- 172.21.15.78:0/3262880593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c014070 con 0x564db3a446b0 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc88b7fe640 1 -- 172.21.15.78:0/3262880593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c003fe0 con 0x564db3a446b0 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc88b7fe640 1 -- 172.21.15.78:0/3262880593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c010980 con 0x564db3a446b0 2024-04-25T14:01:26.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3262880593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 msgr2=0x564db3e24ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e24ce0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc89c003ec0 tx=0x7fc89c008f30 comp rx=0 tx=0).stop 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3262880593 shutdown_connections 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a65390 0x564db3e05ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a446b0 0x564db3e24ce0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3262880593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a3c4d0 0x564db3e247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.017+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3262880593 >> 172.21.15.78:0/3262880593 conn(0x564db3a594e0 msgr2=0x564db3e0c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3262880593 shutdown_connections 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3262880593 wait complete. 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 -- start start 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 Processor -- start 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e16ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a446b0 0x564db3e170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a65390 0x564db3e85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01a9a00 con 0x564db3a65390 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01aae70 con 0x564db3a446b0 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a01aa0d0 con 0x564db3a3c4d0 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8aefe7740 10 client.0 _ll_register_callbacks cb 0x7fc89c008ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8a65ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a65390 0x564db3e85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8a65ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a65390 0x564db3e85340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40416/0 (socket says 172.21.15.78:40416) 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8a65ee640 1 -- 172.21.15.78:0/3201619186 learned_addr learned my addr 172.21.15.78:0/3201619186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.018+0000 7fc8a55ec640 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a446b0 0x564db3e170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e16ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8a65ee640 1 -- 172.21.15.78:0/3201619186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 msgr2=0x564db3e16ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e16ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8a65ee640 1 -- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a446b0 msgr2=0x564db3e170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a446b0 0x564db3e170e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8a65ee640 1 -- 172.21.15.78:0/3201619186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db3e178c0 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8a65ee640 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a65390 0x564db3e85340 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc898003bf0 tx=0x7fc8980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7337431695533d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8897fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8897fa640 1 -- 172.21.15.78:0/3201619186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8980045d0 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8897fa640 1 -- 172.21.15.78:0/3201619186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc898004770 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c003fe0 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8897fa640 1 -- 172.21.15.78:0/3201619186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc8980305e0 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8aefe7740 10 client.16014 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564db3e86790 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.019+0000 7fc8897fa640 1 -- 172.21.15.78:0/3201619186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c003fe0 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8897fa640 1 -- 172.21.15.78:0/3201619186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc89804f020 con 0x564db3a65390 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8aefe7740 10 client.16014 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8aefe7740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:26.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8aefe7740 20 client.16014 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db39f93a0 0x564db3e891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8aefe7740 4 client.16014 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564db3d73800 con 0x564db39f93a0 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db39f93a0 0x564db3e891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8a5ded640 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db39f93a0 0x564db3e891b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753493de48df5a49 server_cookie=6f69081373800aa9 in_seq=0 out_seq=0 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8897fa640 10 client.16014 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8897fa640 1 -- 172.21.15.78:0/3201619186 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564db3d73800 con 0x564db39f93a0 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8897fa640 10 client.16014 handle_command_reply: tid=0 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.020+0000 7fc8aefe7740 1 client.16014 shutdown 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a65390 msgr2=0x564db3e85340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a65390 0x564db3e85340 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc898003bf0 tx=0x7fc8980095c0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 shutdown_connections 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564db3a65390 0x564db3e85340 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564db3a446b0 0x564db3e170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564db39f93a0 0x564db3e891b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 --2- 172.21.15.78:0/3201619186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564db3a3c4d0 0x564db3e16ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8897fa640 0 client.16014 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 >> 172.21.15.78:0/3201619186 conn(0x564db3a594e0 msgr2=0x564db3e05ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 shutdown_connections 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 1 -- 172.21.15.78:0/3201619186 wait complete. 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8aefe7740 20 client.16014 trim_cache size 0 max 16384 2024-04-25T14:01:26.028 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:01:26.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc88c0afaf0 msgr2=0x7fc88c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:26.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc88c0afaf0 0x7fc88c0b2000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc898009a60 tx=0x7fc898031830 comp rx=0 tx=0).stop 2024-04-25T14:01:26.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a01029e0 msgr2=0x7fc8a006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:26.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a01029e0 0x7fc8a006a920 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc89000b8e0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 shutdown_connections 2024-04-25T14:01:26.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8a0108920 0x7fc8a01a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc8a0103c20 0x7fc8a019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc88c0afaf0 0x7fc88c0b2000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 --2- 172.21.15.78:0/4259371507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a01029e0 0x7fc8a006a920 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:26.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 >> 172.21.15.78:0/4259371507 conn(0x7fc8a00fdbd0 msgr2=0x7fc8a0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:26.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 shutdown_connections 2024-04-25T14:01:26.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:26.021+0000 7fc8a8051640 1 -- 172.21.15.78:0/4259371507 wait complete. 2024-04-25T14:01:26.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:26 smithi129 ceph-mon[32569]: pgmap v406: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:26.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:26 smithi078 ceph-mon[29163]: pgmap v406: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:27.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:26 smithi112 ceph-mon[32617]: pgmap v406: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:27.033 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:01:27.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:01:27.034 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:01:27.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 -- 172.21.15.78:0/3902953274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c103c30 msgr2=0x7fa34c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 --2- 172.21.15.78:0/3902953274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c103c30 0x7fa34c104040 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa340009900 tx=0x7fa34002a410 comp rx=0 tx=0).stop 2024-04-25T14:01:27.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 -- 172.21.15.78:0/3902953274 shutdown_connections 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 --2- 172.21.15.78:0/3902953274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa34c108930 0x7fa34c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 --2- 172.21.15.78:0/3902953274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c103c30 0x7fa34c104040 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 --2- 172.21.15.78:0/3902953274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34c1029f0 0x7fa34c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 -- 172.21.15.78:0/3902953274 >> 172.21.15.78:0/3902953274 conn(0x7fa34c0fdbe0 msgr2=0x7fa34c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 -- 172.21.15.78:0/3902953274 shutdown_connections 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.167+0000 7fa352784640 1 -- 172.21.15.78:0/3902953274 wait complete. 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 Processor -- start 2024-04-25T14:01:27.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 -- start start 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c1029f0 0x7fa34c06bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34c103c30 0x7fa34c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa34c108930 0x7fa34c1a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10e4f0 con 0x7fa34c1029f0 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10e370 con 0x7fa34c103c30 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa352784640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10e670 con 0x7fa34c108930 2024-04-25T14:01:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c1029f0 0x7fa34c06bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c1029f0 0x7fa34c06bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60940/0 (socket says 172.21.15.78:60940) 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.168+0000 7fa34bfff640 1 -- 172.21.15.78:0/3602020575 learned_addr learned my addr 172.21.15.78:0/3602020575 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34c103c30 0x7fa34c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa350d21640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa34c108930 0x7fa34c1a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa34bfff640 1 -- 172.21.15.78:0/3602020575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa34c108930 msgr2=0x7fa34c1a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa34bfff640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa34c108930 0x7fa34c1a39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa34bfff640 1 -- 172.21.15.78:0/3602020575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34c103c30 msgr2=0x7fa34c1a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa34bfff640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34c103c30 0x7fa34c1a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa34bfff640 1 -- 172.21.15.78:0/3602020575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c1a3ff0 con 0x7fa34c1029f0 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa34bfff640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c1029f0 0x7fa34c06bd20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00e9f0 tx=0x7fa33c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa3497fa640 1 -- 172.21.15.78:0/3602020575 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00ccc0 con 0x7fa34c1029f0 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa3497fa640 1 -- 172.21.15.78:0/3602020575 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c0035f0 con 0x7fa34c1029f0 2024-04-25T14:01:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa3497fa640 1 -- 172.21.15.78:0/3602020575 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00ce60 con 0x7fa34c1029f0 2024-04-25T14:01:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c1a42e0 con 0x7fa34c1029f0 2024-04-25T14:01:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.169+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c1a4830 con 0x7fa34c1029f0 2024-04-25T14:01:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 Processor -- start 2024-04-25T14:01:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 -- start start 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa3497fa640 1 -- 172.21.15.78:0/3602020575 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa33c004020 con 0x7fa34c1029f0 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x5590442854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x559044285a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ec3730 0x55904428c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5590441eab00 con 0x559043ebb550 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559044186920 con 0x559043ee4410 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa359638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5590441e6ac0 con 0x559043ec3730 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x559044285a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x559044285a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60952/0 (socket says 172.21.15.78:60952) 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa34bfff640 1 -- 172.21.15.78:0/2939328611 learned_addr learned my addr 172.21.15.78:0/2939328611 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ec3730 0x55904428c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.172+0000 7fa350d21640 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x5590442854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa34bfff640 1 -- 172.21.15.78:0/2939328611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ec3730 msgr2=0x55904428c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa34bfff640 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ec3730 0x55904428c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa34bfff640 1 -- 172.21.15.78:0/2939328611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 msgr2=0x5590442854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa34bfff640 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x5590442854a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa34bfff640 1 -- 172.21.15.78:0/2939328611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559043fc47e0 con 0x559043ebb550 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa3497fa640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa3340afaf0 0x7fa3340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa34bfff640 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x559044285a20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0e2fd0 tx=0x7fa33c0e71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c981876987bf802e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa350d21640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa3340afaf0 0x7fa3340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa3327fc640 1 -- 172.21.15.78:0/2939328611 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0a33f0 con 0x559043ebb550 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa3497fa640 1 -- 172.21.15.78:0/3602020575 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa33c0defc0 con 0x7fa34c1029f0 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa3327fc640 1 -- 172.21.15.78:0/2939328611 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c004310 con 0x559043ebb550 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.173+0000 7fa3327fc640 1 -- 172.21.15.78:0/2939328611 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0abb40 con 0x559043ebb550 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 -- 172.21.15.78:0/2939328611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 msgr2=0x559044285a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa350d21640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa3340afaf0 0x7fa3340b2000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x559044286670 tx=0x7fa3440096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x559044285a20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0e2fd0 tx=0x7fa33c0e71a0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 -- 172.21.15.78:0/2939328611 shutdown_connections 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ec3730 0x55904428c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x559044285a20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 --2- 172.21.15.78:0/2939328611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x5590442854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 -- 172.21.15.78:0/2939328611 >> 172.21.15.78:0/2939328611 conn(0x559043ed8560 msgr2=0x559044280e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 -- 172.21.15.78:0/2939328611 shutdown_connections 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.174+0000 7fa359638740 1 -- 172.21.15.78:0/2939328611 wait complete. 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 -- start start 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 Processor -- start 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x55904428f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ec3730 0x55904428f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590442966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55904428d370 con 0x559043ebb550 2024-04-25T14:01:27.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590441845d0 con 0x559043ec3730 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55904428d4f0 con 0x559043ee4410 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa359638740 10 client.0 _ll_register_callbacks cb 0x559044172080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x55904428f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x55904428f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60968/0 (socket says 172.21.15.78:60968) 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.175+0000 7fa34bfff640 1 -- 172.21.15.78:0/1432115387 learned_addr learned my addr 172.21.15.78:0/1432115387 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ec3730 0x55904428f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa350d21640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590442966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34bfff640 1 -- 172.21.15.78:0/1432115387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 msgr2=0x5590442966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590442966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34bfff640 1 -- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ec3730 msgr2=0x55904428f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ec3730 0x55904428f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34bfff640 1 -- 172.21.15.78:0/1432115387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559044024f70 con 0x559043ebb550 2024-04-25T14:01:27.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ec3730 0x55904428f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x55904428f240 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0a35b0 tx=0x7fa33c0a7120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93c076d2329a74a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa32b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1432115387 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0a6ca0 con 0x559043ebb550 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1432115387 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa33c0a79a0 con 0x559043ebb550 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590441f8710 con 0x559043ebb550 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1432115387 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa33c0a6e80 con 0x559043ebb550 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa359638740 10 client.16026 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559043ebc170 con 0x559043ebb550 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1432115387 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5590441f8710 con 0x559043ebb550 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1432115387 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa33c0aff00 con 0x559043ebb550 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.176+0000 7fa359638740 10 client.16026 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa359638740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa359638740 20 client.16026 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:27.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa359638740 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043ea31f0 0x55904429c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa359638740 4 client.16026 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590441f2580 con 0x559043ea31f0 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043ea31f0 0x55904429c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa32b7fe640 10 client.16026 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.177+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043ea31f0 0x55904429c7a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6896195b0a911da server_cookie=9d5f474dbf09f384 in_seq=0 out_seq=0 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.178+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1432115387 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590441f2580 con 0x559043ea31f0 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.178+0000 7fa32b7fe640 10 client.16026 handle_command_reply: tid=0 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.178+0000 7fa359638740 1 client.16026 shutdown 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.178+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 msgr2=0x55904428f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.178+0000 7fa359638740 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x55904428f240 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0a35b0 tx=0x7fa33c0a7120 comp rx=0 tx=0).stop 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 shutdown_connections 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043ea31f0 0x55904429c7a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590442966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ec3730 0x55904428f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa32b7fe640 0 client.16026 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 --2- 172.21.15.78:0/1432115387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ebb550 0x55904428f240 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 >> 172.21.15.78:0/1432115387 conn(0x559043ed8560 msgr2=0x55904427fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 shutdown_connections 2024-04-25T14:01:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 1 -- 172.21.15.78:0/1432115387 wait complete. 2024-04-25T14:01:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.179+0000 7fa359638740 20 client.16026 trim_cache size 0 max 16384 2024-04-25T14:01:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 Processor -- start 2024-04-25T14:01:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 -- start start 2024-04-25T14:01:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ebb550 0x55904428f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x559044296bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1afed0 con 0x559043ec3730 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ad7a0 con 0x559043ee4410 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa359638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ab5f0 con 0x559043ebb550 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60970/0 (socket says 172.21.15.78:60970) 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1843714523 learned_addr learned my addr 172.21.15.78:0/1843714523 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ebb550 0x55904428f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa350d21640 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x559044296bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1843714523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ebb550 msgr2=0x55904428f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ebb550 0x55904428f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1843714523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 msgr2=0x559044296bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x559044296bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1843714523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904427c2c0 con 0x559043ec3730 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa350d21640 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x559044296bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.197+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296670 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa340031440 tx=0x7fa3400314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38980cf42ad366d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa331ffb640 1 -- 172.21.15.78:0/1843714523 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340035040 con 0x559043ec3730 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa331ffb640 1 -- 172.21.15.78:0/1843714523 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa340005040 con 0x559043ec3730 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa331ffb640 1 -- 172.21.15.78:0/1843714523 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340004840 con 0x559043ec3730 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa359638740 1 -- 172.21.15.78:0/1843714523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 msgr2=0x559044296670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa359638740 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296670 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa340031440 tx=0x7fa3400314a0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa359638740 1 -- 172.21.15.78:0/1843714523 shutdown_connections 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa359638740 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ee4410 0x559044296bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa359638740 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296670 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa359638740 1 --2- 172.21.15.78:0/1843714523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ebb550 0x55904428f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.198+0000 7fa359638740 1 -- 172.21.15.78:0/1843714523 >> 172.21.15.78:0/1843714523 conn(0x559043ed8560 msgr2=0x559044284660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 -- 172.21.15.78:0/1843714523 shutdown_connections 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 -- 172.21.15.78:0/1843714523 wait complete. 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 -- start start 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 Processor -- start 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ebb550 0x559044295be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590443028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1abc60 con 0x559043ec3730 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ad0d0 con 0x559043ebb550 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c1ac330 con 0x559043ee4410 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa359638740 10 client.0 _ll_register_callbacks cb 0x7fa33c0e2e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa34b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa34b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60986/0 (socket says 172.21.15.78:60986) 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.199+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1451411959 learned_addr learned my addr 172.21.15.78:0/1451411959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ebb550 0x559044295be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1451411959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 msgr2=0x5590443028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa350d21640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590443028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590443028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ebb550 msgr2=0x559044295be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ebb550 0x559044295be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34b7fe640 1 -- 172.21.15.78:0/1451411959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559044302df0 con 0x559043ec3730 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa350d21640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590443028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ebb550 0x559044295be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa34b7fe640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296180 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa340004180 tx=0x7fa3400041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d279e6cb39e4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa32b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1451411959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340035030 con 0x559043ec3730 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1451411959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa340042040 con 0x559043ec3730 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa340005040 con 0x559043ec3730 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1451411959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa340005400 con 0x559043ec3730 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa359638740 10 client.16032 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559044303f60 con 0x559043ec3730 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.200+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1451411959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa340005040 con 0x559043ec3730 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1451411959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa340053030 con 0x559043ec3730 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa359638740 10 client.16032 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa359638740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa359638740 20 client.16032 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa359638740 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043e78630 0x559044306960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa359638740 4 client.16032 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55904416dac0 con 0x559043e78630 2024-04-25T14:01:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043e78630 0x559044306960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa32b7fe640 10 client.16032 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.201+0000 7fa34bfff640 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043e78630 0x559044306960 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8918b79347cae2 server_cookie=2bd9fe4d3a259989 in_seq=0 out_seq=0 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa32b7fe640 1 -- 172.21.15.78:0/1451411959 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55904416dac0 con 0x559043e78630 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa32b7fe640 10 client.16032 handle_command_reply: tid=0 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 client.16032 shutdown 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 msgr2=0x559044296180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296180 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa340004180 tx=0x7fa3400041b0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 shutdown_connections 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559043e78630 0x559044306960 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559043ee4410 0x5590443028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559043ec3730 0x559044296180 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa32b7fe640 0 client.16032 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 --2- 172.21.15.78:0/1451411959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559043ebb550 0x559044295be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 >> 172.21.15.78:0/1451411959 conn(0x559043ed8560 msgr2=0x559044284660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 shutdown_connections 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 1 -- 172.21.15.78:0/1451411959 wait complete. 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.203+0000 7fa359638740 20 client.16032 trim_cache size 0 max 16384 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "aa832fea-8459-47f9-a1fe-7925d5a38428", 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:01:27.209 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:01:27.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa3340afaf0 msgr2=0x7fa3340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa3340afaf0 0x7fa3340b2000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x559044286670 tx=0x7fa3440096c0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c1029f0 msgr2=0x7fa34c06bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c1029f0 0x7fa34c06bd20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00e9f0 tx=0x7fa33c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 shutdown_connections 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa34c108930 0x7fa34c1a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa34c103c30 0x7fa34c1a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa3340afaf0 0x7fa3340b2000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 --2- 172.21.15.78:0/3602020575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa34c1029f0 0x7fa34c06bd20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 >> 172.21.15.78:0/3602020575 conn(0x7fa34c0fdbe0 msgr2=0x7fa34c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 shutdown_connections 2024-04-25T14:01:27.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.204+0000 7fa352784640 1 -- 172.21.15.78:0/3602020575 wait complete. 2024-04-25T14:01:27.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:01:27.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:01:27.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.326+0000 7fd134461640 1 -- 172.21.15.78:0/1116900338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c103ae0 msgr2=0x7fd12c1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.326+0000 7fd134461640 1 --2- 172.21.15.78:0/1116900338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c103ae0 0x7fd12c1047e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd128009900 tx=0x7fd12802a410 comp rx=0 tx=0).stop 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- 172.21.15.78:0/1116900338 shutdown_connections 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 --2- 172.21.15.78:0/1116900338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd12c109a20 0x7fd12c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 --2- 172.21.15.78:0/1116900338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd12c104d20 0x7fd12c105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 --2- 172.21.15.78:0/1116900338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c103ae0 0x7fd12c1047e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- 172.21.15.78:0/1116900338 >> 172.21.15.78:0/1116900338 conn(0x7fd12c0fecd0 msgr2=0x7fd12c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- 172.21.15.78:0/1116900338 shutdown_connections 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- 172.21.15.78:0/1116900338 wait complete. 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 Processor -- start 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- start start 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd12c103ae0 0x7fd12c076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c104d20 0x7fd12c074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd12c109a20 0x7fd12c075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c10f5e0 con 0x7fd12c104d20 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c10f460 con 0x7fd12c103ae0 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd134461640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c10f760 con 0x7fd12c109a20 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd1319fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c104d20 0x7fd12c074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd1319fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c104d20 0x7fd12c074ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32774/0 (socket says 172.21.15.78:32774) 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.327+0000 7fd1319fc640 1 -- 172.21.15.78:0/473733998 learned_addr learned my addr 172.21.15.78:0/473733998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1321fd640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd12c103ae0 0x7fd12c076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1329fe640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd12c109a20 0x7fd12c075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1319fc640 1 -- 172.21.15.78:0/473733998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd12c109a20 msgr2=0x7fd12c075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1319fc640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd12c109a20 0x7fd12c075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1319fc640 1 -- 172.21.15.78:0/473733998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd12c103ae0 msgr2=0x7fd12c076850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1319fc640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd12c103ae0 0x7fd12c076850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1319fc640 1 -- 172.21.15.78:0/473733998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c075b60 con 0x7fd12c104d20 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1321fd640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd12c103ae0 0x7fd12c076850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1319fc640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c104d20 0x7fd12c074ef0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e9f0 tx=0x7fd11c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1237fe640 1 -- 172.21.15.78:0/473733998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00cd80 con 0x7fd12c104d20 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1237fe640 1 -- 172.21.15.78:0/473733998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11c0054d0 con 0x7fd12c104d20 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c1a42d0 con 0x7fd12c104d20 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd1237fe640 1 -- 172.21.15.78:0/473733998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c010690 con 0x7fd12c104d20 2024-04-25T14:01:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.328+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c1a4740 con 0x7fd12c104d20 2024-04-25T14:01:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd1237fe640 1 -- 172.21.15.78:0/473733998 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd11c004e90 con 0x7fd12c104d20 2024-04-25T14:01:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 Processor -- start 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 -- start start 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560449150d80 con 0x560448e4a390 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56044910a8b0 con 0x560448e296b0 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56044914cd40 con 0x560448e214d0 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd1329fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd1329fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491eba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32776/0 (socket says 172.21.15.78:32776) 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd1329fe640 1 -- 172.21.15.78:0/2336771794 learned_addr learned my addr 172.21.15.78:0/2336771794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd1319fc640 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.331+0000 7fd1321fd640 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1237fe640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1180afaf0 0x7fd1180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1329fe640 1 -- 172.21.15.78:0/2336771794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 msgr2=0x5604491ebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1329fe640 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491ebfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1329fe640 1 -- 172.21.15.78:0/2336771794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 msgr2=0x5604491ecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1329fe640 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491ecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1329fe640 1 -- 172.21.15.78:0/2336771794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604490dba40 con 0x560448e4a390 2024-04-25T14:01:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1237fe640 1 -- 172.21.15.78:0/473733998 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd11c0df150 con 0x7fd12c104d20 2024-04-25T14:01:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1329fe640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1180afaf0 0x7fd1180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd1329fe640 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491eba60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd1240103f0 tx=0x7fd1240108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0cb1b4449ddcd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd120bfe640 1 -- 172.21.15.78:0/2336771794 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12401c900 con 0x560448e4a390 2024-04-25T14:01:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd120bfe640 1 -- 172.21.15.78:0/2336771794 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1240113e0 con 0x560448e4a390 2024-04-25T14:01:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.332+0000 7fd120bfe640 1 -- 172.21.15.78:0/2336771794 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12401b610 con 0x560448e4a390 2024-04-25T14:01:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 -- 172.21.15.78:0/2336771794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 msgr2=0x5604491eba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd1329fe640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1180afaf0 0x7fd1180b2000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd124020000 tx=0x7fd12401d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491eba60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd1240103f0 tx=0x7fd1240108b0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 -- 172.21.15.78:0/2336771794 shutdown_connections 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491ecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491ebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/2336771794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491eba60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 -- 172.21.15.78:0/2336771794 >> 172.21.15.78:0/2336771794 conn(0x560448e3e4e0 msgr2=0x5604491e7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 -- 172.21.15.78:0/2336771794 shutdown_connections 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.333+0000 7fd13b3ee740 1 -- 172.21.15.78:0/2336771794 wait complete. 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 -- start start 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 Processor -- start 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604491ed780 con 0x560448e4a390 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5604490d7d10 con 0x560448e296b0 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5604491ed900 con 0x560448e214d0 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd13b3ee740 10 client.0 _ll_register_callbacks cb 0x560449104210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491fcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32790/0 (socket says 172.21.15.78:32790) 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 -- 172.21.15.78:0/3700460732 learned_addr learned my addr 172.21.15.78:0/3700460732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1319fc640 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1321fd640 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 -- 172.21.15.78:0/3700460732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 msgr2=0x5604491f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 -- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 msgr2=0x5604491f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 -- 172.21.15.78:0/3700460732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560448f2c7d0 con 0x560448e4a390 2024-04-25T14:01:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd1329fe640 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491fcd60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd12401b630 tx=0x7fd12401b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147e247a79219195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.334+0000 7fd115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd115ffb640 1 -- 172.21.15.78:0/3700460732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124009b90 con 0x560448e4a390 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd115ffb640 1 -- 172.21.15.78:0/3700460732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd1240117a0 con 0x560448e4a390 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1240113e0 con 0x560448e4a390 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd115ffb640 1 -- 172.21.15.78:0/3700460732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd12401b8f0 con 0x560448e4a390 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 10 client.16044 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560448e623f0 con 0x560448e4a390 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd115ffb640 1 -- 172.21.15.78:0/3700460732 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1240113e0 con 0x560448e4a390 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd115ffb640 1 -- 172.21.15.78:0/3700460732 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd12402d070 con 0x560448e4a390 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 10 client.16044 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 20 client.16044 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448e08f60 0x560449203040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 4 client.16044 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56044915e990 con 0x560448e08f60 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.335+0000 7fd1321fd640 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448e08f60 0x560449203040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.336+0000 7fd115ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.336+0000 7fd1321fd640 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448e08f60 0x560449203040 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4d07a1632c38e4 server_cookie=cd8963b337bece0f in_seq=0 out_seq=0 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd115ffb640 1 -- 172.21.15.78:0/3700460732 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56044915e990 con 0x560448e08f60 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd115ffb640 10 client.16044 handle_command_reply: tid=0 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 client.16044 shutdown 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 msgr2=0x5604491fcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491fcd60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd12401b630 tx=0x7fd12401b660 comp rx=0 tx=0).stop 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 shutdown_connections 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e4a390 0x5604491fcd60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448e08f60 0x560449203040 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3700460732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e214d0 0x5604491f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd115ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 >> 172.21.15.78:0/3700460732 conn(0x560448e3e4e0 msgr2=0x5604491ea180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 shutdown_connections 2024-04-25T14:01:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3700460732 wait complete. 2024-04-25T14:01:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.337+0000 7fd13b3ee740 20 client.16044 trim_cache size 0 max 16384 2024-04-25T14:01:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 Processor -- start 2024-04-25T14:01:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 -- start start 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e214d0 0x5604491f4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e296b0 0x5604491fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604491fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1afd80 con 0x560448e296b0 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1ad650 con 0x560448e214d0 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.354+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1ab4a0 con 0x560448e4a390 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e296b0 0x5604491fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e296b0 0x5604491fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32800/0 (socket says 172.21.15.78:32800) 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 -- 172.21.15.78:0/1252505557 learned_addr learned my addr 172.21.15.78:0/1252505557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1329fe640 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604491fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1321fd640 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e214d0 0x5604491f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 -- 172.21.15.78:0/1252505557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 msgr2=0x5604491fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604491fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 -- 172.21.15.78:0/1252505557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e214d0 msgr2=0x5604491f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e214d0 0x5604491f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 -- 172.21.15.78:0/1252505557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560448e220f0 con 0x560448e296b0 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1321fd640 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e214d0 0x5604491f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd1319fc640 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e296b0 0x5604491fcc80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0a75b0 tx=0x7fd11c0a74b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691b1feac32ded57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd117fff640 1 -- 172.21.15.78:0/1252505557 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0a7130 con 0x560448e296b0 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd117fff640 1 -- 172.21.15.78:0/1252505557 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11c005680 con 0x560448e296b0 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.355+0000 7fd117fff640 1 -- 172.21.15.78:0/1252505557 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0ce050 con 0x560448e296b0 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 -- 172.21.15.78:0/1252505557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e296b0 msgr2=0x5604491fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e296b0 0x5604491fcc80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0a75b0 tx=0x7fd11c0a74b0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 -- 172.21.15.78:0/1252505557 shutdown_connections 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604491fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e296b0 0x5604491fcc80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/1252505557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e214d0 0x5604491f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 -- 172.21.15.78:0/1252505557 >> 172.21.15.78:0/1252505557 conn(0x560448e3e4e0 msgr2=0x5604491f3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 -- 172.21.15.78:0/1252505557 shutdown_connections 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 -- 172.21.15.78:0/1252505557 wait complete. 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 -- start start 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.356+0000 7fd13b3ee740 1 Processor -- start 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e214d0 0x5604491fa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491fab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604492693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1abb10 con 0x560448e214d0 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1acf80 con 0x560448e296b0 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c1ac1e0 con 0x560448e4a390 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 10 client.0 _ll_register_callbacks cb 0x7fd12401f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e214d0 0x5604491fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e214d0 0x5604491fa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32806/0 (socket says 172.21.15.78:32806) 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 -- 172.21.15.78:0/3037449461 learned_addr learned my addr 172.21.15.78:0/3037449461 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1329fe640 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604492693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 -- 172.21.15.78:0/3037449461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 msgr2=0x5604492693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604492693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 -- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 msgr2=0x5604491fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1319fc640 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491fab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 -- 172.21.15.78:0/3037449461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604491114b0 con 0x560448e214d0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd1321fd640 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e214d0 0x5604491fa630 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd128000c00 tx=0x7fd128032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8073ef2e788360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd115ffb640 1 -- 172.21.15.78:0/3037449461 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128035040 con 0x560448e214d0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd115ffb640 1 -- 172.21.15.78:0/3037449461 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd128004310 con 0x560448e214d0 2024-04-25T14:01:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd11c005680 con 0x560448e214d0 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd115ffb640 1 -- 172.21.15.78:0/3037449461 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd128005040 con 0x560448e214d0 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 10 client.16050 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.357+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56044926a980 con 0x560448e214d0 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd115ffb640 1 -- 172.21.15.78:0/3037449461 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11c005680 con 0x560448e214d0 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd115ffb640 1 -- 172.21.15.78:0/3037449461 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd128042ce0 con 0x560448e214d0 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd13b3ee740 10 client.16050 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd13b3ee740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd13b3ee740 20 client.16050 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448dde3a0 0x56044926d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd13b3ee740 4 client.16050 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560449158800 con 0x560448dde3a0 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.358+0000 7fd1319fc640 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448dde3a0 0x56044926d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd115ffb640 10 client.16050 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd1319fc640 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448dde3a0 0x56044926d3a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=934776892452c137 server_cookie=6c4b21b9bffb6372 in_seq=0 out_seq=0 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd115ffb640 1 -- 172.21.15.78:0/3037449461 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560449158800 con 0x560448dde3a0 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd115ffb640 10 client.16050 handle_command_reply: tid=0 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 client.16050 shutdown 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e214d0 msgr2=0x5604491fa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e214d0 0x5604491fa630 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd128000c00 tx=0x7fd128032e90 comp rx=0 tx=0).stop 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 shutdown_connections 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560448dde3a0 0x56044926d3a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560448e4a390 0x5604492693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd115ffb640 0 client.16050 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560448e296b0 0x5604491fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 --2- 172.21.15.78:0/3037449461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560448e214d0 0x5604491fa630 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 >> 172.21.15.78:0/3037449461 conn(0x560448e3e4e0 msgr2=0x5604491e59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 shutdown_connections 2024-04-25T14:01:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 1 -- 172.21.15.78:0/3037449461 wait complete. 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.359+0000 7fd13b3ee740 20 client.16050 trim_cache size 0 max 16384 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "aa832fea-8459-47f9-a1fe-7925d5a38428": { 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "aa832fea-8459-47f9-a1fe-7925d5a38428", 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:01:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:01:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:01:27.367 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:01:27.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1180afaf0 msgr2=0x7fd1180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1180afaf0 0x7fd1180b2000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd124020000 tx=0x7fd12401d000 comp rx=0 tx=0).stop 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c104d20 msgr2=0x7fd12c074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c104d20 0x7fd12c074ef0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e9f0 tx=0x7fd11c00edf0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 shutdown_connections 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd12c109a20 0x7fd12c075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1180afaf0 0x7fd1180b2000 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c104d20 0x7fd12c074ef0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 --2- 172.21.15.78:0/473733998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd12c103ae0 0x7fd12c076850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 >> 172.21.15.78:0/473733998 conn(0x7fd12c0fecd0 msgr2=0x7fd12c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 shutdown_connections 2024-04-25T14:01:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:27.360+0000 7fd134461640 1 -- 172.21.15.78:0/473733998 wait complete. 2024-04-25T14:01:27.370 INFO:tasks.cephfs.filesystem:scrub status for tag:aa832fea-8459-47f9-a1fe-7925d5a38428 - {'path': '/', 'tag': 'aa832fea-8459-47f9-a1fe-7925d5a38428', 'options': 'recursive,force'} 2024-04-25T14:01:28.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:27 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:01:28.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:27 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:28.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:27 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:01:28.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:27 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:28.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:27 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:01:28.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:27 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:28.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:27 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:01:28.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:27 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:27 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:01:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:27 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:27 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:01:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:27 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:01:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 -- 172.21.15.78:0/2426803025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 msgr2=0x7f6dc4103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/2426803025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc4103720 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009900 tx=0x7f6db402a410 comp rx=0 tx=0).stop 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 -- 172.21.15.78:0/2426803025 shutdown_connections 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/2426803025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc4108960 0x7f6dc410ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/2426803025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dc4103c60 0x7f6dc4104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/2426803025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc4103720 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 -- 172.21.15.78:0/2426803025 >> 172.21.15.78:0/2426803025 conn(0x7f6dc40fdc10 msgr2=0x7f6dc41000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 -- 172.21.15.78:0/2426803025 shutdown_connections 2024-04-25T14:01:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.055+0000 7f6dc93c7640 1 -- 172.21.15.78:0/2426803025 wait complete. 2024-04-25T14:01:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 Processor -- start 2024-04-25T14:01:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 -- start start 2024-04-25T14:01:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc406ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dc4103c60 0x7f6dc419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc4108960 0x7f6dc41a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410e520 con 0x7f6dc4102a20 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410e3a0 con 0x7f6dc4108960 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc93c7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc410e6a0 con 0x7f6dc4103c60 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc406ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc406ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32830/0 (socket says 172.21.15.78:32830) 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc2ffd640 1 -- 172.21.15.78:0/1259681672 learned_addr learned my addr 172.21.15.78:0/1259681672 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dc4103c60 0x7f6dc419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.056+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc4108960 0x7f6dc41a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc2ffd640 1 -- 172.21.15.78:0/1259681672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dc4103c60 msgr2=0x7f6dc419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dc4103c60 0x7f6dc419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc2ffd640 1 -- 172.21.15.78:0/1259681672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc4108960 msgr2=0x7f6dc41a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc4108960 0x7f6dc41a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc2ffd640 1 -- 172.21.15.78:0/1259681672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc41a1d90 con 0x7f6dc4102a20 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc4108960 0x7f6dc41a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc406ae50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006010 tx=0x7f6db4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc0ff9640 1 -- 172.21.15.78:0/1259681672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db403f030 con 0x7f6dc4102a20 2024-04-25T14:01:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc0ff9640 1 -- 172.21.15.78:0/1259681672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db4003aa0 con 0x7f6dc4102a20 2024-04-25T14:01:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc0ff9640 1 -- 172.21.15.78:0/1259681672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4035070 con 0x7f6dc4102a20 2024-04-25T14:01:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc41a2020 con 0x7f6dc4102a20 2024-04-25T14:01:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.057+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc41a2510 con 0x7f6dc4102a20 2024-04-25T14:01:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.059+0000 7f6dc0ff9640 1 -- 172.21.15.78:0/1259681672 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6db4042030 con 0x7f6dc4102a20 2024-04-25T14:01:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 Processor -- start 2024-04-25T14:01:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 -- start start 2024-04-25T14:01:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4a69970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4a69ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4a70df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0b49cf460 con 0x55a0b46a84d0 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0b4989590 con 0x55a0b46c91b0 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dd0274740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a0b49cb420 con 0x55a0b46a02f0 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dba7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4a70df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dc0ff9640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6db00afae0 0x7f6db00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dba7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4a70df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32832/0 (socket says 172.21.15.78:32832) 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3045552515 learned_addr learned my addr 172.21.15.78:0/3045552515 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4a69970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6db00afae0 0x7f6db00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.060+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4a69ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dc0ff9640 1 -- 172.21.15.78:0/1259681672 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6db40ff1c0 con 0x7f6dc4102a20 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3045552515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 msgr2=0x55a0b4a69ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4a69ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3045552515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 msgr2=0x55a0b4a69970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4a69970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3045552515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b49960d0 con 0x55a0b46a84d0 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4a69970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4a70df0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00abe0 tx=0x7f6dac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c828099458980d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6db00afae0 0x7f6db00b1ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55a0b4a6ab40 tx=0x7f6dbc0094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6db97fa640 1 -- 172.21.15.78:0/3045552515 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac005220 con 0x55a0b46a84d0 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6db97fa640 1 -- 172.21.15.78:0/3045552515 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac0047a0 con 0x55a0b46a84d0 2024-04-25T14:01:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.061+0000 7f6db97fa640 1 -- 172.21.15.78:0/3045552515 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0186c0 con 0x55a0b46a84d0 2024-04-25T14:01:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 -- 172.21.15.78:0/3045552515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 msgr2=0x55a0b4a70df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4a70df0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00abe0 tx=0x7f6dac00f790 comp rx=0 tx=0).stop 2024-04-25T14:01:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 -- 172.21.15.78:0/3045552515 shutdown_connections 2024-04-25T14:01:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4a70df0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4a69ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3045552515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4a69970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 -- 172.21.15.78:0/3045552515 >> 172.21.15.78:0/3045552515 conn(0x55a0b46bd300 msgr2=0x55a0b4a65300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 -- 172.21.15.78:0/3045552515 shutdown_connections 2024-04-25T14:01:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 -- 172.21.15.78:0/3045552515 wait complete. 2024-04-25T14:01:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.062+0000 7f6dd0274740 1 -- start start 2024-04-25T14:01:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 1 Processor -- start 2024-04-25T14:01:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 0x55a0b4a73b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a7aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0b4a71840 con 0x55a0b46c91b0 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0b4956b60 con 0x55a0b46a02f0 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a0b4a719c0 con 0x55a0b46a84d0 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dd0274740 10 client.0 _ll_register_callbacks cb 0x55a0b497b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a7aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a7aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32838/0 (socket says 172.21.15.78:32838) 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 -- 172.21.15.78:0/2820990760 learned_addr learned my addr 172.21.15.78:0/2820990760 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 0x55a0b4a73b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 -- 172.21.15.78:0/2820990760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 msgr2=0x55a0b4a73b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 0x55a0b4a73b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 -- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 msgr2=0x55a0b4a735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc37fe640 1 -- 172.21.15.78:0/2820990760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b46e1210 con 0x55a0b46c91b0 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.063+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a7aa90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009e50 tx=0x7f6dbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acd954a32d40d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6daa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6daa7fc640 1 -- 172.21.15.78:0/2820990760 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc025030 con 0x55a0b46c91b0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6daa7fc640 1 -- 172.21.15.78:0/2820990760 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6dbc01b070 con 0x55a0b46c91b0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dac0047a0 con 0x55a0b46c91b0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6daa7fc640 1 -- 172.21.15.78:0/2820990760 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6dbc031050 con 0x55a0b46c91b0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 10 client.16062 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a0b47ad620 con 0x55a0b46c91b0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6daa7fc640 1 -- 172.21.15.78:0/2820990760 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac0047a0 con 0x55a0b46c91b0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6daa7fc640 1 -- 172.21.15.78:0/2820990760 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6dbc01e050 con 0x55a0b46c91b0 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 10 client.16062 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 20 client.16062 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b4687d80 0x55a0b4a81350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 4 client.16062 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.064+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0b49dd070 con 0x55a0b4687d80 2024-04-25T14:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.065+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b4687d80 0x55a0b4a81350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.065+0000 7f6daa7fc640 10 client.16062 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.065+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b4687d80 0x55a0b4a81350 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1397d8ddddde9fec server_cookie=d1f27f61c74b602c in_seq=0 out_seq=0 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6daa7fc640 1 -- 172.21.15.78:0/2820990760 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0b49dd070 con 0x55a0b4687d80 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6daa7fc640 10 client.16062 handle_command_reply: tid=0 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 client.16062 shutdown 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 msgr2=0x55a0b4a7aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a7aa90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009e50 tx=0x7f6dbc004a40 comp rx=0 tx=0).stop 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 shutdown_connections 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a7aa90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b4687d80 0x55a0b4a81350 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 0x55a0b4a73b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 --2- 172.21.15.78:0/2820990760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 >> 172.21.15.78:0/2820990760 conn(0x55a0b46bd300 msgr2=0x55a0b4a68210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6daa7fc640 0 client.16062 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 shutdown_connections 2024-04-25T14:01:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 1 -- 172.21.15.78:0/2820990760 wait complete. 2024-04-25T14:01:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.066+0000 7f6dd0274740 20 client.16062 trim_cache size 0 max 16384 2024-04-25T14:01:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.081+0000 7f6dd0274740 1 Processor -- start 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.081+0000 7f6dd0274740 1 -- start start 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a64cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 0x55a0b4a65200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a68100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dd0274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41adba0 con 0x55a0b46c91b0 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dd0274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41ab470 con 0x55a0b46a02f0 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dd0274740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a92c0 con 0x55a0b46a84d0 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a68100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32846/0 (socket says 172.21.15.78:32846) 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 -- 172.21.15.78:0/1830076304 learned_addr learned my addr 172.21.15.78:0/1830076304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 -- 172.21.15.78:0/1830076304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 msgr2=0x55a0b4a65200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 0x55a0b4a65200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 -- 172.21.15.78:0/1830076304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 msgr2=0x55a0b4a64cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a64cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a64cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 -- 172.21.15.78:0/1830076304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b4a711b0 con 0x55a0b46c91b0 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a68100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00a010 tx=0x7f6dbc009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8871c255a75644b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6db8ff9640 1 -- 172.21.15.78:0/1830076304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc03a070 con 0x55a0b46c91b0 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6db8ff9640 1 -- 172.21.15.78:0/1830076304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc009ca0 con 0x55a0b46c91b0 2024-04-25T14:01:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.082+0000 7f6db8ff9640 1 -- 172.21.15.78:0/1830076304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc025070 con 0x55a0b46c91b0 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 -- 172.21.15.78:0/1830076304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 msgr2=0x55a0b4a68100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a68100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00a010 tx=0x7f6dbc009a70 comp rx=0 tx=0).stop 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 -- 172.21.15.78:0/1830076304 shutdown_connections 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46c91b0 0x55a0b4a68100 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a84d0 0x55a0b4a65200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 --2- 172.21.15.78:0/1830076304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46a02f0 0x55a0b4a64cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 -- 172.21.15.78:0/1830076304 >> 172.21.15.78:0/1830076304 conn(0x55a0b46bd300 msgr2=0x55a0b4a698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 -- 172.21.15.78:0/1830076304 shutdown_connections 2024-04-25T14:01:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.083+0000 7f6dd0274740 1 -- 172.21.15.78:0/1830076304 wait complete. 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 -- start start 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 Processor -- start 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4ae4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4ae5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4aec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a9930 con 0x55a0b46a84d0 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41aada0 con 0x55a0b46c91b0 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41aa000 con 0x55a0b46a02f0 2024-04-25T14:01:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dd0274740 10 client.0 _ll_register_callbacks cb 0x7f6dbc0124f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4ae5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4ae5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32860/0 (socket says 172.21.15.78:32860) 2024-04-25T14:01:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3847826082 learned_addr learned my addr 172.21.15.78:0/3847826082 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4ae4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3847826082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 msgr2=0x55a0b4ae4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4ae4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dc37fe640 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4aec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 msgr2=0x55a0b4aec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4aec3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 -- 172.21.15.78:0/3847826082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b4a6e4a0 con 0x55a0b46a84d0 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6dba7fc640 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4ae5440 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00ab30 tx=0x7f6dac005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566488ee6b22ffa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.084+0000 7f6daa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6daa7fc640 1 -- 172.21.15.78:0/3847826082 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac029070 con 0x55a0b46a84d0 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6daa7fc640 1 -- 172.21.15.78:0/3847826082 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6dac005500 con 0x55a0b46a84d0 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dbc009ca0 con 0x55a0b46a84d0 2024-04-25T14:01:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6daa7fc640 1 -- 172.21.15.78:0/3847826082 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6dac034050 con 0x55a0b46a84d0 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 10 client.16068 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a0b4aeda50 con 0x55a0b46a84d0 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6daa7fc640 1 -- 172.21.15.78:0/3847826082 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc009ca0 con 0x55a0b46a84d0 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6daa7fc640 1 -- 172.21.15.78:0/3847826082 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6dac023670 con 0x55a0b46a84d0 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 10 client.16068 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 20 client.16068 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b465d1c0 0x55a0b4af0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 4 client.16068 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0b4a81940 con 0x55a0b465d1c0 2024-04-25T14:01:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.085+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b465d1c0 0x55a0b4af0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.086+0000 7f6daa7fc640 10 client.16068 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.086+0000 7f6dc2ffd640 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b465d1c0 0x55a0b4af0120 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd02adbba1d11ee server_cookie=b56b560bde11cac0 in_seq=0 out_seq=0 2024-04-25T14:01:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6daa7fc640 1 -- 172.21.15.78:0/3847826082 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0b4a81940 con 0x55a0b465d1c0 2024-04-25T14:01:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6daa7fc640 10 client.16068 handle_command_reply: tid=0 2024-04-25T14:01:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 client.16068 shutdown 2024-04-25T14:01:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 msgr2=0x55a0b4ae5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4ae5440 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00ab30 tx=0x7f6dac005200 comp rx=0 tx=0).stop 2024-04-25T14:01:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 shutdown_connections 2024-04-25T14:01:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0b46c91b0 0x55a0b4aec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0b46a84d0 0x55a0b4ae5440 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a0b465d1c0 0x55a0b4af0120 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 --2- 172.21.15.78:0/3847826082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0b46a02f0 0x55a0b4ae4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6daa7fc640 0 client.16068 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 >> 172.21.15.78:0/3847826082 conn(0x55a0b46bd300 msgr2=0x55a0b4a698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 shutdown_connections 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 1 -- 172.21.15.78:0/3847826082 wait complete. 2024-04-25T14:01:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.099+0000 7f6dd0274740 20 client.16068 trim_cache size 0 max 16384 2024-04-25T14:01:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:01:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:01:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:01:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:01:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6db00afae0 msgr2=0x7f6db00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6db00afae0 0x7f6db00b1ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55a0b4a6ab40 tx=0x7f6dbc0094a0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 msgr2=0x7f6dc406ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc406ae50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006010 tx=0x7f6db4031a50 comp rx=0 tx=0).stop 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 shutdown_connections 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc4108960 0x7f6dc41a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6dc4103c60 0x7f6dc419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6db00afae0 0x7f6db00b1ff0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 --2- 172.21.15.78:0/1259681672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6dc4102a20 0x7f6dc406ae50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 >> 172.21.15.78:0/1259681672 conn(0x7f6dc40fdc10 msgr2=0x7f6dc4104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 shutdown_connections 2024-04-25T14:01:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:29.100+0000 7f6dc93c7640 1 -- 172.21.15.78:0/1259681672 wait complete. 2024-04-25T14:01:29.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:28 smithi112 ceph-mon[32617]: pgmap v407: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:29.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:28 smithi129 ceph-mon[32569]: pgmap v407: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:28 smithi078 ceph-mon[29163]: pgmap v407: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:30 smithi129 ceph-mon[32569]: pgmap v408: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:30 smithi078 ceph-mon[29163]: pgmap v408: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:30.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:30 smithi112 ceph-mon[32617]: pgmap v408: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:31.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:31 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:01:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:31 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:01:31.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:31 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:01:32.572 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:32 smithi078 ceph-mon[29163]: pgmap v409: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:01:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:32 smithi129 ceph-mon[32569]: pgmap v409: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:01:32.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:32 smithi112 ceph-mon[32617]: pgmap v409: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:01:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:34 smithi129 ceph-mon[32569]: pgmap v410: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:01:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:34 smithi078 ceph-mon[29163]: pgmap v410: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:01:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:34 smithi112 ceph-mon[32617]: pgmap v410: 97 pgs: 97 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:01:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:35 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:01:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:35 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:01:35.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:35 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:01:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:36 smithi078 ceph-mon[29163]: pgmap v411: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:01:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:01:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:01:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:36 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:36 smithi112 ceph-mon[32617]: pgmap v411: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:01:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:01:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:01:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:36 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:36 smithi129 ceph-mon[32569]: pgmap v411: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:01:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:01:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:01:37.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:36 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:01:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:38 smithi129 ceph-mon[32569]: pgmap v412: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:01:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:38 smithi078 ceph-mon[29163]: pgmap v412: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:01:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:38 smithi112 ceph-mon[32617]: pgmap v412: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:01:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:40 smithi129 ceph-mon[32569]: pgmap v413: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:40 smithi078 ceph-mon[29163]: pgmap v413: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:40 smithi112 ceph-mon[32617]: pgmap v413: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 -- 172.21.15.78:0/3615723426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 msgr2=0x7f9760103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 --2- 172.21.15.78:0/3615723426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f9760103710 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f975c009900 tx=0x7f975c02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 -- 172.21.15.78:0/3615723426 shutdown_connections 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 --2- 172.21.15.78:0/3615723426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9760108950 0x7f976010ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 --2- 172.21.15.78:0/3615723426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760103c50 0x7f9760104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 --2- 172.21.15.78:0/3615723426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f9760103710 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 -- 172.21.15.78:0/3615723426 >> 172.21.15.78:0/3615723426 conn(0x7f97600fdc00 msgr2=0x7f97601000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 -- 172.21.15.78:0/3615723426 shutdown_connections 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 -- 172.21.15.78:0/3615723426 wait complete. 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 Processor -- start 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.202+0000 7f9767784640 1 -- start start 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9767784640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f97600757f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9767784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9760103c50 0x7f976019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9767784640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760108950 0x7f97601a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9767784640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f976010e4f0 con 0x7f9760102a10 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9767784640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f976010e370 con 0x7f9760108950 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9767784640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f976010e670 con 0x7f9760103c50 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f97600757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f97600757f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56616/0 (socket says 172.21.15.78:56616) 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 -- 172.21.15.78:0/2897731827 learned_addr learned my addr 172.21.15.78:0/2897731827 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9764d1f640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9760103c50 0x7f976019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765d21640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760108950 0x7f97601a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 -- 172.21.15.78:0/2897731827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9760103c50 msgr2=0x7f976019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9760103c50 0x7f976019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 -- 172.21.15.78:0/2897731827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760108950 msgr2=0x7f97601a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760108950 0x7f97601a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 -- 172.21.15.78:0/2897731827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97601a1d90 con 0x7f9760102a10 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9765520640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f97600757f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f975c006010 tx=0x7f975c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f97567fc640 1 -- 172.21.15.78:0/2897731827 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c03f030 con 0x7f9760102a10 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f97567fc640 1 -- 172.21.15.78:0/2897731827 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c003aa0 con 0x7f9760102a10 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.203+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97601a2020 con 0x7f9760102a10 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.204+0000 7f97567fc640 1 -- 172.21.15.78:0/2897731827 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c035070 con 0x7f9760102a10 2024-04-25T14:01:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.204+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601a2510 con 0x7f9760102a10 2024-04-25T14:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 Processor -- start 2024-04-25T14:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 -- start start 2024-04-25T14:01:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc96970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc9ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f61fbfc460 con 0x55f61f8cd2f0 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f61fbb6590 con 0x55f61f8d54d0 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f976e638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f61fbf8420 con 0x55f61f8f61b0 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f97567fc640 1 -- 172.21.15.78:0/2897731827 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f975c042030 con 0x7f9760102a10 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765d21640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc96970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc96ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56622/0 (socket says 172.21.15.78:56622) 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765d21640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc96970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:60194/0 (socket says 172.21.15.78:60194) 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765520640 1 -- 172.21.15.78:0/66983793 learned_addr learned my addr 172.21.15.78:0/66983793 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9764d1f640 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc9ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765520640 1 -- 172.21.15.78:0/66983793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 msgr2=0x55f61fc96970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765520640 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc96970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765520640 1 -- 172.21.15.78:0/66983793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 msgr2=0x55f61fc9ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.207+0000 7f9765520640 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc9ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f9765520640 1 -- 172.21.15.78:0/66983793 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61fbc30d0 con 0x55f61f8cd2f0 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f9764d1f640 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f9765d21640 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc96970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f9765520640 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc96ef0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f975c107480 tx=0x7f975c102f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff663803226a8ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f97567fc640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f974c0afae0 0x7f974c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f974b7fe640 1 -- 172.21.15.78:0/66983793 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0c3d40 con 0x55f61f8cd2f0 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f9765d21640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f974c0afae0 0x7f974c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f974b7fe640 1 -- 172.21.15.78:0/66983793 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c10c030 con 0x55f61f8cd2f0 2024-04-25T14:01:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f974b7fe640 1 -- 172.21.15.78:0/66983793 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0c6e60 con 0x55f61f8cd2f0 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.208+0000 7f97567fc640 1 -- 172.21.15.78:0/2897731827 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f975c0ff220 con 0x7f9760102a10 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 -- 172.21.15.78:0/66983793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 msgr2=0x55f61fc96ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc96ef0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f975c107480 tx=0x7f975c102f90 comp rx=0 tx=0).stop 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f9765d21640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f974c0afae0 0x7f974c0b1ff0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55f61fc97b40 tx=0x7f9758009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 -- 172.21.15.78:0/66983793 shutdown_connections 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc96ef0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 --2- 172.21.15.78:0/66983793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc96970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 -- 172.21.15.78:0/66983793 >> 172.21.15.78:0/66983793 conn(0x55f61f8ea300 msgr2=0x55f61fc92300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 -- 172.21.15.78:0/66983793 shutdown_connections 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 -- 172.21.15.78:0/66983793 wait complete. 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 -- start start 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.209+0000 7f976e638740 1 Processor -- start 2024-04-25T14:01:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f976e638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fca0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f976e638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fca08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f976e638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fca77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f976e638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f61fc9e600 con 0x55f61f8cd2f0 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f976e638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f61fb83b60 con 0x55f61f8d54d0 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f976e638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f61fc9e780 con 0x55f61f8f61b0 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f976e638740 10 client.0 _ll_register_callbacks cb 0x55f61fba82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fca0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fca0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56632/0 (socket says 172.21.15.78:56632) 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 -- 172.21.15.78:0/2009093771 learned_addr learned my addr 172.21.15.78:0/2009093771 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9764d1f640 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fca08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765d21640 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fca77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 -- 172.21.15.78:0/2009093771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 msgr2=0x55f61fca77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fca77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 -- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 msgr2=0x55f61fca08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fca08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.210+0000 7f9765520640 1 -- 172.21.15.78:0/2009093771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61f90e210 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f9765520640 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fca0370 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f975c0c7200 tx=0x7f975c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3f8e2eba23f5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f9748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f9748ff9640 1 -- 172.21.15.78:0/2009093771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c049020 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f9748ff9640 1 -- 172.21.15.78:0/2009093771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f975c03b460 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c10c030 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f9748ff9640 1 -- 172.21.15.78:0/2009093771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f975c0c7040 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 10 client.16080 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f61f9da620 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f9748ff9640 1 -- 172.21.15.78:0/2009093771 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c10c030 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f9748ff9640 1 -- 172.21.15.78:0/2009093771 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f975c003c50 con 0x55f61f8cd2f0 2024-04-25T14:01:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 10 client.16080 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 20 client.16080 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f8b4d80 0x55f61fcadd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 4 client.16080 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.211+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61fc0a070 con 0x55f61f8b4d80 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.212+0000 7f9764d1f640 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f8b4d80 0x55f61fcadd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.212+0000 7f9748ff9640 10 client.16080 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.212+0000 7f9764d1f640 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f8b4d80 0x55f61fcadd20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21111f49522019b server_cookie=b45388885f06171e in_seq=0 out_seq=0 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f9748ff9640 1 -- 172.21.15.78:0/2009093771 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f61fc0a070 con 0x55f61f8b4d80 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f9748ff9640 10 client.16080 handle_command_reply: tid=0 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 client.16080 shutdown 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 msgr2=0x55f61fca0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fca0370 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f975c0c7200 tx=0x7f975c009260 comp rx=0 tx=0).stop 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 shutdown_connections 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f8b4d80 0x55f61fcadd20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fca77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fca08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f9748ff9640 0 client.16080 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 --2- 172.21.15.78:0/2009093771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fca0370 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 >> 172.21.15.78:0/2009093771 conn(0x55f61f8ea300 msgr2=0x55f61fc95210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 shutdown_connections 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 1 -- 172.21.15.78:0/2009093771 wait complete. 2024-04-25T14:01:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.213+0000 7f976e638740 20 client.16080 trim_cache size 0 max 16384 2024-04-25T14:01:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 Processor -- start 2024-04-25T14:01:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 -- start start 2024-04-25T14:01:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc95100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc95640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97601adbd0 con 0x55f61f8cd2f0 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97601ab4a0 con 0x55f61f8d54d0 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f976e638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97601a92f0 con 0x55f61f8f61b0 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc95100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc95100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56646/0 (socket says 172.21.15.78:56646) 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 -- 172.21.15.78:0/1337576025 learned_addr learned my addr 172.21.15.78:0/1337576025 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9764d1f640 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc95640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765d21640 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 -- 172.21.15.78:0/1337576025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 msgr2=0x55f61fc91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 -- 172.21.15.78:0/1337576025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 msgr2=0x55f61fc95640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc95640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765520640 1 -- 172.21.15.78:0/1337576025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61fc964f0 con 0x55f61f8cd2f0 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9764d1f640 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc95640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.228+0000 7f9765d21640 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f9765520640 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc95100 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f975c0c7680 tx=0x7f975c107200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e6275298a84931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f974affd640 1 -- 172.21.15.78:0/1337576025 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c049050 con 0x55f61f8cd2f0 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f974affd640 1 -- 172.21.15.78:0/1337576025 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c10c030 con 0x55f61f8cd2f0 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f974affd640 1 -- 172.21.15.78:0/1337576025 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0c7230 con 0x55f61f8cd2f0 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 -- 172.21.15.78:0/1337576025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 msgr2=0x55f61fc95100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc95100 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f975c0c7680 tx=0x7f975c107200 comp rx=0 tx=0).stop 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 -- 172.21.15.78:0/1337576025 shutdown_connections 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8f61b0 0x55f61fc91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8d54d0 0x55f61fc95640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 --2- 172.21.15.78:0/1337576025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8cd2f0 0x55f61fc95100 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 -- 172.21.15.78:0/1337576025 >> 172.21.15.78:0/1337576025 conn(0x55f61f8ea300 msgr2=0x55f61fc9aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 -- 172.21.15.78:0/1337576025 shutdown_connections 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.229+0000 7f976e638740 1 -- 172.21.15.78:0/1337576025 wait complete. 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 -- start start 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 Processor -- start 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8cd2f0 0x55f61fca3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8d54d0 0x55f61fca44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8f61b0 0x55f61fca53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97601a9960 con 0x55f61f8d54d0 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97601aadd0 con 0x55f61f8f61b0 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97601aa030 con 0x55f61f8cd2f0 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f976e638740 10 client.0 _ll_register_callbacks cb 0x7f97580094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f9764d1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8d54d0 0x55f61fca44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f9764d1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8d54d0 0x55f61fca44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56656/0 (socket says 172.21.15.78:56656) 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f9764d1f640 1 -- 172.21.15.78:0/2236044949 learned_addr learned my addr 172.21.15.78:0/2236044949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f9765520640 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8cd2f0 0x55f61fca3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.230+0000 7f9765d21640 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8f61b0 0x55f61fca53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9764d1f640 1 -- 172.21.15.78:0/2236044949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8cd2f0 msgr2=0x55f61fca3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9764d1f640 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8cd2f0 0x55f61fca3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9764d1f640 1 -- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8f61b0 msgr2=0x55f61fca53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9764d1f640 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8f61b0 0x55f61fca53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9764d1f640 1 -- 172.21.15.78:0/2236044949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61fca59b0 con 0x55f61f8d54d0 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9764d1f640 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8d54d0 0x55f61fca44c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f975000b620 tx=0x7f975000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f310e0c5e66a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9748ff9640 1 -- 172.21.15.78:0/2236044949 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750021030 con 0x55f61f8d54d0 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9748ff9640 1 -- 172.21.15.78:0/2236044949 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9750003dd0 con 0x55f61f8d54d0 2024-04-25T14:01:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c10c030 con 0x55f61f8d54d0 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9748ff9640 1 -- 172.21.15.78:0/2236044949 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9750005020 con 0x55f61f8d54d0 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 10 client.16086 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f61fca6c20 con 0x55f61f8d54d0 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9748ff9640 1 -- 172.21.15.78:0/2236044949 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c10c030 con 0x55f61f8d54d0 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f9748ff9640 1 -- 172.21.15.78:0/2236044949 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f97500047a0 con 0x55f61f8d54d0 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 10 client.16086 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 20 client.16086 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f88a1c0 0x55f61fd18390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 4 client.16086 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.231+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f61fc03ee0 con 0x55f61f88a1c0 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.232+0000 7f9765520640 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f88a1c0 0x55f61fd18390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.232+0000 7f9748ff9640 10 client.16086 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.232+0000 7f9765520640 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f88a1c0 0x55f61fd18390 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8a973b9775d569 server_cookie=ddfeb1de80a7ba04 in_seq=0 out_seq=0 2024-04-25T14:01:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f9748ff9640 1 -- 172.21.15.78:0/2236044949 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f61fc03ee0 con 0x55f61f88a1c0 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f9748ff9640 10 client.16086 handle_command_reply: tid=0 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 client.16086 shutdown 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8d54d0 msgr2=0x55f61fca44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8d54d0 0x55f61fca44c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f975000b620 tx=0x7f975000be90 comp rx=0 tx=0).stop 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 shutdown_connections 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f61f8f61b0 0x55f61fca53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f61f8d54d0 0x55f61fca44c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f61f88a1c0 0x55f61fd18390 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 --2- 172.21.15.78:0/2236044949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f61f8cd2f0 0x55f61fca3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f9748ff9640 0 client.16086 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.245+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 >> 172.21.15.78:0/2236044949 conn(0x55f61f8ea300 msgr2=0x55f61fc95100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 shutdown_connections 2024-04-25T14:01:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f976e638740 1 -- 172.21.15.78:0/2236044949 wait complete. 2024-04-25T14:01:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f976e638740 20 client.16086 trim_cache size 0 max 16384 2024-04-25T14:01:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:01:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:01:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:01:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:01:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f974c0afae0 msgr2=0x7f974c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f974c0afae0 0x7f974c0b1ff0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55f61fc97b40 tx=0x7f9758009620 comp rx=0 tx=0).stop 2024-04-25T14:01:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 msgr2=0x7f97600757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f97600757f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f975c006010 tx=0x7f975c031a50 comp rx=0 tx=0).stop 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 shutdown_connections 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9760108950 0x7f97601a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9760103c50 0x7f976019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f974c0afae0 0x7f974c0b1ff0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 --2- 172.21.15.78:0/2897731827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9760102a10 0x7f97600757f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 >> 172.21.15.78:0/2897731827 conn(0x7f97600fdc00 msgr2=0x7f9760104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 shutdown_connections 2024-04-25T14:01:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:42.246+0000 7f9767784640 1 -- 172.21.15.78:0/2897731827 wait complete. 2024-04-25T14:01:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:42 smithi078 ceph-mon[29163]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:42.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:42 smithi129 ceph-mon[32569]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:42.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:42 smithi112 ceph-mon[32617]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:01:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:44 smithi129 ceph-mon[32569]: pgmap v415: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T14:01:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:44 smithi078 ceph-mon[29163]: pgmap v415: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T14:01:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:44 smithi112 ceph-mon[32617]: pgmap v415: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T14:01:44.871 INFO:tasks.workunit.client.0.smithi078.stdout:1/600: open fd = 3 2024-04-25T14:01:44.871 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.871 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.871 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.871 INFO:tasks.workunit.client.0.smithi078.stdout:2/600: open fd = 3 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:3/600: open fd = 3 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:4/600: open fd = 3 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:5/600: open fd = 3 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.872 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:6/600: open fd = 3 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:7/600: open fd = 3 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:8/600: open fd = 3 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.873 INFO:tasks.workunit.client.0.smithi078.stdout:9/600: open fd = 3 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:10/600: open fd = 3 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:11/600: open fd = 3 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:12/600: open fd = 3 2024-04-25T14:01:44.874 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:13/600: open fd = 3 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:14/600: open fd = 3 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.875 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:15/600: open fd = 3 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:16/600: open fd = 3 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.876 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.877 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.877 INFO:tasks.workunit.client.0.smithi078.stdout:17/600: open fd = 3 2024-04-25T14:01:44.877 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.877 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.877 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.877 INFO:tasks.workunit.client.0.smithi078.stdout:18/600: open fd = 3 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:19/600: open fd = 3 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:20/600: open fd = 3 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:21/600: open fd = 3 2024-04-25T14:01:44.878 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:22/600: open fd = 3 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:23/600: open fd = 3 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:24/600: open fd = 3 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.879 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:25/600: open fd = 3 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:26/600: open fd = 3 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:27/600: open fd = 3 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:28/600: open fd = 3 2024-04-25T14:01:44.880 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.881 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.881 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.881 INFO:tasks.workunit.client.0.smithi078.stdout:29/600: open fd = 3 2024-04-25T14:01:44.881 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.881 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.881 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.881 INFO:tasks.workunit.client.0.smithi078.stdout:30/600: open fd = 3 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:31/600: open fd = 3 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:32/600: open fd = 3 2024-04-25T14:01:44.882 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:33/600: open fd = 3 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:34/600: open fd = 3 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:35/600: open fd = 3 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.883 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:36/600: open fd = 3 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:37/600: open fd = 3 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:38/600: open fd = 3 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:39/600: open fd = 3 2024-04-25T14:01:44.884 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:40/600: open fd = 3 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:41/600: open fd = 3 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:42/600: open fd = 3 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.885 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:43/600: open fd = 3 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:44/600: open fd = 3 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:45/600: open fd = 3 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.886 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:46/600: open fd = 3 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:47/600: open fd = 3 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:48/600: open fd = 3 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.887 INFO:tasks.workunit.client.0.smithi078.stdout:49/600: open fd = 3 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:50/600: open fd = 3 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:51/600: open fd = 3 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:52/600: open fd = 3 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.888 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:53/600: open fd = 3 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:54/600: open fd = 3 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:55/600: open fd = 3 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.889 INFO:tasks.workunit.client.0.smithi078.stdout:56/600: open fd = 3 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:57/600: open fd = 3 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:58/600: open fd = 3 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:59/600: open fd = 3 2024-04-25T14:01:44.890 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:60/600: open fd = 3 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:61/600: open fd = 3 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.891 INFO:tasks.workunit.client.0.smithi078.stdout:62/600: open fd = 3 2024-04-25T14:01:44.892 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.892 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.892 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:44.892 INFO:tasks.workunit.client.0.smithi078.stdout:63/600: open fd = 3 2024-04-25T14:01:44.892 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.892 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:01:44.892 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:01:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:46 smithi129 ceph-mon[32569]: pgmap v416: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T14:01:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:46 smithi078 ceph-mon[29163]: pgmap v416: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T14:01:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:46 smithi112 ceph-mon[32617]: pgmap v416: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T14:01:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:48 smithi129 ceph-mon[32569]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:48 smithi078 ceph-mon[29163]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:48 smithi112 ceph-mon[32617]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:50 smithi129 ceph-mon[32569]: pgmap v418: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:01:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:50 smithi078 ceph-mon[29163]: pgmap v418: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:01:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:50 smithi112 ceph-mon[32617]: pgmap v418: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:01:52.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:52 smithi112 ceph-mon[32617]: pgmap v419: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:01:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:52 smithi129 ceph-mon[32569]: pgmap v419: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:01:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:52 smithi078 ceph-mon[29163]: pgmap v419: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:01:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:54 smithi129 ceph-mon[32569]: pgmap v420: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:01:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:54 smithi078 ceph-mon[29163]: pgmap v420: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:01:54.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:54 smithi112 ceph-mon[32617]: pgmap v420: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:01:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 -- 172.21.15.78:0/159233189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0101790 msgr2=0x7f1fc0101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 --2- 172.21.15.78:0/159233189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0101790 0x7f1fc0101c00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009900 tx=0x7f1fb402a410 comp rx=0 tx=0).stop 2024-04-25T14:01:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 -- 172.21.15.78:0/159233189 shutdown_connections 2024-04-25T14:01:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 --2- 172.21.15.78:0/159233189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc0102290 0x7f1fc010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 --2- 172.21.15.78:0/159233189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0101790 0x7f1fc0101c00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 --2- 172.21.15.78:0/159233189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc01050e0 0x7f1fc01054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 -- 172.21.15.78:0/159233189 >> 172.21.15.78:0/159233189 conn(0x7f1fc00fd390 msgr2=0x7f1fc00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 -- 172.21.15.78:0/159233189 shutdown_connections 2024-04-25T14:01:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 -- 172.21.15.78:0/159233189 wait complete. 2024-04-25T14:01:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.350+0000 7f1fc5346640 1 Processor -- start 2024-04-25T14:01:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fc5346640 1 -- start start 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fc5346640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc0101790 0x7f1fc019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fc5346640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0102290 0x7f1fc019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fc5346640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc01050e0 0x7f1fc01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fc5346640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc010f670 con 0x7f1fc0102290 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fc5346640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc010f4f0 con 0x7f1fc0101790 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fc5346640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc010f7f0 con 0x7f1fc01050e0 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0102290 0x7f1fc019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0102290 0x7f1fc019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58952/0 (socket says 172.21.15.78:58952) 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 -- 172.21.15.78:0/1121919342 learned_addr learned my addr 172.21.15.78:0/1121919342 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc0101790 0x7f1fc019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc01050e0 0x7f1fc01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 -- 172.21.15.78:0/1121919342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc01050e0 msgr2=0x7f1fc01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc01050e0 0x7f1fc01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 -- 172.21.15.78:0/1121919342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc0101790 msgr2=0x7f1fc019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc0101790 0x7f1fc019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 -- 172.21.15.78:0/1121919342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc01a1d60 con 0x7f1fc0102290 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc01050e0 0x7f1fc01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.351+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0102290 0x7f1fc019a760 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4006010 tx=0x7f1fb4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.352+0000 7f1fa7fff640 1 -- 172.21.15.78:0/1121919342 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb403f030 con 0x7f1fc0102290 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.352+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc01a1ff0 con 0x7f1fc0102290 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.352+0000 7f1fa7fff640 1 -- 172.21.15.78:0/1121919342 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb4003cb0 con 0x7f1fc0102290 2024-04-25T14:01:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.352+0000 7f1fa7fff640 1 -- 172.21.15.78:0/1121919342 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4035070 con 0x7f1fc0102290 2024-04-25T14:01:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.352+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01a2400 con 0x7f1fc0102290 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.354+0000 7f1fa7fff640 1 -- 172.21.15.78:0/1121919342 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1fb4042030 con 0x7f1fc0102290 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.354+0000 7f1fcc1f7740 1 Processor -- start 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.354+0000 7f1fcc1f7740 1 -- start start 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12265970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11e9c2f0 0x555b12265ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ea44d0 0x555b1226cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555b121cb460 con 0x555b11ec51b0 2024-04-25T14:01:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555b12185590 con 0x555b11ea44d0 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555b121c7420 con 0x555b11e9c2f0 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12265970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fa7fff640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1fa80afae0 0x7f1fa80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12265970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58966/0 (socket says 172.21.15.78:58966) 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/914510334 learned_addr learned my addr 172.21.15.78:0/914510334 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11e9c2f0 0x555b12265ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ea44d0 0x555b1226cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1fa80afae0 0x7f1fa80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fa7fff640 1 -- 172.21.15.78:0/1121919342 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1fb40ff2c0 con 0x7f1fc0102290 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/914510334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11e9c2f0 msgr2=0x555b12265ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11e9c2f0 0x555b12265ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/914510334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ea44d0 msgr2=0x555b1226cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ea44d0 0x555b1226cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/914510334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b121920d0 con 0x555b11ec51b0 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ea44d0 0x555b1226cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12265970 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8014a90 tx=0x7f1fb8014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e74acbe8230b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.355+0000 7f1fa57fa640 1 -- 172.21.15.78:0/914510334 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8014530 con 0x555b11ec51b0 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fa57fa640 1 -- 172.21.15.78:0/914510334 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb8023070 con 0x555b11ec51b0 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fa57fa640 1 -- 172.21.15.78:0/914510334 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800cd70 con 0x555b11ec51b0 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/914510334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 msgr2=0x555b12265970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1fa80afae0 0x7f1fa80b1ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8022000 tx=0x7f1fb8020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12265970 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8014a90 tx=0x7f1fb8014f50 comp rx=0 tx=0).stop 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/914510334 shutdown_connections 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ea44d0 0x555b1226cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11e9c2f0 0x555b12265ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/914510334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12265970 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/914510334 >> 172.21.15.78:0/914510334 conn(0x555b11eb9300 msgr2=0x555b12261300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/914510334 shutdown_connections 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.356+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/914510334 wait complete. 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 -- start start 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 Processor -- start 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11e9c2f0 0x555b1226f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b1226fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ec51b0 0x555b12276b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555b1226d840 con 0x555b11e9c2f0 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555b12152b60 con 0x555b11ec51b0 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555b1226d9c0 con 0x555b11ea44d0 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fcc1f7740 10 client.0 _ll_register_callbacks cb 0x555b121772b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fbeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11e9c2f0 0x555b1226f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fbeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11e9c2f0 0x555b1226f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58968/0 (socket says 172.21.15.78:58968) 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fbeffd640 1 -- 172.21.15.78:0/3854155137 learned_addr learned my addr 172.21.15.78:0/3854155137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b1226fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.357+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ec51b0 0x555b12276b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fbeffd640 1 -- 172.21.15.78:0/3854155137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 msgr2=0x555b1226fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b1226fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fbeffd640 1 -- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ec51b0 msgr2=0x555b12276b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ec51b0 0x555b12276b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fbeffd640 1 -- 172.21.15.78:0/3854155137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b11edd210 con 0x555b11e9c2f0 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11e9c2f0 0x555b1226f730 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0010820 tx=0x7f1fb0010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f7665d06e68eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/3854155137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000fc50 con 0x555b11e9c2f0 2024-04-25T14:01:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/3854155137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1fb000fdf0 con 0x555b11e9c2f0 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb8023070 con 0x555b11e9c2f0 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/3854155137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1fb0004030 con 0x555b11e9c2f0 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 10 client.16098 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555b11fa9620 con 0x555b11e9c2f0 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/3854155137 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb8023070 con 0x555b11e9c2f0 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/3854155137 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1fb0019400 con 0x555b11e9c2f0 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 10 client.16098 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 20 client.16098 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e83d80 0x555b1227d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 4 client.16098 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.358+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b121d9070 con 0x555b11e83d80 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.359+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e83d80 0x555b1227d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.359+0000 7f1fa2ff5640 10 client.16098 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.359+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e83d80 0x555b1227d440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ee3560a82fbca server_cookie=7d50fe48ab8aa4e4 in_seq=0 out_seq=0 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/3854155137 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b121d9070 con 0x555b11e83d80 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fa2ff5640 10 client.16098 handle_command_reply: tid=0 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 client.16098 shutdown 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11e9c2f0 msgr2=0x555b1226f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11e9c2f0 0x555b1226f730 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0010820 tx=0x7f1fb0010ce0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 shutdown_connections 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11ec51b0 0x555b12276b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e83d80 0x555b1227d440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b1226fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/3854155137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11e9c2f0 0x555b1226f730 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fa2ff5640 0 client.16098 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 >> 172.21.15.78:0/3854155137 conn(0x555b11eb9300 msgr2=0x555b12264210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 shutdown_connections 2024-04-25T14:01:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/3854155137 wait complete. 2024-04-25T14:01:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.360+0000 7f1fcc1f7740 20 client.16098 trim_cache size 0 max 16384 2024-04-25T14:01:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 Processor -- start 2024-04-25T14:01:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 -- start start 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b12282bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b12283130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12276a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01adb40 con 0x555b11ec51b0 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01ab410 con 0x555b11e9c2f0 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01a9260 con 0x555b11ea44d0 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12276a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12276a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58976/0 (socket says 172.21.15.78:58976) 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/1223503503 learned_addr learned my addr 172.21.15.78:0/1223503503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b12282bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b12283130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/1223503503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 msgr2=0x555b12283130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b12283130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/1223503503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 msgr2=0x555b12282bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b12282bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.375+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/1223503503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b12276fc0 con 0x555b11ec51b0 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12276a80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005fa0 tx=0x7f1fb8005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1969f30a92b7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fa4ff9640 1 -- 172.21.15.78:0/1223503503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8014050 con 0x555b11ec51b0 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fa4ff9640 1 -- 172.21.15.78:0/1223503503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb8023070 con 0x555b11ec51b0 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fa4ff9640 1 -- 172.21.15.78:0/1223503503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800cbd0 con 0x555b11ec51b0 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/1223503503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 msgr2=0x555b12276a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12276a80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005fa0 tx=0x7f1fb8005f30 comp rx=0 tx=0).stop 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/1223503503 shutdown_connections 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b12276a80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b12283130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.376+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/1223503503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b12282bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/1223503503 >> 172.21.15.78:0/1223503503 conn(0x555b11eb9300 msgr2=0x555b1226dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/1223503503 shutdown_connections 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/1223503503 wait complete. 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 -- start start 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 Processor -- start 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b122df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b122e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b122e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01a98d0 con 0x555b11ec51b0 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01aad40 con 0x555b11e9c2f0 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01a9fa0 con 0x555b11ea44d0 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fcc1f7740 10 client.0 _ll_register_callbacks cb 0x7f1fb000a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b122e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b122e4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58978/0 (socket says 172.21.15.78:58978) 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/2270824104 learned_addr learned my addr 172.21.15.78:0/2270824104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b122df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.377+0000 7f1fbe7fc640 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b122e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/2270824104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 msgr2=0x555b122e1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b122e1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 msgr2=0x555b122df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b122df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fbf7fe640 1 -- 172.21.15.78:0/2270824104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b122e4850 con 0x555b11ec51b0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fbf7fe640 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b122e4210 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c750 tx=0x7f1fb8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c7f3327d407246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/2270824104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800c360 con 0x555b11ec51b0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/2270824104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1fb800c950 con 0x555b11ec51b0 2024-04-25T14:01:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb8023070 con 0x555b11ec51b0 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/2270824104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1fb800c500 con 0x555b11ec51b0 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fcc1f7740 10 client.16104 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555b122e5ae0 con 0x555b11ec51b0 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/2270824104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb8023070 con 0x555b11ec51b0 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.378+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/2270824104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1fb800f040 con 0x555b11ec51b0 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fcc1f7740 10 client.16104 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fcc1f7740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fcc1f7740 20 client.16104 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e591c0 0x555b122e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fcc1f7740 4 client.16104 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b121d2ee0 con 0x555b11e591c0 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e591c0 0x555b122e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fa2ff5640 10 client.16104 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.379+0000 7f1fbeffd640 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e591c0 0x555b122e82c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1b49053442752 server_cookie=db2b59b709740533 in_seq=0 out_seq=0 2024-04-25T14:01:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fa2ff5640 1 -- 172.21.15.78:0/2270824104 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b121d2ee0 con 0x555b11e591c0 2024-04-25T14:01:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fa2ff5640 10 client.16104 handle_command_reply: tid=0 2024-04-25T14:01:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 client.16104 shutdown 2024-04-25T14:01:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 msgr2=0x555b122e4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b122e4210 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c750 tx=0x7f1fb8009520 comp rx=0 tx=0).stop 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 shutdown_connections 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b11ec51b0 0x555b122e4210 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555b11e591c0 0x555b122e82c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b11ea44d0 0x555b122e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 --2- 172.21.15.78:0/2270824104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555b11e9c2f0 0x555b122df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fa2ff5640 0 client.16104 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 >> 172.21.15.78:0/2270824104 conn(0x555b11eb9300 msgr2=0x555b12282da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 shutdown_connections 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 1 -- 172.21.15.78:0/2270824104 wait complete. 2024-04-25T14:01:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.393+0000 7f1fcc1f7740 20 client.16104 trim_cache size 0 max 16384 2024-04-25T14:01:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:01:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:01:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:01:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1fa80afae0 msgr2=0x7f1fa80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1fa80afae0 0x7f1fa80b1ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8022000 tx=0x7f1fb8020000 comp rx=0 tx=0).stop 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0102290 msgr2=0x7f1fc019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0102290 0x7f1fc019a760 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4006010 tx=0x7f1fb4004340 comp rx=0 tx=0).stop 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 shutdown_connections 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc01050e0 0x7f1fc01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1fa80afae0 0x7f1fa80b1ff0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fc0102290 0x7f1fc019a760 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 --2- 172.21.15.78:0/1121919342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fc0101790 0x7f1fc019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 >> 172.21.15.78:0/1121919342 conn(0x7f1fc00fd390 msgr2=0x7f1fc0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 shutdown_connections 2024-04-25T14:01:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:01:55.394+0000 7f1fc5346640 1 -- 172.21.15.78:0/1121919342 wait complete. 2024-04-25T14:01:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:56 smithi129 ceph-mon[32569]: pgmap v421: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:01:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:56 smithi078 ceph-mon[29163]: pgmap v421: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:01:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:56 smithi112 ceph-mon[32617]: pgmap v421: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:01:57.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:01:57.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:01:57.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 -- 172.21.15.78:0/1780848750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864109a30 msgr2=0x7f486410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 --2- 172.21.15.78:0/1780848750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864109a30 0x7f486410be50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f485c009ca0 tx=0x7f485c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 -- 172.21.15.78:0/1780848750 shutdown_connections 2024-04-25T14:01:57.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 --2- 172.21.15.78:0/1780848750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864109a30 0x7f486410be50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 --2- 172.21.15.78:0/1780848750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4864104d30 0x7f4864105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 --2- 172.21.15.78:0/1780848750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864103af0 0x7f48641047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 -- 172.21.15.78:0/1780848750 >> 172.21.15.78:0/1780848750 conn(0x7f48640fece0 msgr2=0x7f4864101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 -- 172.21.15.78:0/1780848750 shutdown_connections 2024-04-25T14:01:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.501+0000 7f4868d3f640 1 -- 172.21.15.78:0/1780848750 wait complete. 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 Processor -- start 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 -- start start 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864103af0 0x7f486419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4864104d30 0x7f486419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a30 0x7f48641a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f486410f620 con 0x7f4864103af0 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f486410f4a0 con 0x7f4864104d30 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4868d3f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f486410f7a0 con 0x7f4864109a30 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864103af0 0x7f486419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864103af0 0x7f486419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37412/0 (socket says 172.21.15.78:37412) 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 -- 172.21.15.78:0/3739548645 learned_addr learned my addr 172.21.15.78:0/3739548645 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:57.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4861d9b640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4864104d30 0x7f486419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f4862d9d640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a30 0x7f48641a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 -- 172.21.15.78:0/3739548645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a30 msgr2=0x7f48641a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a30 0x7f48641a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 -- 172.21.15.78:0/3739548645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4864104d30 msgr2=0x7f486419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4864104d30 0x7f486419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.502+0000 7f486259c640 1 -- 172.21.15.78:0/3739548645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48641a1d30 con 0x7f4864103af0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.503+0000 7f486259c640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864103af0 0x7f486419a1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4854005d00 tx=0x7f485400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.503+0000 7f48537fe640 1 -- 172.21.15.78:0/3739548645 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854017070 con 0x7f4864103af0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.503+0000 7f48537fe640 1 -- 172.21.15.78:0/3739548645 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4854009c30 con 0x7f4864103af0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.503+0000 7f48537fe640 1 -- 172.21.15.78:0/3739548645 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485400f620 con 0x7f4864103af0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.503+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48641a1fc0 con 0x7f4864103af0 2024-04-25T14:01:57.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.503+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48641a2430 con 0x7f4864103af0 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.505+0000 7f48537fe640 1 -- 172.21.15.78:0/3739548645 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f485401a070 con 0x7f4864103af0 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.505+0000 7f486fbf5740 1 Processor -- start 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486fbf5740 1 -- start start 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e2f390 0x5637b31d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31d53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31d6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486fbf5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637b3135d80 con 0x5637b2e2f390 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486fbf5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5637b30ef8b0 con 0x5637b2e064d0 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486fbf5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637b3131d40 con 0x5637b2e0e6b0 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f4862d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e2f390 0x5637b31d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f4862d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e2f390 0x5637b31d4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37414/0 (socket says 172.21.15.78:37414) 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f4862d9d640 1 -- 172.21.15.78:0/4235900914 learned_addr learned my addr 172.21.15.78:0/4235900914 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f48537fe640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f484c0afaf0 0x7f484c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f4861d9b640 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.506+0000 7f486259c640 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31d53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 -- 172.21.15.78:0/4235900914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 msgr2=0x5637b31d6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31d6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 -- 172.21.15.78:0/4235900914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 msgr2=0x5637b31d53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31d53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 -- 172.21.15.78:0/4235900914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b30c0a40 con 0x5637b2e2f390 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f48537fe640 1 -- 172.21.15.78:0/3739548645 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f48540de740 con 0x7f4864103af0 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f486259c640 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31d53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f484c0afaf0 0x7f484c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e2f390 0x5637b31d4e70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f485c0011c0 tx=0x7f485c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2ec5ea94ccd57e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4850ff9640 1 -- 172.21.15.78:0/4235900914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c004040 con 0x5637b2e2f390 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4862d9d640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f484c0afaf0 0x7f484c0b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f485c04c000 tx=0x7f485c049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4850ff9640 1 -- 172.21.15.78:0/4235900914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f485c036040 con 0x5637b2e2f390 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.507+0000 7f4850ff9640 1 -- 172.21.15.78:0/4235900914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c02fa20 con 0x5637b2e2f390 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 -- 172.21.15.78:0/4235900914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e2f390 msgr2=0x5637b31d4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e2f390 0x5637b31d4e70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f485c0011c0 tx=0x7f485c009800 comp rx=0 tx=0).stop 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 -- 172.21.15.78:0/4235900914 shutdown_connections 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31d53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 --2- 172.21.15.78:0/4235900914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e2f390 0x5637b31d4e70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 -- 172.21.15.78:0/4235900914 >> 172.21.15.78:0/4235900914 conn(0x5637b2e234e0 msgr2=0x5637b31cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 -- 172.21.15.78:0/4235900914 shutdown_connections 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 -- 172.21.15.78:0/4235900914 wait complete. 2024-04-25T14:01:57.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.508+0000 7f486fbf5740 1 -- start start 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 1 Processor -- start 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e064d0 0x5637b31debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637b31d6a10 con 0x5637b2e0e6b0 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5637b30bcd10 con 0x5637b2e2f390 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637b31d6b90 con 0x5637b2e064d0 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486fbf5740 10 client.0 _ll_register_callbacks cb 0x5637b30e9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31df0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37418/0 (socket says 172.21.15.78:37418) 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 -- 172.21.15.78:0/3430706468 learned_addr learned my addr 172.21.15.78:0/3430706468 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f486259c640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e064d0 0x5637b31debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4862d9d640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 -- 172.21.15.78:0/3430706468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e064d0 msgr2=0x5637b31debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e064d0 0x5637b31debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 -- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 msgr2=0x5637b31e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 -- 172.21.15.78:0/3430706468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b2f117d0 con 0x5637b2e0e6b0 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4862d9d640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.509+0000 7f4861d9b640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31df0f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f485800bf40 tx=0x7f48580128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd8ac7934e47a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f4849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f4849ffb640 1 -- 172.21.15.78:0/3430706468 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485801f060 con 0x5637b2e0e6b0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f4849ffb640 1 -- 172.21.15.78:0/3430706468 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f48580108a0 con 0x5637b2e0e6b0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f485c036040 con 0x5637b2e0e6b0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f4849ffb640 1 -- 172.21.15.78:0/3430706468 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4858010aa0 con 0x5637b2e0e6b0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 10 client.16116 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5637b2e473f0 con 0x5637b2e0e6b0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f4849ffb640 1 -- 172.21.15.78:0/3430706468 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f485c036040 con 0x5637b2e0e6b0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f4849ffb640 1 -- 172.21.15.78:0/3430706468 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4858019030 con 0x5637b2e0e6b0 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 10 client.16116 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:57.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 20 client.16116 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dedf60 0x5637b31ec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 4 client.16116 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.510+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637b3143990 con 0x5637b2dedf60 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.511+0000 7f486259c640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dedf60 0x5637b31ec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.511+0000 7f4849ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.511+0000 7f486259c640 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dedf60 0x5637b31ec520 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a675927c49a24f server_cookie=330863990b64ccdf in_seq=0 out_seq=0 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f4849ffb640 1 -- 172.21.15.78:0/3430706468 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5637b3143990 con 0x5637b2dedf60 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f4849ffb640 10 client.16116 handle_command_reply: tid=0 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 client.16116 shutdown 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 msgr2=0x5637b31df0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31df0f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f485800bf40 tx=0x7f48580128e0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 shutdown_connections 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31df0f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dedf60 0x5637b31ec520 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 --2- 172.21.15.78:0/3430706468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e064d0 0x5637b31debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f4849ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 >> 172.21.15.78:0/3430706468 conn(0x5637b2e234e0 msgr2=0x5637b31cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 shutdown_connections 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 1 -- 172.21.15.78:0/3430706468 wait complete. 2024-04-25T14:01:57.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.512+0000 7f486fbf5740 20 client.16116 trim_cache size 0 max 16384 2024-04-25T14:01:57.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 Processor -- start 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 -- start start 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e064d0 0x5637b31da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48641ab9d0 con 0x5637b2e064d0 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48641a92a0 con 0x5637b2e2f390 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486fbf5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48641a70f0 con 0x5637b2e0e6b0 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e064d0 0x5637b31da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e064d0 0x5637b31da0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37434/0 (socket says 172.21.15.78:37434) 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.530+0000 7f486259c640 1 -- 172.21.15.78:0/187482117 learned_addr learned my addr 172.21.15.78:0/187482117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f4862d9d640 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f4861d9b640 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f486259c640 1 -- 172.21.15.78:0/187482117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 msgr2=0x5637b31cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f486259c640 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f486259c640 1 -- 172.21.15.78:0/187482117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 msgr2=0x5637b31e5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f486259c640 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f486259c640 1 -- 172.21.15.78:0/187482117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b2e070f0 con 0x5637b2e064d0 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f486259c640 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e064d0 0x5637b31da0c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4854010c40 tx=0x7f48540c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7c7e3bfa9800e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f484bfff640 1 -- 172.21.15.78:0/187482117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854021040 con 0x5637b2e064d0 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f484bfff640 1 -- 172.21.15.78:0/187482117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48540a6720 con 0x5637b2e064d0 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.531+0000 7f484bfff640 1 -- 172.21.15.78:0/187482117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48540aafc0 con 0x5637b2e064d0 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- 172.21.15.78:0/187482117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e064d0 msgr2=0x5637b31da0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e064d0 0x5637b31da0c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4854010c40 tx=0x7f48540c9040 comp rx=0 tx=0).stop 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- 172.21.15.78:0/187482117 shutdown_connections 2024-04-25T14:01:57.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e2f390 0x5637b31e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e0e6b0 0x5637b31cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 --2- 172.21.15.78:0/187482117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e064d0 0x5637b31da0c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- 172.21.15.78:0/187482117 >> 172.21.15.78:0/187482117 conn(0x5637b2e234e0 msgr2=0x5637b31cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- 172.21.15.78:0/187482117 shutdown_connections 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- 172.21.15.78:0/187482117 wait complete. 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- start start 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 Processor -- start 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e2f390 0x5637b3252880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48641a7760 con 0x5637b2e0e6b0 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48641a8bd0 con 0x5637b2e064d0 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48641a7e30 con 0x5637b2e2f390 2024-04-25T14:01:57.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.532+0000 7f486fbf5740 10 client.0 _ll_register_callbacks cb 0x7f485800b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31e3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37442/0 (socket says 172.21.15.78:37442) 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 -- 172.21.15.78:0/1906879550 learned_addr learned my addr 172.21.15.78:0/1906879550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f486259c640 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 -- 172.21.15.78:0/1906879550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e2f390 msgr2=0x5637b3252880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e2f390 0x5637b3252880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 -- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 msgr2=0x5637b31e35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31e35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 -- 172.21.15.78:0/1906879550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b30f64b0 con 0x5637b2e0e6b0 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f486259c640 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4861d9b640 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31e3b10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f485800c010 tx=0x7f4858010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca89c4d50b9986c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:57.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4849ffb640 1 -- 172.21.15.78:0/1906879550 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580108a0 con 0x5637b2e0e6b0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4849ffb640 1 -- 172.21.15.78:0/1906879550 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4858004020 con 0x5637b2e0e6b0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48540a6720 con 0x5637b2e0e6b0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4849ffb640 1 -- 172.21.15.78:0/1906879550 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f485801f030 con 0x5637b2e0e6b0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f486fbf5740 10 client.16122 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5637b3253d00 con 0x5637b2e0e6b0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.533+0000 7f4849ffb640 1 -- 172.21.15.78:0/1906879550 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48540a6720 con 0x5637b2e0e6b0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f4849ffb640 1 -- 172.21.15.78:0/1906879550 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f485802c070 con 0x5637b2e0e6b0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486fbf5740 10 client.16122 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486fbf5740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486fbf5740 20 client.16122 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486fbf5740 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dc33a0 0x5637b3256770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486fbf5740 4 client.16122 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5637b31d9560 con 0x5637b2dc33a0 2024-04-25T14:01:57.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486259c640 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dc33a0 0x5637b3256770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f4849ffb640 10 client.16122 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.534+0000 7f486259c640 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dc33a0 0x5637b3256770 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab98e33cd6a31c server_cookie=bead6b205bf569ac in_seq=0 out_seq=0 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4849ffb640 1 -- 172.21.15.78:0/1906879550 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5637b31d9560 con 0x5637b2dc33a0 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4849ffb640 10 client.16122 handle_command_reply: tid=0 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 client.16122 shutdown 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 msgr2=0x5637b31e3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31e3b10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f485800c010 tx=0x7f4858010c40 comp rx=0 tx=0).stop 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 shutdown_connections 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b2dc33a0 0x5637b3256770 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b2e2f390 0x5637b3252880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b2e0e6b0 0x5637b31e3b10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4849ffb640 0 client.16122 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:57.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 --2- 172.21.15.78:0/1906879550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b2e064d0 0x5637b31e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 >> 172.21.15.78:0/1906879550 conn(0x5637b2e234e0 msgr2=0x5637b31cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 shutdown_connections 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 1 -- 172.21.15.78:0/1906879550 wait complete. 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f486fbf5740 20 client.16122 trim_cache size 0 max 16384 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:01:57.542 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:01:57.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f484c0afaf0 msgr2=0x7f484c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f484c0afaf0 0x7f484c0b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f485c04c000 tx=0x7f485c049000 comp rx=0 tx=0).stop 2024-04-25T14:01:57.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864103af0 msgr2=0x7f486419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864103af0 0x7f486419a1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4854005d00 tx=0x7f485400dde0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 shutdown_connections 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a30 0x7f48641a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4864104d30 0x7f486419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f484c0afaf0 0x7f484c0b2000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 --2- 172.21.15.78:0/3739548645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4864103af0 0x7f486419a1f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 >> 172.21.15.78:0/3739548645 conn(0x7f48640fece0 msgr2=0x7f4864105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 shutdown_connections 2024-04-25T14:01:57.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.535+0000 7f4868d3f640 1 -- 172.21.15.78:0/3739548645 wait complete. 2024-04-25T14:01:57.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:01:57.547 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:01:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 -- 172.21.15.78:0/3960642077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74102a00 msgr2=0x7eff74103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 --2- 172.21.15.78:0/3960642077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74102a00 0x7eff74103700 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009ca0 tx=0x7eff6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 -- 172.21.15.78:0/3960642077 shutdown_connections 2024-04-25T14:01:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 --2- 172.21.15.78:0/3960642077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff74108940 0x7eff7410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 --2- 172.21.15.78:0/3960642077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff74103c40 0x7eff74104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 --2- 172.21.15.78:0/3960642077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74102a00 0x7eff74103700 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 -- 172.21.15.78:0/3960642077 >> 172.21.15.78:0/3960642077 conn(0x7eff740fdbf0 msgr2=0x7eff741000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 -- 172.21.15.78:0/3960642077 shutdown_connections 2024-04-25T14:01:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.656+0000 7eff78a05640 1 -- 172.21.15.78:0/3960642077 wait complete. 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 Processor -- start 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 -- start start 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff74102a00 0x7eff74075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff74103c40 0x7eff7419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74108940 0x7eff741a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff7410e4e0 con 0x7eff74108940 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff7410e360 con 0x7eff74102a00 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff78a05640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff7410e660 con 0x7eff74103c40 2024-04-25T14:01:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74108940 0x7eff741a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff72d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74108940 0x7eff741a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37458/0 (socket says 172.21.15.78:37458) 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff72d9d640 1 -- 172.21.15.78:0/1387162830 learned_addr learned my addr 172.21.15.78:0/1387162830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff71d9b640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff74103c40 0x7eff7419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.657+0000 7eff7259c640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff74102a00 0x7eff74075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff72d9d640 1 -- 172.21.15.78:0/1387162830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff74103c40 msgr2=0x7eff7419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff72d9d640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff74103c40 0x7eff7419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff72d9d640 1 -- 172.21.15.78:0/1387162830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff74102a00 msgr2=0x7eff74075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff72d9d640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff74102a00 0x7eff74075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff72d9d640 1 -- 172.21.15.78:0/1387162830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff741a1d30 con 0x7eff74108940 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff72d9d640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74108940 0x7eff741a1510 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7eff6400d750 tx=0x7eff6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff6b7fe640 1 -- 172.21.15.78:0/1387162830 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64017070 con 0x7eff74108940 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff6b7fe640 1 -- 172.21.15.78:0/1387162830 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff640054d0 con 0x7eff74108940 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff741a1fc0 con 0x7eff74108940 2024-04-25T14:01:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff6b7fe640 1 -- 172.21.15.78:0/1387162830 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64004c60 con 0x7eff74108940 2024-04-25T14:01:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.658+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff741a2430 con 0x7eff74108940 2024-04-25T14:01:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.659+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff74107750 con 0x7eff74108940 2024-04-25T14:01:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.660+0000 7eff6b7fe640 1 -- 172.21.15.78:0/1387162830 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7eff64009bd0 con 0x7eff74108940 2024-04-25T14:01:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.661+0000 7eff6b7fe640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff600afaf0 0x7eff600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.661+0000 7eff7259c640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff600afaf0 0x7eff600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.661+0000 7eff6b7fe640 1 -- 172.21.15.78:0/1387162830 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7eff640de560 con 0x7eff74108940 2024-04-25T14:01:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.661+0000 7eff7259c640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff600afaf0 0x7eff600b2000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff6c000f80 tx=0x7eff6c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.666+0000 7eff6b7fe640 1 -- 172.21.15.78:0/1387162830 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff640a6ae0 con 0x7eff74108940 2024-04-25T14:01:57.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.878+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff740637a0 con 0x7eff74108940 2024-04-25T14:01:57.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.879+0000 7eff6b7fe640 1 -- 172.21.15.78:0/1387162830 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7eff640aade0 con 0x7eff74108940 2024-04-25T14:01:57.879 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:01:57.879 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:01:57.879 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff600afaf0 msgr2=0x7eff600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff600afaf0 0x7eff600b2000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff6c000f80 tx=0x7eff6c004a40 comp rx=0 tx=0).stop 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74108940 msgr2=0x7eff741a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74108940 0x7eff741a1510 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7eff6400d750 tx=0x7eff6400dc10 comp rx=0 tx=0).stop 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 shutdown_connections 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff600afaf0 0x7eff600b2000 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff74108940 0x7eff741a1510 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff74103c40 0x7eff7419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 --2- 172.21.15.78:0/1387162830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff74102a00 0x7eff74075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:57.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 >> 172.21.15.78:0/1387162830 conn(0x7eff740fdbf0 msgr2=0x7eff74104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:57.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 shutdown_connections 2024-04-25T14:01:57.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:57.881+0000 7eff78a05640 1 -- 172.21.15.78:0/1387162830 wait complete. 2024-04-25T14:01:57.893 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.003+0000 7f393e2e0640 1 -- 172.21.15.78:0/584861526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3938106f40 msgr2=0x7f3938109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.003+0000 7f393e2e0640 1 --2- 172.21.15.78:0/584861526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3938106f40 0x7f3938109360 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f392c009900 tx=0x7f392c02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 -- 172.21.15.78:0/584861526 shutdown_connections 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 --2- 172.21.15.78:0/584861526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39381098a0 0x7f393810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 --2- 172.21.15.78:0/584861526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3938106f40 0x7f3938109360 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 --2- 172.21.15.78:0/584861526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39381045e0 0x7f3938106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 -- 172.21.15.78:0/584861526 >> 172.21.15.78:0/584861526 conn(0x7f39380fdbc0 msgr2=0x7f3938100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 -- 172.21.15.78:0/584861526 shutdown_connections 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 -- 172.21.15.78:0/584861526 wait complete. 2024-04-25T14:01:58.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 Processor -- start 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.004+0000 7f393e2e0640 1 -- start start 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39381045e0 0x7f393819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3938106f40 0x7f393819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39381098a0 0x7f39381a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f393810f600 con 0x7f39381045e0 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f393810f480 con 0x7f39381098a0 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f393810f780 con 0x7f3938106f40 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39381045e0 0x7f393819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39381045e0 0x7f393819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37490/0 (socket says 172.21.15.78:37490) 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 -- 172.21.15.78:0/2465339595 learned_addr learned my addr 172.21.15.78:0/2465339595 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393c87d640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39381098a0 0x7f39381a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f39377fe640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3938106f40 0x7f393819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 -- 172.21.15.78:0/2465339595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3938106f40 msgr2=0x7f393819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3938106f40 0x7f393819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 -- 172.21.15.78:0/2465339595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39381098a0 msgr2=0x7f39381a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39381098a0 0x7f39381a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 -- 172.21.15.78:0/2465339595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39381a1c60 con 0x7f39381045e0 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f3937fff640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39381045e0 0x7f393819a1d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f392800c970 tx=0x7f392800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f39357fa640 1 -- 172.21.15.78:0/2465339595 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928014070 con 0x7f39381045e0 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f39357fa640 1 -- 172.21.15.78:0/2465339595 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3928004440 con 0x7f39381045e0 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39381a1f50 con 0x7f39381045e0 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f39357fa640 1 -- 172.21.15.78:0/2465339595 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928003e50 con 0x7f39381045e0 2024-04-25T14:01:58.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.005+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381a2410 con 0x7f39381045e0 2024-04-25T14:01:58.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 Processor -- start 2024-04-25T14:01:58.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f39357fa640 1 -- 172.21.15.78:0/2465339595 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3928017070 con 0x7f39381045e0 2024-04-25T14:01:58.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 -- start start 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa5a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa5ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55961a9bed80 con 0x55961a68f4d0 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55961a978a20 con 0x55961a6976b0 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f394518b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55961a9bad40 con 0x55961a6b8390 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa5ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa5ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37492/0 (socket says 172.21.15.78:37492) 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 -- 172.21.15.78:0/157708639 learned_addr learned my addr 172.21.15.78:0/157708639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f393c87d640 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa5a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 -- 172.21.15.78:0/157708639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 msgr2=0x55961aa5a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa5a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f39377fe640 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 -- 172.21.15.78:0/157708639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 msgr2=0x55961aa5ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa5ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f3937fff640 1 -- 172.21.15.78:0/157708639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961a983c30 con 0x55961a68f4d0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.008+0000 7f39357fa640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39200afd10 0x7f39200b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.009+0000 7f393c87d640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39200afd10 0x7f39200b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.009+0000 7f3937fff640 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa5ad90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f39280e4c10 tx=0x7f39280e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac26689d6acfe3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.009+0000 7f39357fa640 1 -- 172.21.15.78:0/2465339595 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f39280de710 con 0x7f39381045e0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.009+0000 7f391e7fc640 1 -- 172.21.15.78:0/157708639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280a2aa0 con 0x55961a68f4d0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.009+0000 7f391e7fc640 1 -- 172.21.15.78:0/157708639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392804f020 con 0x55961a68f4d0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.009+0000 7f391e7fc640 1 -- 172.21.15.78:0/157708639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280a6350 con 0x55961a68f4d0 2024-04-25T14:01:58.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 -- 172.21.15.78:0/157708639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 msgr2=0x55961aa5ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa5ad90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f39280e4c10 tx=0x7f39280e6690 comp rx=0 tx=0).stop 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f393c87d640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39200afd10 0x7f39200b2220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55961aa5b860 tx=0x7f3930007540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 -- 172.21.15.78:0/157708639 shutdown_connections 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa5ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa5ad90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 --2- 172.21.15.78:0/157708639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa5a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 -- 172.21.15.78:0/157708639 >> 172.21.15.78:0/157708639 conn(0x55961a6ac4e0 msgr2=0x55961aa56020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 -- 172.21.15.78:0/157708639 shutdown_connections 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 -- 172.21.15.78:0/157708639 wait complete. 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 -- start start 2024-04-25T14:01:58.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.010+0000 7f394518b740 1 Processor -- start 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f394518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f394518b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a6976b0 0x55961aa64ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f394518b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa6b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f394518b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55961aa5c530 con 0x55961a6976b0 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f394518b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55961a945dc0 con 0x55961a68f4d0 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f394518b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55961aa5c6b0 con 0x55961a6b8390 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f394518b740 10 client.0 _ll_register_callbacks cb 0x55961a961600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a6976b0 0x55961aa64ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a6976b0 0x55961aa64ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37506/0 (socket says 172.21.15.78:37506) 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 -- 172.21.15.78:0/3019492304 learned_addr learned my addr 172.21.15.78:0/3019492304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f393c87d640 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa6b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f3937fff640 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 -- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 msgr2=0x55961aa6b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa6b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 -- 172.21.15.78:0/3019492304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 msgr2=0x55961aa64590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa64590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 -- 172.21.15.78:0/3019492304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961a6d03f0 con 0x55961a6976b0 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39377fe640 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a6976b0 0x55961aa64ad0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f392c009900 tx=0x7f392c0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752b6ccb973b55c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39177fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39177fe640 1 -- 172.21.15.78:0/3019492304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c035040 con 0x55961a6976b0 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.011+0000 7f39177fe640 1 -- 172.21.15.78:0/3019492304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f392c004450 con 0x55961a6976b0 2024-04-25T14:01:58.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392804f020 con 0x55961a6976b0 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f39177fe640 1 -- 172.21.15.78:0/3019492304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f392c0035f0 con 0x55961a6976b0 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 10 client.16140 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55961a79a7d0 con 0x55961a6976b0 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f39177fe640 1 -- 172.21.15.78:0/3019492304 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392804f020 con 0x55961a6976b0 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f39177fe640 1 -- 172.21.15.78:0/3019492304 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f392c03a040 con 0x55961a6976b0 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 10 client.16140 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 20 client.16140 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a676f60 0x55961aa72290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 4 client.16140 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55961a9cc990 con 0x55961a676f60 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.012+0000 7f3937fff640 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a676f60 0x55961aa72290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.013+0000 7f3937fff640 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a676f60 0x55961aa72290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=276ee8eb274040ad server_cookie=55b02279e68fc3c2 in_seq=0 out_seq=0 2024-04-25T14:01:58.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.013+0000 7f39177fe640 10 client.16140 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f39177fe640 1 -- 172.21.15.78:0/3019492304 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55961a9cc990 con 0x55961a676f60 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f39177fe640 10 client.16140 handle_command_reply: tid=0 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 client.16140 shutdown 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a6976b0 msgr2=0x55961aa64ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a6976b0 0x55961aa64ad0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f392c009900 tx=0x7f392c0317d0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 shutdown_connections 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a676f60 0x55961aa72290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6b8390 0x55961aa6b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f39177fe640 0 client.16140 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a6976b0 0x55961aa64ad0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 --2- 172.21.15.78:0/3019492304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa64590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 >> 172.21.15.78:0/3019492304 conn(0x55961a6ac4e0 msgr2=0x55961aa58f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 shutdown_connections 2024-04-25T14:01:58.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 1 -- 172.21.15.78:0/3019492304 wait complete. 2024-04-25T14:01:58.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.014+0000 7f394518b740 20 client.16140 trim_cache size 0 max 16384 2024-04-25T14:01:58.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.027+0000 7f394518b740 1 Processor -- start 2024-04-25T14:01:58.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.027+0000 7f394518b740 1 -- start start 2024-04-25T14:01:58.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f394518b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a676f60 0x55961aa63430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f394518b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f394518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa76a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f394518b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39381adb30 con 0x55961a68f4d0 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f394518b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39381ab400 con 0x55961a6976b0 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f394518b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a9250 con 0x55961a676f60 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa6b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37514/0 (socket says 172.21.15.78:37514) 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 -- 172.21.15.78:0/444361465 learned_addr learned my addr 172.21.15.78:0/444361465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f3937fff640 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a676f60 0x55961aa63430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f393c87d640 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa76a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 -- 172.21.15.78:0/444361465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a676f60 msgr2=0x55961aa63430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a676f60 0x55961aa63430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 -- 172.21.15.78:0/444361465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 msgr2=0x55961aa76a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa76a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 -- 172.21.15.78:0/444361465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961aa5fce0 con 0x55961a68f4d0 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f393c87d640 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa76a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f39377fe640 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa6b8e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f392c005940 tx=0x7f392c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731033080ccd9695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f391dffb640 1 -- 172.21.15.78:0/444361465 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c035040 con 0x55961a68f4d0 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f391dffb640 1 -- 172.21.15.78:0/444361465 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c054020 con 0x55961a68f4d0 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.028+0000 7f391dffb640 1 -- 172.21.15.78:0/444361465 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c03d690 con 0x55961a68f4d0 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 -- 172.21.15.78:0/444361465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 msgr2=0x55961aa6b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa6b8e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f392c005940 tx=0x7f392c001e70 comp rx=0 tx=0).stop 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 -- 172.21.15.78:0/444361465 shutdown_connections 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a6976b0 0x55961aa76a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a68f4d0 0x55961aa6b8e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 --2- 172.21.15.78:0/444361465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a676f60 0x55961aa63430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 -- 172.21.15.78:0/444361465 >> 172.21.15.78:0/444361465 conn(0x55961a6ac4e0 msgr2=0x55961aa62d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 -- 172.21.15.78:0/444361465 shutdown_connections 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 -- 172.21.15.78:0/444361465 wait complete. 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.029+0000 7f394518b740 1 -- start start 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 1 Processor -- start 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a676f60 0x55961aa6aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6976b0 0x55961aad8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a98c0 con 0x55961a676f60 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39381aad30 con 0x55961a68f4d0 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39381a9f90 con 0x55961a6976b0 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f394518b740 10 client.0 _ll_register_callbacks cb 0x7f392c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a676f60 0x55961aa6aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a676f60 0x55961aa6aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37524/0 (socket says 172.21.15.78:37524) 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 -- 172.21.15.78:0/2430809935 learned_addr learned my addr 172.21.15.78:0/2430809935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f393c87d640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6976b0 0x55961aad8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f39377fe640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa6b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 -- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6976b0 msgr2=0x55961aad8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6976b0 0x55961aad8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 -- 172.21.15.78:0/2430809935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 msgr2=0x55961aa6b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa6b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 -- 172.21.15.78:0/2430809935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961aa63970 con 0x55961a676f60 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f39377fe640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f3937fff640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a676f60 0x55961aa6aac0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f39280e41b0 tx=0x7f392800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841c7a2342bd8a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.030+0000 7f39177fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f39177fe640 1 -- 172.21.15.78:0/2430809935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280a6530 con 0x55961a676f60 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f39177fe640 1 -- 172.21.15.78:0/2430809935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f39280e4730 con 0x55961a676f60 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392c054020 con 0x55961a676f60 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f39177fe640 1 -- 172.21.15.78:0/2430809935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f39280a69b0 con 0x55961a676f60 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 10 client.16146 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55961aad9c00 con 0x55961a676f60 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f39177fe640 1 -- 172.21.15.78:0/2430809935 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c054020 con 0x55961a676f60 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f39177fe640 1 -- 172.21.15.78:0/2430809935 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f39280a2e70 con 0x55961a676f60 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 10 client.16146 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 20 client.16146 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a6b8390 0x55961aadc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 4 client.16146 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55961a9c6800 con 0x55961a6b8390 2024-04-25T14:01:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.031+0000 7f39377fe640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a6b8390 0x55961aadc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f39177fe640 10 client.16146 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f39377fe640 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a6b8390 0x55961aadc1d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae7914b3649a968b server_cookie=e0db9a79df8a8c67 in_seq=0 out_seq=0 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f39177fe640 1 -- 172.21.15.78:0/2430809935 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55961a9c6800 con 0x55961a6b8390 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f39177fe640 10 client.16146 handle_command_reply: tid=0 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 client.16146 shutdown 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a676f60 msgr2=0x55961aa6aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a676f60 0x55961aa6aac0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f39280e41b0 tx=0x7f392800cf60 comp rx=0 tx=0).stop 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 shutdown_connections 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55961a6b8390 0x55961aadc1d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55961a6976b0 0x55961aad8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55961a68f4d0 0x55961aa6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f39177fe640 0 client.16146 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 --2- 172.21.15.78:0/2430809935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55961a676f60 0x55961aa6aac0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 >> 172.21.15.78:0/2430809935 conn(0x55961a6ac4e0 msgr2=0x55961aa58e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 shutdown_connections 2024-04-25T14:01:58.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 1 -- 172.21.15.78:0/2430809935 wait complete. 2024-04-25T14:01:58.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.032+0000 7f394518b740 20 client.16146 trim_cache size 0 max 16384 2024-04-25T14:01:58.040 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39200afd10 msgr2=0x7f39200b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39200afd10 0x7f39200b2220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55961aa5b860 tx=0x7f3930007540 comp rx=0 tx=0).stop 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39381045e0 msgr2=0x7f393819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39381045e0 0x7f393819a1d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f392800c970 tx=0x7f392800ce30 comp rx=0 tx=0).stop 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 shutdown_connections 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39381098a0 0x7f39381a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3938106f40 0x7f393819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39200afd10 0x7f39200b2220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 --2- 172.21.15.78:0/2465339595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39381045e0 0x7f393819a1d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 >> 172.21.15.78:0/2465339595 conn(0x7f39380fdbc0 msgr2=0x7f3938105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 shutdown_connections 2024-04-25T14:01:58.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.033+0000 7f393e2e0640 1 -- 172.21.15.78:0/2465339595 wait complete. 2024-04-25T14:01:58.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:01:58.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 -- 172.21.15.78:0/3345275842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa280103c20 msgr2=0x7fa280104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 --2- 172.21.15.78:0/3345275842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa280103c20 0x7fa280104030 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa270009900 tx=0x7fa27002a410 comp rx=0 tx=0).stop 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 -- 172.21.15.78:0/3345275842 shutdown_connections 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 --2- 172.21.15.78:0/3345275842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa280108920 0x7fa28010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 --2- 172.21.15.78:0/3345275842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa280103c20 0x7fa280104030 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 --2- 172.21.15.78:0/3345275842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2801029e0 0x7fa2801036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 -- 172.21.15.78:0/3345275842 >> 172.21.15.78:0/3345275842 conn(0x7fa2800fdbd0 msgr2=0x7fa280100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 -- 172.21.15.78:0/3345275842 shutdown_connections 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.154+0000 7fa284e42640 1 -- 172.21.15.78:0/3345275842 wait complete. 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 Processor -- start 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 -- start start 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2801029e0 0x7fa280075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa280103c20 0x7fa28019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280108920 0x7fa2801a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010e500 con 0x7fa2801029e0 2024-04-25T14:01:58.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010e380 con 0x7fa280103c20 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa284e42640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010e680 con 0x7fa280108920 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa27e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2801029e0 0x7fa280075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa27e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2801029e0 0x7fa280075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37528/0 (socket says 172.21.15.78:37528) 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa27e59c640 1 -- 172.21.15.78:0/2392323687 learned_addr learned my addr 172.21.15.78:0/2392323687 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280108920 0x7fa2801a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.155+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa280103c20 0x7fa28019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa27e59c640 1 -- 172.21.15.78:0/2392323687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280108920 msgr2=0x7fa2801a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa27e59c640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280108920 0x7fa2801a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa27e59c640 1 -- 172.21.15.78:0/2392323687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa280103c20 msgr2=0x7fa28019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa27e59c640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa280103c20 0x7fa28019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa27e59c640 1 -- 172.21.15.78:0/2392323687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2801a1d20 con 0x7fa2801029e0 2024-04-25T14:01:58.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa27e59c640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2801029e0 0x7fa280075860 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa27400e9f0 tx=0x7fa27400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa26f7fe640 1 -- 172.21.15.78:0/2392323687 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27400ccc0 con 0x7fa2801029e0 2024-04-25T14:01:58.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa26f7fe640 1 -- 172.21.15.78:0/2392323687 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2740035f0 con 0x7fa2801029e0 2024-04-25T14:01:58.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2801a1fb0 con 0x7fa2801029e0 2024-04-25T14:01:58.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa26f7fe640 1 -- 172.21.15.78:0/2392323687 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27400ce60 con 0x7fa2801029e0 2024-04-25T14:01:58.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.156+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801a2420 con 0x7fa2801029e0 2024-04-25T14:01:58.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.158+0000 7fa28bdd0740 1 Processor -- start 2024-04-25T14:01:58.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa28bdd0740 1 -- start start 2024-04-25T14:01:58.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f2080810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f2080d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f2081a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa26f7fe640 1 -- 172.21.15.78:0/2392323687 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa274004030 con 0x7fa2801029e0 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581f1fe4d80 con 0x5581f1cde390 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581f1f9ea20 con 0x5581f1cbd6b0 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5581f1fe0d40 con 0x5581f1cb54d0 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f2080810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f2080810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37542/0 (socket says 172.21.15.78:37542) 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 -- 172.21.15.78:0/3583876042 learned_addr learned my addr 172.21.15.78:0/3583876042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27e59c640 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f2080d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f2081a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 -- 172.21.15.78:0/3583876042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 msgr2=0x5581f2080d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f2080d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 -- 172.21.15.78:0/3583876042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 msgr2=0x5581f2081a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f2081a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 -- 172.21.15.78:0/3583876042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f1fa9c30 con 0x5581f1cde390 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f2081a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.159+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f2080810 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa278007f40 tx=0x7fa27800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bd66ecd66d6992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.160+0000 7fa26f7fe640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2680afaf0 0x7fa2680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.160+0000 7fa26cff9640 1 -- 172.21.15.78:0/3583876042 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278010900 con 0x5581f1cde390 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.160+0000 7fa27e59c640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2680afaf0 0x7fa2680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.160+0000 7fa26cff9640 1 -- 172.21.15.78:0/3583876042 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2780183f0 con 0x5581f1cde390 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.160+0000 7fa26cff9640 1 -- 172.21.15.78:0/3583876042 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278017600 con 0x5581f1cde390 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.160+0000 7fa26f7fe640 1 -- 172.21.15.78:0/2392323687 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa2740df040 con 0x7fa2801029e0 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3583876042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 msgr2=0x5581f2080810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa27e59c640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2680afaf0 0x7fa2680b2000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5581f2080530 tx=0x7fa274090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f2080810 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa278007f40 tx=0x7fa27800f850 comp rx=0 tx=0).stop 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3583876042 shutdown_connections 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f2081a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f2080d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3583876042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f2080810 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3583876042 >> 172.21.15.78:0/3583876042 conn(0x5581f1cd24e0 msgr2=0x5581f207c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3583876042 shutdown_connections 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3583876042 wait complete. 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- start start 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 Processor -- start 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f208a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f208aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f20919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581f2082530 con 0x5581f1cde390 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581f1f6bdc0 con 0x5581f1cbd6b0 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5581f20826b0 con 0x5581f1cb54d0 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.161+0000 7fa28bdd0740 10 client.0 _ll_register_callbacks cb 0x5581f1f87600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f20919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f20919a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37556/0 (socket says 172.21.15.78:37556) 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 -- 172.21.15.78:0/237966733 learned_addr learned my addr 172.21.15.78:0/237966733 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f208a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f208aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 -- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 msgr2=0x5581f208a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f208a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 -- 172.21.15.78:0/237966733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 msgr2=0x5581f208aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f208aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 -- 172.21.15.78:0/237966733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f1cf63f0 con 0x5581f1cde390 2024-04-25T14:01:58.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f20919a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa278018ba0 tx=0x7fa278012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f40a5791da73f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa265ffb640 1 -- 172.21.15.78:0/237966733 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278004c40 con 0x5581f1cde390 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa265ffb640 1 -- 172.21.15.78:0/237966733 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa278004de0 con 0x5581f1cde390 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2780183f0 con 0x5581f1cde390 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa265ffb640 1 -- 172.21.15.78:0/237966733 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa278017650 con 0x5581f1cde390 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa28bdd0740 10 client.16158 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.162+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5581f1dc07d0 con 0x5581f1cde390 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa265ffb640 1 -- 172.21.15.78:0/237966733 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2780183f0 con 0x5581f1cde390 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa265ffb640 1 -- 172.21.15.78:0/237966733 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa27802f030 con 0x5581f1cde390 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa28bdd0740 10 client.16158 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa28bdd0740 10 client.16158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa28bdd0740 20 client.16158 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:58.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c9cf60 0x5581f2097ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa28bdd0740 4 client.16158 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581f1ff2990 con 0x5581f1c9cf60 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.163+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c9cf60 0x5581f2097ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.164+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c9cf60 0x5581f2097ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35346a37b3821610 server_cookie=7a1d0e90aadc2e6b in_seq=0 out_seq=0 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.164+0000 7fa265ffb640 10 client.16158 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.164+0000 7fa265ffb640 1 -- 172.21.15.78:0/237966733 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581f1ff2990 con 0x5581f1c9cf60 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.164+0000 7fa265ffb640 10 client.16158 handle_command_reply: tid=0 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.164+0000 7fa28bdd0740 1 client.16158 shutdown 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.164+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 msgr2=0x5581f20919a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.164+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f20919a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa278018ba0 tx=0x7fa278012040 comp rx=0 tx=0).stop 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 shutdown_connections 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cde390 0x5581f20919a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cbd6b0 0x5581f208aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c9cf60 0x5581f2097ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/237966733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f208a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa265ffb640 0 client.16158 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 >> 172.21.15.78:0/237966733 conn(0x5581f1cd24e0 msgr2=0x5581f207ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 shutdown_connections 2024-04-25T14:01:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 1 -- 172.21.15.78:0/237966733 wait complete. 2024-04-25T14:01:58.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.165+0000 7fa28bdd0740 20 client.16158 trim_cache size 0 max 16384 2024-04-25T14:01:58.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 Processor -- start 2024-04-25T14:01:58.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 -- start start 2024-04-25T14:01:58.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f207ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f20918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cde390 0x5581f207bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801adbd0 con 0x5581f1cbd6b0 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801ab4a0 con 0x5581f1cde390 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801a92f0 con 0x5581f1cb54d0 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f20918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f20918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37572/0 (socket says 172.21.15.78:37572) 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.178+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3540139477 learned_addr learned my addr 172.21.15.78:0/3540139477 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f207ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cde390 0x5581f207bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3540139477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 msgr2=0x5581f207ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f207ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3540139477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cde390 msgr2=0x5581f207bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cde390 0x5581f207bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3540139477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f2091e00 con 0x5581f1cbd6b0 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f20918c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa27002a930 tx=0x7fa270031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7863d72663f6a85d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa267fff640 1 -- 172.21.15.78:0/3540139477 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270035040 con 0x5581f1cbd6b0 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa267fff640 1 -- 172.21.15.78:0/3540139477 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa270048070 con 0x5581f1cbd6b0 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.179+0000 7fa267fff640 1 -- 172.21.15.78:0/3540139477 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270004a40 con 0x5581f1cbd6b0 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3540139477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 msgr2=0x5581f20918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f20918c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa27002a930 tx=0x7fa270031400 comp rx=0 tx=0).stop 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3540139477 shutdown_connections 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cde390 0x5581f207bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f20918c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3540139477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cb54d0 0x5581f207ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3540139477 >> 172.21.15.78:0/3540139477 conn(0x5581f1cd24e0 msgr2=0x5581f2085230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3540139477 shutdown_connections 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3540139477 wait complete. 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- start start 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 Processor -- start 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cb54d0 0x5581f208ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f208f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cde390 0x5581f20902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801a9960 con 0x5581f1cbd6b0 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801aadd0 con 0x5581f1cb54d0 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2801aa030 con 0x5581f1cde390 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa28bdd0740 10 client.0 _ll_register_callbacks cb 0x7fa278007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f208f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa27dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f208f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37574/0 (socket says 172.21.15.78:37574) 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.180+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3913322688 learned_addr learned my addr 172.21.15.78:0/3913322688 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cb54d0 0x5581f208ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cde390 0x5581f20902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cde390 msgr2=0x5581f20902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cde390 0x5581f20902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3913322688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cb54d0 msgr2=0x5581f208ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cb54d0 0x5581f208ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27dd9b640 1 -- 172.21.15.78:0/3913322688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f20908a0 con 0x5581f1cbd6b0 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cde390 0x5581f20902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa27dd9b640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f208f3b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa270004490 tx=0x7fa2700044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748b4b89f7ed33ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa265ffb640 1 -- 172.21.15.78:0/3913322688 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270035040 con 0x5581f1cbd6b0 2024-04-25T14:01:58.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa265ffb640 1 -- 172.21.15.78:0/3913322688 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa2700052d0 con 0x5581f1cbd6b0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa270048070 con 0x5581f1cbd6b0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa265ffb640 1 -- 172.21.15.78:0/3913322688 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa2700054d0 con 0x5581f1cbd6b0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa28bdd0740 10 client.16164 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5581f2091b50 con 0x5581f1cbd6b0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa265ffb640 1 -- 172.21.15.78:0/3913322688 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa270048070 con 0x5581f1cbd6b0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.181+0000 7fa265ffb640 1 -- 172.21.15.78:0/3913322688 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa270043ce0 con 0x5581f1cbd6b0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa28bdd0740 10 client.16164 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa28bdd0740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa28bdd0740 20 client.16164 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c723a0 0x5581f21028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa28bdd0740 4 client.16164 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581f1fec800 con 0x5581f1c723a0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c723a0 0x5581f21028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa27ed9d640 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c723a0 0x5581f21028b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca7afa05f580b7eb server_cookie=2021fa66acf57963 in_seq=0 out_seq=0 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.182+0000 7fa265ffb640 10 client.16164 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa265ffb640 1 -- 172.21.15.78:0/3913322688 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581f1fec800 con 0x5581f1c723a0 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa265ffb640 10 client.16164 handle_command_reply: tid=0 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 client.16164 shutdown 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 msgr2=0x5581f208f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f208f3b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa270004490 tx=0x7fa2700044c0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 shutdown_connections 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5581f1c723a0 0x5581f21028b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581f1cde390 0x5581f20902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa265ffb640 0 client.16164 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581f1cbd6b0 0x5581f208f3b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 --2- 172.21.15.78:0/3913322688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581f1cb54d0 0x5581f208ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 >> 172.21.15.78:0/3913322688 conn(0x5581f1cd24e0 msgr2=0x5581f207ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 shutdown_connections 2024-04-25T14:01:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 1 -- 172.21.15.78:0/3913322688 wait complete. 2024-04-25T14:01:58.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa28bdd0740 20 client.16164 trim_cache size 0 max 16384 2024-04-25T14:01:58.190 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:01:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2680afaf0 msgr2=0x7fa2680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2680afaf0 0x7fa2680b2000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5581f2080530 tx=0x7fa274090040 comp rx=0 tx=0).stop 2024-04-25T14:01:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2801029e0 msgr2=0x7fa280075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2801029e0 0x7fa280075860 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa27400e9f0 tx=0x7fa27400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 shutdown_connections 2024-04-25T14:01:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa280108920 0x7fa2801a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa280103c20 0x7fa28019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2680afaf0 0x7fa2680b2000 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 --2- 172.21.15.78:0/2392323687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2801029e0 0x7fa280075860 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 >> 172.21.15.78:0/2392323687 conn(0x7fa2800fdbd0 msgr2=0x7fa280104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 shutdown_connections 2024-04-25T14:01:58.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.183+0000 7fa284e42640 1 -- 172.21.15.78:0/2392323687 wait complete. 2024-04-25T14:01:58.193 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:57 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1387162830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:58.194 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:01:58.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:57 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1387162830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:58.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 -- 172.21.15.78:0/2444391498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb581045d0 msgr2=0x7feb581069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 --2- 172.21.15.78:0/2444391498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb581045d0 0x7feb581069f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7feb48009900 tx=0x7feb4802a410 comp rx=0 tx=0).stop 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 -- 172.21.15.78:0/2444391498 shutdown_connections 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 --2- 172.21.15.78:0/2444391498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb58109890 0x7feb5810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 --2- 172.21.15.78:0/2444391498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb58106f30 0x7feb58109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 --2- 172.21.15.78:0/2444391498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb581045d0 0x7feb581069f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 -- 172.21.15.78:0/2444391498 >> 172.21.15.78:0/2444391498 conn(0x7feb580fdbd0 msgr2=0x7feb58100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 -- 172.21.15.78:0/2444391498 shutdown_connections 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.304+0000 7feb5da34640 1 -- 172.21.15.78:0/2444391498 wait complete. 2024-04-25T14:01:58.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 Processor -- start 2024-04-25T14:01:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 -- start start 2024-04-25T14:01:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb581045d0 0x7feb5819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb58106f30 0x7feb5819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb58109890 0x7feb581a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb5810f5e0 con 0x7feb58109890 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb5810f460 con 0x7feb58106f30 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.305+0000 7feb5da34640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb5810f760 con 0x7feb581045d0 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb58109890 0x7feb581a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb58109890 0x7feb581a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37590/0 (socket says 172.21.15.78:37590) 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 -- 172.21.15.78:0/274561517 learned_addr learned my addr 172.21.15.78:0/274561517 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb567fc640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb58106f30 0x7feb5819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb56ffd640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb581045d0 0x7feb5819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 -- 172.21.15.78:0/274561517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb581045d0 msgr2=0x7feb5819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb581045d0 0x7feb5819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 -- 172.21.15.78:0/274561517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb58106f30 msgr2=0x7feb5819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb58106f30 0x7feb5819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 -- 172.21.15.78:0/274561517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb581a1d70 con 0x7feb58109890 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb56ffd640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb581045d0 0x7feb5819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb577fe640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb58109890 0x7feb581a17b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ccb0 tx=0x7feb5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb5ca32640 1 -- 172.21.15.78:0/274561517 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50003970 con 0x7feb58109890 2024-04-25T14:01:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb5ca32640 1 -- 172.21.15.78:0/274561517 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb50004030 con 0x7feb58109890 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb581a2060 con 0x7feb58109890 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb5ca32640 1 -- 172.21.15.78:0/274561517 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5000f6b0 con 0x7feb58109890 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.306+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb581a25b0 con 0x7feb58109890 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 Processor -- start 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb5ca32640 1 -- 172.21.15.78:0/274561517 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7feb50003b10 con 0x7feb58109890 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 -- start start 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d52810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d52d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d53a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558db2cb6d80 con 0x558db29874d0 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558db2c70a20 con 0x558db298f6b0 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb649be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558db2cb2d40 con 0x558db29b0390 2024-04-25T14:01:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d52d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d52d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37594/0 (socket says 172.21.15.78:37594) 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb56ffd640 1 -- 172.21.15.78:0/3348376163 learned_addr learned my addr 172.21.15.78:0/3348376163 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb567fc640 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d53a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.309+0000 7feb5ca32640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feb400afb70 0x7feb400b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb567fc640 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d52810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb56ffd640 1 -- 172.21.15.78:0/3348376163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 msgr2=0x558db2d52810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb56ffd640 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d52810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb56ffd640 1 -- 172.21.15.78:0/3348376163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 msgr2=0x558db2d53a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb56ffd640 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d53a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb577fe640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feb400afb70 0x7feb400b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb5ca32640 1 -- 172.21.15.78:0/274561517 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7feb500dec80 con 0x7feb58109890 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb56ffd640 1 -- 172.21.15.78:0/3348376163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558db2c7bc30 con 0x558db29874d0 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb56ffd640 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d52d90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feb480096c0 tx=0x7feb48032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00dee3964f0b8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb3dffb640 1 -- 172.21.15.78:0/3348376163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4803f070 con 0x558db29874d0 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb577fe640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feb400afb70 0x7feb400b2080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb500a6e50 tx=0x7feb500047b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb3dffb640 1 -- 172.21.15.78:0/3348376163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb48004030 con 0x558db29874d0 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.310+0000 7feb3dffb640 1 -- 172.21.15.78:0/3348376163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48035070 con 0x558db29874d0 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 -- 172.21.15.78:0/3348376163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 msgr2=0x558db2d52d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d52d90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feb480096c0 tx=0x7feb48032de0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 -- 172.21.15.78:0/3348376163 shutdown_connections 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d53a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d52d90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 --2- 172.21.15.78:0/3348376163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d52810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 -- 172.21.15.78:0/3348376163 >> 172.21.15.78:0/3348376163 conn(0x558db29a44e0 msgr2=0x558db2d4e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 -- 172.21.15.78:0/3348376163 shutdown_connections 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.311+0000 7feb649be740 1 -- 172.21.15.78:0/3348376163 wait complete. 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 -- start start 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 Processor -- start 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d5c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d5caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d63a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558db2d544d0 con 0x558db29874d0 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558db2c3ddc0 con 0x558db298f6b0 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558db2d54650 con 0x558db29b0390 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb649be740 10 client.0 _ll_register_callbacks cb 0x558db2c59600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d5c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d5c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37608/0 (socket says 172.21.15.78:37608) 2024-04-25T14:01:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb567fc640 1 -- 172.21.15.78:0/1871411773 learned_addr learned my addr 172.21.15.78:0/1871411773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb56ffd640 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d5caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.312+0000 7feb567fc640 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d63a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb567fc640 1 -- 172.21.15.78:0/1871411773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 msgr2=0x558db2d63a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb567fc640 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d63a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb567fc640 1 -- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 msgr2=0x558db2d5caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb567fc640 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d5caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb567fc640 1 -- 172.21.15.78:0/1871411773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db29c83f0 con 0x558db29874d0 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb567fc640 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d5c560 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0037a0 tx=0x7feb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730d82642ba29b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb36ffd640 1 -- 172.21.15.78:0/1871411773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c012a40 con 0x558db29874d0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb36ffd640 1 -- 172.21.15.78:0/1871411773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7feb4c012be0 con 0x558db29874d0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb48004030 con 0x558db29874d0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb36ffd640 1 -- 172.21.15.78:0/1871411773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feb4c005020 con 0x558db29874d0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb649be740 10 client.16176 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558db2a927d0 con 0x558db29874d0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.313+0000 7feb36ffd640 1 -- 172.21.15.78:0/1871411773 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb48004030 con 0x558db29874d0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb36ffd640 1 -- 172.21.15.78:0/1871411773 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7feb4c01d070 con 0x558db29874d0 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb649be740 10 client.16176 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb649be740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb649be740 20 client.16176 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb649be740 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db296ef60 0x558db2d6a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb649be740 4 client.16176 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558db2cc4990 con 0x558db296ef60 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb56ffd640 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db296ef60 0x558db2d6a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb36ffd640 10 client.16176 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.314+0000 7feb56ffd640 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db296ef60 0x558db2d6a300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3baffa145027a7 server_cookie=253a30b489bf1b29 in_seq=0 out_seq=0 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb36ffd640 1 -- 172.21.15.78:0/1871411773 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558db2cc4990 con 0x558db296ef60 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb36ffd640 10 client.16176 handle_command_reply: tid=0 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 client.16176 shutdown 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 msgr2=0x558db2d5c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d5c560 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0037a0 tx=0x7feb4c0046c0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 shutdown_connections 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db296ef60 0x558db2d6a300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29b0390 0x558db2d63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb36ffd640 0 client.16176 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2d5caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 --2- 172.21.15.78:0/1871411773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d5c560 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 >> 172.21.15.78:0/1871411773 conn(0x558db29a44e0 msgr2=0x558db2d50f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 shutdown_connections 2024-04-25T14:01:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.315+0000 7feb649be740 1 -- 172.21.15.78:0/1871411773 wait complete. 2024-04-25T14:01:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.316+0000 7feb649be740 20 client.16176 trim_cache size 0 max 16384 2024-04-25T14:01:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 Processor -- start 2024-04-25T14:01:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 -- start start 2024-04-25T14:01:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db296ef60 0x558db2d5c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29874d0 0x558db2d63990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db298f6b0 0x558db2d5c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb581adb20 con 0x558db298f6b0 2024-04-25T14:01:58.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb581ab3f0 con 0x558db296ef60 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb649be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb581a9240 con 0x558db29874d0 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db298f6b0 0x558db2d5c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db298f6b0 0x558db2d5c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37624/0 (socket says 172.21.15.78:37624) 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 -- 172.21.15.78:0/1358173618 learned_addr learned my addr 172.21.15.78:0/1358173618 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb56ffd640 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29874d0 0x558db2d63990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 -- 172.21.15.78:0/1358173618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29874d0 msgr2=0x558db2d63990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29874d0 0x558db2d63990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 -- 172.21.15.78:0/1358173618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db296ef60 msgr2=0x558db2d5c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db296ef60 0x558db2d5c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.329+0000 7feb567fc640 1 -- 172.21.15.78:0/1358173618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558db2d52360 con 0x558db298f6b0 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.330+0000 7feb567fc640 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db298f6b0 0x558db2d5c670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0045b0 tx=0x7feb4c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13146e30f84e97e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.330+0000 7feb3d7fa640 1 -- 172.21.15.78:0/1358173618 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c012910 con 0x558db298f6b0 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.330+0000 7feb3d7fa640 1 -- 172.21.15.78:0/1358173618 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4c012ab0 con 0x558db298f6b0 2024-04-25T14:01:58.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.330+0000 7feb3d7fa640 1 -- 172.21.15.78:0/1358173618 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c021840 con 0x558db298f6b0 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.330+0000 7feb649be740 1 -- 172.21.15.78:0/1358173618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db298f6b0 msgr2=0x558db2d5c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db298f6b0 0x558db2d5c670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0045b0 tx=0x7feb4c005d30 comp rx=0 tx=0).stop 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- 172.21.15.78:0/1358173618 shutdown_connections 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db298f6b0 0x558db2d5c670 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db29874d0 0x558db2d63990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 --2- 172.21.15.78:0/1358173618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db296ef60 0x558db2d5c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- 172.21.15.78:0/1358173618 >> 172.21.15.78:0/1358173618 conn(0x558db29a44e0 msgr2=0x558db2d48d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- 172.21.15.78:0/1358173618 shutdown_connections 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- 172.21.15.78:0/1358173618 wait complete. 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- start start 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 Processor -- start 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db296ef60 0x558db2d60ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d61010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2dd0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb581a98b0 con 0x558db29874d0 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb581aad20 con 0x558db298f6b0 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb581a9f80 con 0x558db296ef60 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.331+0000 7feb649be740 10 client.0 _ll_register_callbacks cb 0x7feb4c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d61010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d61010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37628/0 (socket says 172.21.15.78:37628) 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 -- 172.21.15.78:0/1438021117 learned_addr learned my addr 172.21.15.78:0/1438021117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb567fc640 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db296ef60 0x558db2d60ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb567fc640 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2dd0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 -- 172.21.15.78:0/1438021117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db296ef60 msgr2=0x558db2d60ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db296ef60 0x558db2d60ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 -- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 msgr2=0x558db2dd0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2dd0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 -- 172.21.15.78:0/1438021117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558db2dd0ba0 con 0x558db29874d0 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb56ffd640 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d61010 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feb48009900 tx=0x7feb480058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db24ee01be7b43db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb36ffd640 1 -- 172.21.15.78:0/1438021117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48053060 con 0x558db29874d0 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb36ffd640 1 -- 172.21.15.78:0/1438021117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7feb480315a0 con 0x558db29874d0 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb4c012ab0 con 0x558db29874d0 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb36ffd640 1 -- 172.21.15.78:0/1438021117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feb48047070 con 0x558db29874d0 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb649be740 10 client.16182 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:58.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558db2dd1e20 con 0x558db29874d0 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.332+0000 7feb36ffd640 1 -- 172.21.15.78:0/1438021117 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4c012ab0 con 0x558db29874d0 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb36ffd640 1 -- 172.21.15.78:0/1438021117 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7feb4803b430 con 0x558db29874d0 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb649be740 10 client.16182 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb649be740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb649be740 20 client.16182 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb649be740 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db29b0390 0x558db2dd43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb649be740 4 client.16182 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558db2cbe800 con 0x558db29b0390 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb567fc640 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db29b0390 0x558db2dd43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb36ffd640 10 client.16182 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:58.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.333+0000 7feb567fc640 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db29b0390 0x558db2dd43f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7074f4935af7e8bc server_cookie=64eccbd5d8c2dbfb in_seq=0 out_seq=0 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb36ffd640 1 -- 172.21.15.78:0/1438021117 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558db2cbe800 con 0x558db29b0390 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb36ffd640 10 client.16182 handle_command_reply: tid=0 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 client.16182 shutdown 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 msgr2=0x558db2d61010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d61010 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feb48009900 tx=0x7feb480058b0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 shutdown_connections 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558db298f6b0 0x558db2dd0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558db29874d0 0x558db2d61010 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558db29b0390 0x558db2dd43f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 --2- 172.21.15.78:0/1438021117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558db296ef60 0x558db2d60ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 >> 172.21.15.78:0/1438021117 conn(0x558db29a44e0 msgr2=0x558db2d5c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb36ffd640 0 client.16182 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 shutdown_connections 2024-04-25T14:01:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 1 -- 172.21.15.78:0/1438021117 wait complete. 2024-04-25T14:01:58.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb649be740 20 client.16182 trim_cache size 0 max 16384 2024-04-25T14:01:58.342 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:01:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feb400afb70 msgr2=0x7feb400b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feb400afb70 0x7feb400b2080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb500a6e50 tx=0x7feb500047b0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb58109890 msgr2=0x7feb581a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:58.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb58109890 0x7feb581a17b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feb5000ccb0 tx=0x7feb5000a620 comp rx=0 tx=0).stop 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 shutdown_connections 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feb400afb70 0x7feb400b2080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb58109890 0x7feb581a17b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb58106f30 0x7feb5819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 --2- 172.21.15.78:0/274561517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb581045d0 0x7feb5819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.334+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 >> 172.21.15.78:0/274561517 conn(0x7feb580fdbd0 msgr2=0x7feb58106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.335+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 shutdown_connections 2024-04-25T14:01:58.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:58.335+0000 7feb5da34640 1 -- 172.21.15.78:0/274561517 wait complete. 2024-04-25T14:01:58.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:57 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1387162830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:01:59.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:58 smithi078 ceph-mon[29163]: pgmap v422: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:59.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:58 smithi112 ceph-mon[32617]: pgmap v422: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:59.346 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:01:59.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:01:59.346 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:01:59.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:58 smithi129 ceph-mon[32569]: pgmap v422: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:01:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 -- 172.21.15.78:0/4114606624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128103af0 msgr2=0x7fb1281047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 --2- 172.21.15.78:0/4114606624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128103af0 0x7fb1281047f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb124009ca0 tx=0x7fb12402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 -- 172.21.15.78:0/4114606624 shutdown_connections 2024-04-25T14:01:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 --2- 172.21.15.78:0/4114606624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128109a30 0x7fb12810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 --2- 172.21.15.78:0/4114606624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb128104d30 0x7fb128105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 --2- 172.21.15.78:0/4114606624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128103af0 0x7fb1281047f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 -- 172.21.15.78:0/4114606624 >> 172.21.15.78:0/4114606624 conn(0x7fb1280fece0 msgr2=0x7fb128101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 -- 172.21.15.78:0/4114606624 shutdown_connections 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.474+0000 7fb1301d7640 1 -- 172.21.15.78:0/4114606624 wait complete. 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 Processor -- start 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 -- start start 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128103af0 0x7fb12819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128104d30 0x7fb12819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb128109a30 0x7fb1281a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb12810f5f0 con 0x7fb128104d30 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb12810f470 con 0x7fb128109a30 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb1301d7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb12810f770 con 0x7fb128103af0 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128104d30 0x7fb12819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128104d30 0x7fb12819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37640/0 (socket says 172.21.15.78:37640) 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 -- 172.21.15.78:0/1693280903 learned_addr learned my addr 172.21.15.78:0/1693280903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12df73640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128103af0 0x7fb12819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12e774640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb128109a30 0x7fb1281a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 -- 172.21.15.78:0/1693280903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128103af0 msgr2=0x7fb12819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128103af0 0x7fb12819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 -- 172.21.15.78:0/1693280903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb128109a30 msgr2=0x7fb1281a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb128109a30 0x7fb1281a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.475+0000 7fb12d772640 1 -- 172.21.15.78:0/1693280903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1281a1d80 con 0x7fb128104d30 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.476+0000 7fb12d772640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128104d30 0x7fb12819a7f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.476+0000 7fb11effd640 1 -- 172.21.15.78:0/1693280903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800cd80 con 0x7fb128104d30 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.476+0000 7fb11effd640 1 -- 172.21.15.78:0/1693280903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1180108e0 con 0x7fb128104d30 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.476+0000 7fb11effd640 1 -- 172.21.15.78:0/1693280903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118018980 con 0x7fb128104d30 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.476+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1281a2070 con 0x7fb128104d30 2024-04-25T14:01:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.476+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1281a25c0 con 0x7fb128104d30 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 Processor -- start 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb11effd640 1 -- 172.21.15.78:0/1693280903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb118010430 con 0x7fb128104d30 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 -- start start 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac754a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5574fabdab00 con 0x5574fa8ab550 2024-04-25T14:01:59.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574fab76920 con 0x5574fa8d4410 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb137164740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574fabd6ac0 con 0x5574fa8b3730 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb12df73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb12df73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac75a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37650/0 (socket says 172.21.15.78:37650) 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb12df73640 1 -- 172.21.15.78:0/3572440228 learned_addr learned my addr 172.21.15.78:0/3572440228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb12d772640 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb12e774640 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb11effd640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1140afaf0 0x7fb1140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb12df73640 1 -- 172.21.15.78:0/3572440228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 msgr2=0x5574fac7c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.479+0000 7fb12df73640 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb12df73640 1 -- 172.21.15.78:0/3572440228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 msgr2=0x5574fac754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb12df73640 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb12df73640 1 -- 172.21.15.78:0/3572440228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fa9b47e0 con 0x5574fa8ab550 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb12e774640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1140afaf0 0x7fb1140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb11effd640 1 -- 172.21.15.78:0/1693280903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb1180def30 con 0x7fb128104d30 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb12df73640 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac75a20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb1281033c0 tx=0x7fb124004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61850252c0a48f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb113fff640 1 -- 172.21.15.78:0/3572440228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124031650 con 0x5574fa8ab550 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb113fff640 1 -- 172.21.15.78:0/3572440228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb124036030 con 0x5574fa8ab550 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.480+0000 7fb113fff640 1 -- 172.21.15.78:0/3572440228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124004590 con 0x5574fa8ab550 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 -- 172.21.15.78:0/3572440228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 msgr2=0x5574fac75a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac75a20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb1281033c0 tx=0x7fb124004ab0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb12e774640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1140afaf0 0x7fb1140b2000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5574fac76670 tx=0x7fb1200046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 -- 172.21.15.78:0/3572440228 shutdown_connections 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac75a20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 --2- 172.21.15.78:0/3572440228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 -- 172.21.15.78:0/3572440228 >> 172.21.15.78:0/3572440228 conn(0x5574fa8c8560 msgr2=0x5574fac70e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 -- 172.21.15.78:0/3572440228 shutdown_connections 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 -- 172.21.15.78:0/3572440228 wait complete. 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.481+0000 7fb137164740 1 -- start start 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 1 Processor -- start 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac7f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac86500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5574fac7d190 con 0x5574fa8ab550 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5574fab745d0 con 0x5574fa8d4410 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574fac7d310 con 0x5574fa8b3730 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb137164740 10 client.0 _ll_register_callbacks cb 0x5574fab62080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac7f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac7f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37666/0 (socket says 172.21.15.78:37666) 2024-04-25T14:01:59.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 -- 172.21.15.78:0/2552884364 learned_addr learned my addr 172.21.15.78:0/2552884364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12d772640 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12e774640 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac86500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 -- 172.21.15.78:0/2552884364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 msgr2=0x5574fac7f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 -- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 msgr2=0x5574fac86500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac86500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.482+0000 7fb12df73640 1 -- 172.21.15.78:0/2552884364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574faa14f70 con 0x5574fa8ab550 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb12df73640 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac7f060 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb124004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633e85d2ae8f1464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb1117fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb1117fa640 1 -- 172.21.15.78:0/2552884364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124052030 con 0x5574fa8ab550 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb1117fa640 1 -- 172.21.15.78:0/2552884364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb124003970 con 0x5574fa8ab550 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124036030 con 0x5574fa8ab550 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb1117fa640 1 -- 172.21.15.78:0/2552884364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb12404a040 con 0x5574fa8ab550 2024-04-25T14:01:59.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb137164740 10 client.16194 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5574fa8ac170 con 0x5574fa8ab550 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb1117fa640 1 -- 172.21.15.78:0/2552884364 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb124036030 con 0x5574fa8ab550 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb1117fa640 1 -- 172.21.15.78:0/2552884364 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb12404d030 con 0x5574fa8ab550 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb137164740 10 client.16194 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb137164740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.483+0000 7fb137164740 20 client.16194 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.484+0000 7fb137164740 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa8931f0 0x5574fac8c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.484+0000 7fb137164740 4 client.16194 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.484+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574fabe8710 con 0x5574fa8931f0 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.484+0000 7fb12d772640 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa8931f0 0x5574fac8c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.484+0000 7fb1117fa640 10 client.16194 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.484+0000 7fb12d772640 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa8931f0 0x5574fac8c620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975a85e712b25091 server_cookie=a95855ca2771e549 in_seq=0 out_seq=0 2024-04-25T14:01:59.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb1117fa640 1 -- 172.21.15.78:0/2552884364 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574fabe8710 con 0x5574fa8931f0 2024-04-25T14:01:59.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb1117fa640 10 client.16194 handle_command_reply: tid=0 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 client.16194 shutdown 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 msgr2=0x5574fac7f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac7f060 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb124004620 comp rx=0 tx=0).stop 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 shutdown_connections 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8d4410 0x5574fac86500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa8931f0 0x5574fac8c620 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8b3730 0x5574fac7f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 --2- 172.21.15.78:0/2552884364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac7f060 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb1117fa640 0 client.16194 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 >> 172.21.15.78:0/2552884364 conn(0x5574fa8c8560 msgr2=0x5574fac6fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 shutdown_connections 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 1 -- 172.21.15.78:0/2552884364 wait complete. 2024-04-25T14:01:59.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.485+0000 7fb137164740 20 client.16194 trim_cache size 0 max 16384 2024-04-25T14:01:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 Processor -- start 2024-04-25T14:01:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 -- start start 2024-04-25T14:01:59.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8ab550 0x5574fac74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac91e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8d4410 0x5574fac92360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281adba0 con 0x5574fa8d4410 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281ab470 con 0x5574fa8b3730 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb137164740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281a92c0 con 0x5574fa8ab550 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8d4410 0x5574fac92360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8d4410 0x5574fac92360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37670/0 (socket says 172.21.15.78:37670) 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 -- 172.21.15.78:0/257840897 learned_addr learned my addr 172.21.15.78:0/257840897 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12d772640 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac91e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12df73640 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8ab550 0x5574fac74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 -- 172.21.15.78:0/257840897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8ab550 msgr2=0x5574fac74340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8ab550 0x5574fac74340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 -- 172.21.15.78:0/257840897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 msgr2=0x5574fac91e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac91e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12e774640 1 -- 172.21.15.78:0/257840897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574fac78f50 con 0x5574fa8d4410 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.504+0000 7fb12d772640 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac91e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb12e774640 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8d4410 0x5574fac92360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb12001a440 tx=0x7fb120015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d477335a69c5fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb1137fe640 1 -- 172.21.15.78:0/257840897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200263f0 con 0x5574fa8d4410 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb1137fe640 1 -- 172.21.15.78:0/257840897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12002b070 con 0x5574fa8d4410 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb1137fe640 1 -- 172.21.15.78:0/257840897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12000e610 con 0x5574fa8d4410 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb137164740 1 -- 172.21.15.78:0/257840897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8d4410 msgr2=0x5574fac92360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb137164740 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8d4410 0x5574fac92360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb12001a440 tx=0x7fb120015000 comp rx=0 tx=0).stop 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb137164740 1 -- 172.21.15.78:0/257840897 shutdown_connections 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb137164740 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8d4410 0x5574fac92360 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb137164740 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac91e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb137164740 1 --2- 172.21.15.78:0/257840897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8ab550 0x5574fac74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.505+0000 7fb137164740 1 -- 172.21.15.78:0/257840897 >> 172.21.15.78:0/257840897 conn(0x5574fa8c8560 msgr2=0x5574fac73c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 -- 172.21.15.78:0/257840897 shutdown_connections 2024-04-25T14:01:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 -- 172.21.15.78:0/257840897 wait complete. 2024-04-25T14:01:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 -- start start 2024-04-25T14:01:59.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 Processor -- start 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac84280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8d4410 0x5574fac856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281a9930 con 0x5574fa8ab550 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281aada0 con 0x5574fa8b3730 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281aa000 con 0x5574fa8d4410 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb137164740 10 client.0 _ll_register_callbacks cb 0x7fb124031870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb12df73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac84280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb12df73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac84280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37674/0 (socket says 172.21.15.78:37674) 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.506+0000 7fb12df73640 1 -- 172.21.15.78:0/1018480343 learned_addr learned my addr 172.21.15.78:0/1018480343 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12e774640 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8d4410 0x5574fac856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12d772640 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12df73640 1 -- 172.21.15.78:0/1018480343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8d4410 msgr2=0x5574fac856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12df73640 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8d4410 0x5574fac856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12df73640 1 -- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 msgr2=0x5574fac847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12df73640 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12df73640 1 -- 172.21.15.78:0/1018480343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574fac85cb0 con 0x5574fa8ab550 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb12df73640 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac84280 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb12402f500 tx=0x7fb1240094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a5dddace79ea95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb1117fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb1117fa640 1 -- 172.21.15.78:0/1018480343 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124052020 con 0x5574fa8ab550 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb1117fa640 1 -- 172.21.15.78:0/1018480343 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb124003970 con 0x5574fa8ab550 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12002b070 con 0x5574fa8ab550 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb1117fa640 1 -- 172.21.15.78:0/1018480343 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb124067050 con 0x5574fa8ab550 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb137164740 10 client.16200 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5574fac86fe0 con 0x5574fa8ab550 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.507+0000 7fb1117fa640 1 -- 172.21.15.78:0/1018480343 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12002b070 con 0x5574fa8ab550 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb1117fa640 1 -- 172.21.15.78:0/1018480343 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb12405f020 con 0x5574fa8ab550 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb137164740 10 client.16200 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb137164740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb137164740 20 client.16200 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb137164740 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa868630 0x5574facf71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb137164740 4 client.16200 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5574fabe2580 con 0x5574fa868630 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb12d772640 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa868630 0x5574facf71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb1117fa640 10 client.16200 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.508+0000 7fb12d772640 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa868630 0x5574facf71b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f65386c9f23e16e server_cookie=bbbe64ec32b0ffa2 in_seq=0 out_seq=0 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.509+0000 7fb1117fa640 1 -- 172.21.15.78:0/1018480343 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5574fabe2580 con 0x5574fa868630 2024-04-25T14:01:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.509+0000 7fb1117fa640 10 client.16200 handle_command_reply: tid=0 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.509+0000 7fb137164740 1 client.16200 shutdown 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.509+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 msgr2=0x5574fac84280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.509+0000 7fb137164740 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac84280 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb12402f500 tx=0x7fb1240094a0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 shutdown_connections 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5574fa868630 0x5574facf71b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574fa8d4410 0x5574fac856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574fa8b3730 0x5574fac847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1117fa640 0 client.16200 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 --2- 172.21.15.78:0/1018480343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574fa8ab550 0x5574fac84280 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 >> 172.21.15.78:0/1018480343 conn(0x5574fa8c8560 msgr2=0x5574fac74660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 shutdown_connections 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 1 -- 172.21.15.78:0/1018480343 wait complete. 2024-04-25T14:01:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb137164740 20 client.16200 trim_cache size 0 max 16384 2024-04-25T14:01:59.516 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:01:59.516 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:01:59.516 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "548a40c7-07b8-46fd-b093-f83196da3f64", 2024-04-25T14:01:59.516 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:01:59.516 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1140afaf0 msgr2=0x7fb1140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1140afaf0 0x7fb1140b2000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5574fac76670 tx=0x7fb1200046c0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128104d30 msgr2=0x7fb12819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128104d30 0x7fb12819a7f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 shutdown_connections 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb128109a30 0x7fb1281a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb1140afaf0 0x7fb1140b2000 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb128104d30 0x7fb12819a7f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 --2- 172.21.15.78:0/1693280903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128103af0 0x7fb12819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 >> 172.21.15.78:0/1693280903 conn(0x7fb1280fece0 msgr2=0x7fb128105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 shutdown_connections 2024-04-25T14:01:59.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.510+0000 7fb1301d7640 1 -- 172.21.15.78:0/1693280903 wait complete. 2024-04-25T14:01:59.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:01:59.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.631+0000 7ff9f916c640 1 -- 172.21.15.78:0/2418006482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f4103c30 msgr2=0x7ff9f4104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.631+0000 7ff9f916c640 1 --2- 172.21.15.78:0/2418006482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f4103c30 0x7ff9f4104040 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009900 tx=0x7ff9dc02a410 comp rx=0 tx=0).stop 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 -- 172.21.15.78:0/2418006482 shutdown_connections 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 --2- 172.21.15.78:0/2418006482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9f4108930 0x7ff9f410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 --2- 172.21.15.78:0/2418006482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f4103c30 0x7ff9f4104040 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 --2- 172.21.15.78:0/2418006482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f41029f0 0x7ff9f41036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 -- 172.21.15.78:0/2418006482 >> 172.21.15.78:0/2418006482 conn(0x7ff9f40fdbc0 msgr2=0x7ff9f4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 -- 172.21.15.78:0/2418006482 shutdown_connections 2024-04-25T14:01:59.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 -- 172.21.15.78:0/2418006482 wait complete. 2024-04-25T14:01:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.632+0000 7ff9f916c640 1 Processor -- start 2024-04-25T14:01:59.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f916c640 1 -- start start 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f916c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f41029f0 0x7ff9f419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f916c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9f4103c30 0x7ff9f419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f916c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f4108930 0x7ff9f41a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f916c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f410e520 con 0x7ff9f41029f0 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f916c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f410e3a0 con 0x7ff9f4108930 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f916c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f410e6a0 con 0x7ff9f4103c30 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f41029f0 0x7ff9f419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f41029f0 0x7ff9f419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37706/0 (socket says 172.21.15.78:37706) 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3997661864 learned_addr learned my addr 172.21.15.78:0/3997661864 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f4108930 0x7ff9f41a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9f4103c30 0x7ff9f419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3997661864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9f4103c30 msgr2=0x7ff9f419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9f4103c30 0x7ff9f419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3997661864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f4108930 msgr2=0x7ff9f41a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f4108930 0x7ff9f41a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3997661864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f41a1d50 con 0x7ff9f41029f0 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f4108930 0x7ff9f41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9f4103c30 0x7ff9f419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f41029f0 0x7ff9f419a210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400e9f0 tx=0x7ff9e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.633+0000 7ff9f0d99640 1 -- 172.21.15.78:0/3997661864 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400ccc0 con 0x7ff9f41029f0 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.634+0000 7ff9f0d99640 1 -- 172.21.15.78:0/3997661864 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e40035f0 con 0x7ff9f41029f0 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.634+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f41a1fe0 con 0x7ff9f41029f0 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.634+0000 7ff9f0d99640 1 -- 172.21.15.78:0/3997661864 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400ce60 con 0x7ff9f41029f0 2024-04-25T14:01:59.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.634+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f41a24a0 con 0x7ff9f41029f0 2024-04-25T14:01:59.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.636+0000 7ff9f0d99640 1 -- 172.21.15.78:0/3997661864 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff9e4010690 con 0x7ff9f41029f0 2024-04-25T14:01:59.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.636+0000 7ffa00017740 1 Processor -- start 2024-04-25T14:01:59.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.636+0000 7ffa00017740 1 -- start start 2024-04-25T14:01:59.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ffa00017740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d0f9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ffa00017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d0fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ffa00017740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d0fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ffa00017740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56262d053d80 con 0x56262cd4d390 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ffa00017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56262d00d8b0 con 0x56262cd244d0 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ffa00017740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56262d04fd40 con 0x56262cd2c6b0 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f0d99640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9e00afaf0 0x7ff9e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d0f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d0f9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37716/0 (socket says 172.21.15.78:37716) 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 -- 172.21.15.78:0/650867626 learned_addr learned my addr 172.21.15.78:0/650867626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9e00afaf0 0x7ff9e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d0fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f0d99640 1 -- 172.21.15.78:0/3997661864 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff9e40defe0 con 0x7ff9f41029f0 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d0fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 -- 172.21.15.78:0/650867626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 msgr2=0x56262d0fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d0fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 -- 172.21.15.78:0/650867626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 msgr2=0x56262d0fbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d0fbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 -- 172.21.15.78:0/650867626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262cfdea40 con 0x56262cd4d390 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d0f9950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec014e40 tx=0x7ff9ec011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d908c5353976d114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.637+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9e00afaf0 0x7ff9e00b2000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009fd0 tx=0x7ff9ec016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.638+0000 7ff9e959a640 1 -- 172.21.15.78:0/650867626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00cc00 con 0x56262cd4d390 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.638+0000 7ff9e959a640 1 -- 172.21.15.78:0/650867626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ec00cda0 con 0x56262cd4d390 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.638+0000 7ff9e959a640 1 -- 172.21.15.78:0/650867626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec005260 con 0x56262cd4d390 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.638+0000 7ffa00017740 1 -- 172.21.15.78:0/650867626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 msgr2=0x56262d0f9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.638+0000 7ffa00017740 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d0f9950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec014e40 tx=0x7ff9ec011750 comp rx=0 tx=0).stop 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- 172.21.15.78:0/650867626 shutdown_connections 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d0fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d0fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 --2- 172.21.15.78:0/650867626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d0f9950 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- 172.21.15.78:0/650867626 >> 172.21.15.78:0/650867626 conn(0x56262cd414e0 msgr2=0x56262d0ea1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- 172.21.15.78:0/650867626 shutdown_connections 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- 172.21.15.78:0/650867626 wait complete. 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- start start 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 Processor -- start 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d1010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d101610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d108570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56262d0ff150 con 0x56262cd4d390 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56262cfdad10 con 0x56262cd244d0 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56262d0ff2d0 con 0x56262cd2c6b0 2024-04-25T14:01:59.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ffa00017740 10 client.0 _ll_register_callbacks cb 0x56262d007210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ff9f359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d108570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ff9f359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d108570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37722/0 (socket says 172.21.15.78:37722) 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.639+0000 7ff9f359e640 1 -- 172.21.15.78:0/3645062702 learned_addr learned my addr 172.21.15.78:0/3645062702 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d1010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9f359e640 1 -- 172.21.15.78:0/3645062702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 msgr2=0x56262d101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9f359e640 1 -- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 msgr2=0x56262d1010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d1010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9f359e640 1 -- 172.21.15.78:0/3645062702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262ce2f7d0 con 0x56262cd4d390 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d108570 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0094c0 tx=0x7ff9ec014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d7db0ffb83776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9da7fc640 1 -- 172.21.15.78:0/3645062702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec035060 con 0x56262cd4d390 2024-04-25T14:01:59.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9da7fc640 1 -- 172.21.15.78:0/3645062702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff9ec004bc0 con 0x56262cd4d390 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ec00cda0 con 0x56262cd4d390 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9da7fc640 1 -- 172.21.15.78:0/3645062702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff9ec004dc0 con 0x56262cd4d390 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ffa00017740 10 client.16212 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56262cd653f0 con 0x56262cd4d390 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.640+0000 7ff9da7fc640 1 -- 172.21.15.78:0/3645062702 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ec00cda0 con 0x56262cd4d390 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ff9da7fc640 1 -- 172.21.15.78:0/3645062702 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff9ec02f030 con 0x56262cd4d390 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ffa00017740 10 client.16212 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ffa00017740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ffa00017740 20 client.16212 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ffa00017740 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cd0bf60 0x56262d10e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ffa00017740 4 client.16212 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56262d061990 con 0x56262cd0bf60 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cd0bf60 0x56262d10e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ff9da7fc640 10 client.16212 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.641+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cd0bf60 0x56262d10e8e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa818c72b47af7b server_cookie=b5fcd946fe48eb63 in_seq=0 out_seq=0 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ff9da7fc640 1 -- 172.21.15.78:0/3645062702 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56262d061990 con 0x56262cd0bf60 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ff9da7fc640 10 client.16212 handle_command_reply: tid=0 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 client.16212 shutdown 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 msgr2=0x56262d108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d108570 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0094c0 tx=0x7ff9ec014070 comp rx=0 tx=0).stop 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 shutdown_connections 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd4d390 0x56262d108570 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cd0bf60 0x56262d10e8e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 --2- 172.21.15.78:0/3645062702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 >> 172.21.15.78:0/3645062702 conn(0x56262cd414e0 msgr2=0x56262d0e89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ff9da7fc640 0 client.16212 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 shutdown_connections 2024-04-25T14:01:59.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 1 -- 172.21.15.78:0/3645062702 wait complete. 2024-04-25T14:01:59.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.642+0000 7ffa00017740 20 client.16212 trim_cache size 0 max 16384 2024-04-25T14:01:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 Processor -- start 2024-04-25T14:01:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 -- start start 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd244d0 0x56262d0ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d1010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd4d390 0x56262d0ea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41aba70 con 0x56262cd244d0 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41a9340 con 0x56262cd4d390 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ffa00017740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41a7190 con 0x56262cd2c6b0 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ff9f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd244d0 0x56262d0ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ff9f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd244d0 0x56262d0ffce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37724/0 (socket says 172.21.15.78:37724) 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.660+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3982718332 learned_addr learned my addr 172.21.15.78:0/3982718332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d1010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd4d390 0x56262d0ea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3982718332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 msgr2=0x56262d1010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d1010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3982718332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd4d390 msgr2=0x56262d0ea050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd4d390 0x56262d0ea050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f2d9d640 1 -- 172.21.15.78:0/3982718332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262cd250f0 con 0x56262cd244d0 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f359e640 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd4d390 0x56262d0ea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd244d0 0x56262d0ffce0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40a7250 tx=0x7ff9e40aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a397cd83f819f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9e8d99640 1 -- 172.21.15.78:0/3982718332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e40a7550 con 0x56262cd244d0 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9e8d99640 1 -- 172.21.15.78:0/3982718332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e40ab900 con 0x56262cd244d0 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.661+0000 7ff9e8d99640 1 -- 172.21.15.78:0/3982718332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4003aa0 con 0x56262cd244d0 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- 172.21.15.78:0/3982718332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd244d0 msgr2=0x56262d0ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd244d0 0x56262d0ffce0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40a7250 tx=0x7ff9e40aa040 comp rx=0 tx=0).stop 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- 172.21.15.78:0/3982718332 shutdown_connections 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd4d390 0x56262d0ea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd2c6b0 0x56262d1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 --2- 172.21.15.78:0/3982718332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd244d0 0x56262d0ffce0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- 172.21.15.78:0/3982718332 >> 172.21.15.78:0/3982718332 conn(0x56262cd414e0 msgr2=0x56262d0e9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- 172.21.15.78:0/3982718332 shutdown_connections 2024-04-25T14:01:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- 172.21.15.78:0/3982718332 wait complete. 2024-04-25T14:01:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- start start 2024-04-25T14:01:59.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 Processor -- start 2024-04-25T14:01:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd2c6b0 0x56262d105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd4d390 0x56262d174c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41a7800 con 0x56262cd2c6b0 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41a8c70 con 0x56262cd244d0 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.662+0000 7ffa00017740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41a7ed0 con 0x56262cd4d390 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ffa00017740 10 client.0 _ll_register_callbacks cb 0x7ff9e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd2c6b0 0x56262d105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd2c6b0 0x56262d105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37738/0 (socket says 172.21.15.78:37738) 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 -- 172.21.15.78:0/4094976636 learned_addr learned my addr 172.21.15.78:0/4094976636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9f359e640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd4d390 0x56262d174c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 -- 172.21.15.78:0/4094976636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd4d390 msgr2=0x56262d174c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd4d390 0x56262d174c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 -- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 msgr2=0x56262d105970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d105970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 -- 172.21.15.78:0/4094976636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262d0144b0 con 0x56262cd2c6b0 2024-04-25T14:01:59.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9ea59c640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd2c6b0 0x56262d105eb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009900 tx=0x7ff9dc032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee8faf1a7c4ce80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9da7fc640 1 -- 172.21.15.78:0/4094976636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc035040 con 0x56262cd2c6b0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9da7fc640 1 -- 172.21.15.78:0/4094976636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff9dc004240 con 0x56262cd2c6b0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e40ab900 con 0x56262cd2c6b0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ff9da7fc640 1 -- 172.21.15.78:0/4094976636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff9dc004440 con 0x56262cd2c6b0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ffa00017740 10 client.16218 fetch_fsmap learned FSMap version 22 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.663+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56262d175e50 con 0x56262cd2c6b0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ff9da7fc640 1 -- 172.21.15.78:0/4094976636 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e40ab900 con 0x56262cd2c6b0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ff9da7fc640 1 -- 172.21.15.78:0/4094976636 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff9dc004b10 con 0x56262cd2c6b0 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ffa00017740 10 client.16218 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ffa00017740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:01:59.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ffa00017740 20 client.16218 populate_metadata read hostname 'smithi078' 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ffa00017740 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cce13a0 0x56262d1788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ffa00017740 4 client.16218 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56262d05b800 con 0x56262cce13a0 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cce13a0 0x56262d1788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ff9da7fc640 10 client.16218 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ff9f2d9d640 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cce13a0 0x56262d1788a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761620cdf530a6eb server_cookie=23725cca448ad80c in_seq=0 out_seq=0 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ff9da7fc640 1 -- 172.21.15.78:0/4094976636 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56262d05b800 con 0x56262cce13a0 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.664+0000 7ff9da7fc640 10 client.16218 handle_command_reply: tid=0 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 client.16218 shutdown 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd2c6b0 msgr2=0x56262d105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd2c6b0 0x56262d105eb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009900 tx=0x7ff9dc032f60 comp rx=0 tx=0).stop 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 shutdown_connections 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56262cce13a0 0x56262d1788a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56262cd4d390 0x56262d174c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9da7fc640 0 client.16218 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56262cd2c6b0 0x56262d105eb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 --2- 172.21.15.78:0/4094976636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262cd244d0 0x56262d105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 >> 172.21.15.78:0/4094976636 conn(0x56262cd414e0 msgr2=0x56262d0f6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 shutdown_connections 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 1 -- 172.21.15.78:0/4094976636 wait complete. 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ffa00017740 20 client.16218 trim_cache size 0 max 16384 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:01:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:01:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "548a40c7-07b8-46fd-b093-f83196da3f64": { 2024-04-25T14:01:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:01:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "548a40c7-07b8-46fd-b093-f83196da3f64", 2024-04-25T14:01:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:01:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:01:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:01:59.672 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9e00afaf0 msgr2=0x7ff9e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9e00afaf0 0x7ff9e00b2000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009fd0 tx=0x7ff9ec016000 comp rx=0 tx=0).stop 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f41029f0 msgr2=0x7ff9f419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f41029f0 0x7ff9f419a210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400e9f0 tx=0x7ff9e400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 shutdown_connections 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9f4108930 0x7ff9f41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9f4103c30 0x7ff9f419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff9e00afaf0 0x7ff9e00b2000 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 --2- 172.21.15.78:0/3997661864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9f41029f0 0x7ff9f419a210 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:01:59.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 >> 172.21.15.78:0/3997661864 conn(0x7ff9f40fdbc0 msgr2=0x7ff9f4109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:01:59.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 shutdown_connections 2024-04-25T14:01:59.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:01:59.665+0000 7ff9f916c640 1 -- 172.21.15.78:0/3997661864 wait complete. 2024-04-25T14:01:59.676 INFO:tasks.cephfs.filesystem:scrub status for tag:548a40c7-07b8-46fd-b093-f83196da3f64 - {'path': '/', 'tag': '548a40c7-07b8-46fd-b093-f83196da3f64', 'options': 'recursive,force'} 2024-04-25T14:02:00.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:59 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:02:00.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:59 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:00.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:59 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:02:00.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:01:59 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:00.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:59 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:02:00.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:59 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:00.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:59 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:02:00.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:01:59 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:59 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:02:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:59 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:59 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:02:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:01:59 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:01 smithi112 ceph-mon[32617]: pgmap v423: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:01 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:02:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:01 smithi129 ceph-mon[32569]: pgmap v423: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:01 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:02:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:01 smithi078 ceph-mon[29163]: pgmap v423: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:01 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:02:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:02 smithi129 ceph-mon[32569]: pgmap v424: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:02 smithi078 ceph-mon[29163]: pgmap v424: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:02.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:02 smithi112 ceph-mon[32617]: pgmap v424: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:04 smithi129 ceph-mon[32569]: pgmap v425: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:04 smithi078 ceph-mon[29163]: pgmap v425: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:04.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:04 smithi112 ceph-mon[32617]: pgmap v425: 97 pgs: 97 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:06 smithi129 ceph-mon[32569]: pgmap v426: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:02:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:06 smithi078 ceph-mon[29163]: pgmap v426: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:02:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:06 smithi112 ceph-mon[32617]: pgmap v426: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 -- 172.21.15.78:0/3764969256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4106f70 msgr2=0x7f4fb4109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 --2- 172.21.15.78:0/3764969256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4106f70 0x7f4fb4109390 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009900 tx=0x7f4fa802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 -- 172.21.15.78:0/3764969256 shutdown_connections 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 --2- 172.21.15.78:0/3764969256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fb41098d0 0x7f4fb410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 --2- 172.21.15.78:0/3764969256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4106f70 0x7f4fb4109390 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 --2- 172.21.15.78:0/3764969256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb4104610 0x7f4fb4106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 -- 172.21.15.78:0/3764969256 >> 172.21.15.78:0/3764969256 conn(0x7f4fb40fdc10 msgr2=0x7f4fb41000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 -- 172.21.15.78:0/3764969256 shutdown_connections 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.499+0000 7f4fbb251640 1 -- 172.21.15.78:0/3764969256 wait complete. 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 Processor -- start 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 -- start start 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4104610 0x7f4fb419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb4106f70 0x7f4fb419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fb41098d0 0x7f4fb41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb410f630 con 0x7f4fb4104610 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb410f4b0 con 0x7f4fb4106f70 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fbb251640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb410f7b0 con 0x7f4fb41098d0 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4104610 0x7f4fb419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4104610 0x7f4fb419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37410/0 (socket says 172.21.15.78:37410) 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 -- 172.21.15.78:0/1570883126 learned_addr learned my addr 172.21.15.78:0/1570883126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fb41098d0 0x7f4fb41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb4106f70 0x7f4fb419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 -- 172.21.15.78:0/1570883126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fb41098d0 msgr2=0x7f4fb41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fb41098d0 0x7f4fb41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 -- 172.21.15.78:0/1570883126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb4106f70 msgr2=0x7f4fb419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb4106f70 0x7f4fb419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.500+0000 7f4fb8fed640 1 -- 172.21.15.78:0/1570883126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb41a1da0 con 0x7f4fb4104610 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.501+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4104610 0x7f4fb419a260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400d770 tx=0x7f4fa400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.501+0000 7f4fb1ffb640 1 -- 172.21.15.78:0/1570883126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4017070 con 0x7f4fb4104610 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.501+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb41a2090 con 0x7f4fb4104610 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.501+0000 7f4fb1ffb640 1 -- 172.21.15.78:0/1570883126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa4009c30 con 0x7f4fb4104610 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.501+0000 7f4fb1ffb640 1 -- 172.21.15.78:0/1570883126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400f620 con 0x7f4fb4104610 2024-04-25T14:02:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.501+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb41a2630 con 0x7f4fb4104610 2024-04-25T14:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.503+0000 7f4fb1ffb640 1 -- 172.21.15.78:0/1570883126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4fa401a070 con 0x7f4fb4104610 2024-04-25T14:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 Processor -- start 2024-04-25T14:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 -- start start 2024-04-25T14:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618317001b0 0x561831aa0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618316d72f0 0x561831aa0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aa7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561831a06460 con 0x5618316d72f0 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5618319c0590 con 0x5618316df4d0 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561831a02420 con 0x5618317001b0 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb8fed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618316d72f0 0x561831aa0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb8fed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618316d72f0 0x561831aa0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37420/0 (socket says 172.21.15.78:37420) 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb1ffb640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4f9c0afb60 0x7f4f9c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb8fed640 1 -- 172.21.15.78:0/2067731053 learned_addr learned my addr 172.21.15.78:0/2067731053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aa7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618317001b0 0x561831aa0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4f9c0afb60 0x7f4f9c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.504+0000 7f4fb1ffb640 1 -- 172.21.15.78:0/1570883126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4fa40de730 con 0x7f4fb4104610 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb8fed640 1 -- 172.21.15.78:0/2067731053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618317001b0 msgr2=0x561831aa0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618317001b0 0x561831aa0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb8fed640 1 -- 172.21.15.78:0/2067731053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 msgr2=0x561831aa7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aa7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb8fed640 1 -- 172.21.15.78:0/2067731053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618319cd0d0 con 0x5618316d72f0 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aa7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618316d72f0 0x561831aa0ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40a2cd0 tx=0x7f4fa4010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff1bb4c4eb9a237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4f9c0afb60 0x7f4f9c0b2070 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x561831aa1b40 tx=0x7f4fac0096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4f9affd640 1 -- 172.21.15.78:0/2067731053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4021070 con 0x5618316d72f0 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4f9affd640 1 -- 172.21.15.78:0/2067731053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa40c7e50 con 0x5618316d72f0 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.505+0000 7f4f9affd640 1 -- 172.21.15.78:0/2067731053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40aaf50 con 0x5618316d72f0 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 -- 172.21.15.78:0/2067731053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618316d72f0 msgr2=0x561831aa0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618316d72f0 0x561831aa0ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40a2cd0 tx=0x7f4fa4010c00 comp rx=0 tx=0).stop 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 -- 172.21.15.78:0/2067731053 shutdown_connections 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aa7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618316d72f0 0x561831aa0ef0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/2067731053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618317001b0 0x561831aa0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 -- 172.21.15.78:0/2067731053 >> 172.21.15.78:0/2067731053 conn(0x5618316f4300 msgr2=0x561831a9c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 -- 172.21.15.78:0/2067731053 shutdown_connections 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 -- 172.21.15.78:0/2067731053 wait complete. 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 -- start start 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.506+0000 7f4fc21e3740 1 Processor -- start 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316d72f0 0x561831aaa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316df4d0 0x561831aaab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561831aa87e0 con 0x5618317001b0 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56183198db60 con 0x5618316d72f0 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561831aa8960 con 0x5618316df4d0 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fc21e3740 10 client.0 _ll_register_callbacks cb 0x5618319b22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37436/0 (socket says 172.21.15.78:37436) 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 -- 172.21.15.78:0/4158099179 learned_addr learned my addr 172.21.15.78:0/4158099179 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316d72f0 0x561831aaa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316df4d0 0x561831aaab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 -- 172.21.15.78:0/4158099179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316df4d0 msgr2=0x561831aaab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316df4d0 0x561831aaab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 -- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316d72f0 msgr2=0x561831aaa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316d72f0 0x561831aaa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb97ee640 1 -- 172.21.15.78:0/4158099179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561831718210 con 0x5618317001b0 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.507+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316d72f0 0x561831aaa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab1ab0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00bbf0 tx=0x7f4fac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176b5fe9b71ed151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4f93fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4f93fff640 1 -- 172.21.15.78:0/4158099179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac025030 con 0x5618317001b0 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4f93fff640 1 -- 172.21.15.78:0/4158099179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4fac01b070 con 0x5618317001b0 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa40c7e50 con 0x5618317001b0 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4f93fff640 1 -- 172.21.15.78:0/4158099179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4fac010d10 con 0x5618317001b0 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 10 client.16230 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5618317e4620 con 0x5618317001b0 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4f93fff640 1 -- 172.21.15.78:0/4158099179 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa40c7e50 con 0x5618317001b0 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4f93fff640 1 -- 172.21.15.78:0/4158099179 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4fac01e050 con 0x5618317001b0 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 10 client.16230 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 20 client.16230 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316bed80 0x561831ab8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 4 client.16230 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.508+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561831a14070 con 0x5618316bed80 2024-04-25T14:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.509+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316bed80 0x561831ab8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.509+0000 7f4f93fff640 10 client.16230 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.509+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316bed80 0x561831ab8320 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b35745b3806840 server_cookie=722eaf066510a8bf in_seq=0 out_seq=0 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4f93fff640 1 -- 172.21.15.78:0/4158099179 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561831a14070 con 0x5618316bed80 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4f93fff640 10 client.16230 handle_command_reply: tid=0 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 client.16230 shutdown 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 msgr2=0x561831ab1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab1ab0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00bbf0 tx=0x7f4fac004a40 comp rx=0 tx=0).stop 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 shutdown_connections 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab1ab0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316bed80 0x561831ab8320 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316df4d0 0x561831aaab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4f93fff640 0 client.16230 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/4158099179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316d72f0 0x561831aaa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 >> 172.21.15.78:0/4158099179 conn(0x5618316f4300 msgr2=0x561831a9f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 shutdown_connections 2024-04-25T14:02:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 1 -- 172.21.15.78:0/4158099179 wait complete. 2024-04-25T14:02:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.510+0000 7f4fc21e3740 20 client.16230 trim_cache size 0 max 16384 2024-04-25T14:02:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.525+0000 7f4fc21e3740 1 Processor -- start 2024-04-25T14:02:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.525+0000 7f4fc21e3740 1 -- start start 2024-04-25T14:02:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.525+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831a9c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831ab19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831a9f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41adc50 con 0x5618317001b0 2024-04-25T14:02:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41ab520 con 0x5618316df4d0 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41a9370 con 0x5618316d72f0 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831a9f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831a9f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37438/0 (socket says 172.21.15.78:37438) 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 -- 172.21.15.78:0/1814144480 learned_addr learned my addr 172.21.15.78:0/1814144480 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831a9c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831ab19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 -- 172.21.15.78:0/1814144480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 msgr2=0x561831a9c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831a9c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 -- 172.21.15.78:0/1814144480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 msgr2=0x561831ab19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831ab19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 -- 172.21.15.78:0/1814144480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561831aa54b0 con 0x5618317001b0 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831ab19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831a9f100 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005810 tx=0x7f4fac010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e692f5835a20bf83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4f9a7fc640 1 -- 172.21.15.78:0/1814144480 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac025040 con 0x5618317001b0 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4f9a7fc640 1 -- 172.21.15.78:0/1814144480 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac005bd0 con 0x5618317001b0 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.526+0000 7f4f9a7fc640 1 -- 172.21.15.78:0/1814144480 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac02ab60 con 0x5618317001b0 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 -- 172.21.15.78:0/1814144480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 msgr2=0x561831a9f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831a9f100 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005810 tx=0x7f4fac010f60 comp rx=0 tx=0).stop 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 -- 172.21.15.78:0/1814144480 shutdown_connections 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831a9f100 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831ab19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/1814144480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831a9c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 -- 172.21.15.78:0/1814144480 >> 172.21.15.78:0/1814144480 conn(0x5618316f4300 msgr2=0x561831a9bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 -- 172.21.15.78:0/1814144480 shutdown_connections 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 -- 172.21.15.78:0/1814144480 wait complete. 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 -- start start 2024-04-25T14:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.527+0000 7f4fc21e3740 1 Processor -- start 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831aaf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aaf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fc21e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41a99e0 con 0x5618317001b0 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41aae50 con 0x5618316df4d0 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fc21e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41aa0b0 con 0x5618316d72f0 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fc21e3740 10 client.0 _ll_register_callbacks cb 0x7f4fac012d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37444/0 (socket says 172.21.15.78:37444) 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 -- 172.21.15.78:0/315176760 learned_addr learned my addr 172.21.15.78:0/315176760 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831aaf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aaf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 -- 172.21.15.78:0/315176760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 msgr2=0x561831aaf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831aaf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 -- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 msgr2=0x561831aaf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aaf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 -- 172.21.15.78:0/315176760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561831ab1f10 con 0x5618317001b0 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb3fff640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aaf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4fb97ee640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab0580 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4fac037040 tx=0x7f4fac009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9958e530d125822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.528+0000 7f4f93fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4f93fff640 1 -- 172.21.15.78:0/315176760 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac044060 con 0x5618317001b0 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4f93fff640 1 -- 172.21.15.78:0/315176760 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4fac01b070 con 0x5618317001b0 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac005bd0 con 0x5618317001b0 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4f93fff640 1 -- 172.21.15.78:0/315176760 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4fac02a7e0 con 0x5618317001b0 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 10 client.16236 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561831ab1b70 con 0x5618317001b0 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4f93fff640 1 -- 172.21.15.78:0/315176760 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac005bd0 con 0x5618317001b0 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4f93fff640 1 -- 172.21.15.78:0/315176760 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4fac01e040 con 0x5618317001b0 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 10 client.16236 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 20 client.16236 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316941c0 0x561831b22d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 4 client.16236 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561831a0dee0 con 0x5618316941c0 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.529+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316941c0 0x561831b22d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.530+0000 7f4f93fff640 10 client.16236 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.530+0000 7f4fb8fed640 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316941c0 0x561831b22d10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ca57926b4d5c260 server_cookie=3ce246d563a87bf7 in_seq=0 out_seq=0 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.543+0000 7f4f93fff640 1 -- 172.21.15.78:0/315176760 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561831a0dee0 con 0x5618316941c0 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.543+0000 7f4f93fff640 10 client.16236 handle_command_reply: tid=0 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 client.16236 shutdown 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 msgr2=0x561831ab0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab0580 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4fac037040 tx=0x7f4fac009d60 comp rx=0 tx=0).stop 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 shutdown_connections 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5618317001b0 0x561831ab0580 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5618316df4d0 0x561831aaf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5618316941c0 0x561831b22d10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 --2- 172.21.15.78:0/315176760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618316d72f0 0x561831aaf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4f93fff640 0 client.16236 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 >> 172.21.15.78:0/315176760 conn(0x5618316f4300 msgr2=0x561831aa8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 shutdown_connections 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 1 -- 172.21.15.78:0/315176760 wait complete. 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.544+0000 7f4fc21e3740 20 client.16236 trim_cache size 0 max 16384 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:02:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:02:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:02:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4f9c0afb60 msgr2=0x7f4f9c0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4f9c0afb60 0x7f4f9c0b2070 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x561831aa1b40 tx=0x7f4fac0096c0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4104610 msgr2=0x7f4fb419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4104610 0x7f4fb419a260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400d770 tx=0x7f4fa400dc30 comp rx=0 tx=0).stop 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 shutdown_connections 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fb41098d0 0x7f4fb41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fb4106f70 0x7f4fb419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4f9c0afb60 0x7f4f9c0b2070 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 --2- 172.21.15.78:0/1570883126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb4104610 0x7f4fb419a260 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 >> 172.21.15.78:0/1570883126 conn(0x7f4fb40fdc10 msgr2=0x7f4fb41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 shutdown_connections 2024-04-25T14:02:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:08.545+0000 7f4fbb251640 1 -- 172.21.15.78:0/1570883126 wait complete. 2024-04-25T14:02:08.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:08 smithi129 ceph-mon[32569]: pgmap v427: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:02:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:08 smithi078 ceph-mon[29163]: pgmap v427: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:02:08.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:08 smithi112 ceph-mon[32617]: pgmap v427: 97 pgs: 97 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:02:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:10 smithi129 ceph-mon[32569]: pgmap v428: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:10 smithi078 ceph-mon[29163]: pgmap v428: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:10.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:10 smithi112 ceph-mon[32617]: pgmap v428: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:12.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:12 smithi129 ceph-mon[32569]: pgmap v429: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:02:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:12 smithi078 ceph-mon[29163]: pgmap v429: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:02:12.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:12 smithi112 ceph-mon[32617]: pgmap v429: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:02:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:14 smithi129 ceph-mon[32569]: pgmap v430: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:14 smithi078 ceph-mon[29163]: pgmap v430: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:14 smithi112 ceph-mon[32617]: pgmap v430: 97 pgs: 97 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:16.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:16 smithi129 ceph-mon[32569]: pgmap v431: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:16.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:16 smithi078 ceph-mon[29163]: pgmap v431: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:16.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:16 smithi112 ceph-mon[32617]: pgmap v431: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:18.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:18 smithi129 ceph-mon[32569]: pgmap v432: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:18.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:18 smithi078 ceph-mon[29163]: pgmap v432: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:18.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:18 smithi112 ceph-mon[32617]: pgmap v432: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:20 smithi129 ceph-mon[32569]: pgmap v433: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:02:20.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:20 smithi078 ceph-mon[29163]: pgmap v433: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:02:20.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:20 smithi112 ceph-mon[32617]: pgmap v433: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:02:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 -- 172.21.15.78:0/4201099080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8108940 msgr2=0x7f4ae810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/4201099080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8108940 0x7f4ae810ad60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009ca0 tx=0x7f4ae002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 -- 172.21.15.78:0/4201099080 shutdown_connections 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/4201099080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8108940 0x7f4ae810ad60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/4201099080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8103c40 0x7f4ae8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/4201099080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae8102a00 0x7f4ae8103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 -- 172.21.15.78:0/4201099080 >> 172.21.15.78:0/4201099080 conn(0x7f4ae80fdbd0 msgr2=0x7f4ae81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 -- 172.21.15.78:0/4201099080 shutdown_connections 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.649+0000 7f4aeef7a640 1 -- 172.21.15.78:0/4201099080 wait complete. 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 Processor -- start 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 -- start start 2024-04-25T14:02:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8102a00 0x7f4ae819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8103c40 0x7f4ae819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae8108940 0x7f4ae81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae810e530 con 0x7f4ae8102a00 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae810e3b0 con 0x7f4ae8108940 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aeef7a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae810e6b0 con 0x7f4ae8103c40 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aecd16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8102a00 0x7f4ae819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aecd16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8102a00 0x7f4ae819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44264/0 (socket says 172.21.15.78:44264) 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4aecd16640 1 -- 172.21.15.78:0/466079190 learned_addr learned my addr 172.21.15.78:0/466079190 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.650+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8103c40 0x7f4ae819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aed517640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae8108940 0x7f4ae81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aecd16640 1 -- 172.21.15.78:0/466079190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8103c40 msgr2=0x7f4ae819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aecd16640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8103c40 0x7f4ae819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aecd16640 1 -- 172.21.15.78:0/466079190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae8108940 msgr2=0x7f4ae81a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aecd16640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae8108940 0x7f4ae81a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aecd16640 1 -- 172.21.15.78:0/466079190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae81a1c90 con 0x7f4ae8102a00 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aed517640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae8108940 0x7f4ae81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aecd16640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8102a00 0x7f4ae819a250 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c970 tx=0x7f4adc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4ae5ffb640 1 -- 172.21.15.78:0/466079190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc007c70 con 0x7f4ae8102a00 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4ae5ffb640 1 -- 172.21.15.78:0/466079190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4adc007e10 con 0x7f4ae8102a00 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae81a1f80 con 0x7f4ae8102a00 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4ae5ffb640 1 -- 172.21.15.78:0/466079190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0187b0 con 0x7f4ae8102a00 2024-04-25T14:02:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.651+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae81a23e0 con 0x7f4ae8102a00 2024-04-25T14:02:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4ae5ffb640 1 -- 172.21.15.78:0/466079190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4adc00fab0 con 0x7f4ae8102a00 2024-04-25T14:02:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 Processor -- start 2024-04-25T14:02:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 -- start start 2024-04-25T14:02:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a04c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a04cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a053df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560069fb2460 con 0x560069c8b4d0 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560069f6c590 con 0x560069c832f0 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560069fae420 con 0x560069cac1b0 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a053df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4ae5ffb640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4ad80afae0 0x7f4ad80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a053df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44266/0 (socket says 172.21.15.78:44266) 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3549821792 learned_addr learned my addr 172.21.15.78:0/3549821792 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4aecd16640 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a04cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4aed517640 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a04c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.654+0000 7f4aed517640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4ad80afae0 0x7f4ad80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ae5ffb640 1 -- 172.21.15.78:0/466079190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4adc0dec50 con 0x7f4ae8102a00 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3549821792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 msgr2=0x56006a04c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a04c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3549821792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 msgr2=0x56006a04cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a04cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3549821792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560069f790d0 con 0x560069c8b4d0 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a053df0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004650 tx=0x7f4ad400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ddd28442dbbc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4aed517640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4ad80afae0 0x7f4ad80b1ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56006a04db40 tx=0x7f4ae0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ad2ffd640 1 -- 172.21.15.78:0/3549821792 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4003bd0 con 0x560069c8b4d0 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ad2ffd640 1 -- 172.21.15.78:0/3549821792 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad4015070 con 0x560069c8b4d0 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.655+0000 7f4ad2ffd640 1 -- 172.21.15.78:0/3549821792 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4004d10 con 0x560069c8b4d0 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3549821792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 msgr2=0x56006a053df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a053df0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004650 tx=0x7f4ad400bba0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3549821792 shutdown_connections 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a053df0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a04cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3549821792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a04c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3549821792 >> 172.21.15.78:0/3549821792 conn(0x560069ca0300 msgr2=0x56006a048300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3549821792 shutdown_connections 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3549821792 wait complete. 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.656+0000 7f4af5e1f740 1 -- start start 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 1 Processor -- start 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a056640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069c8b4d0 0x56006a056b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069cac1b0 0x56006a05dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56006a054840 con 0x560069cac1b0 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560069f39b60 con 0x560069c832f0 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56006a0549c0 con 0x560069c8b4d0 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4af5e1f740 10 client.0 _ll_register_callbacks cb 0x560069f5e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069cac1b0 0x56006a05dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069cac1b0 0x56006a05dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44276/0 (socket says 172.21.15.78:44276) 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 -- 172.21.15.78:0/1743426791 learned_addr learned my addr 172.21.15.78:0/1743426791 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069c8b4d0 0x56006a056b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a056640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 -- 172.21.15.78:0/1743426791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069c8b4d0 msgr2=0x56006a056b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069c8b4d0 0x56006a056b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 -- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 msgr2=0x56006a056640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a056640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aed517640 1 -- 172.21.15.78:0/1743426791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560069cc4210 con 0x560069cac1b0 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.657+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a056640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4aed517640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069cac1b0 0x56006a05dae0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0035000 tx=0x7f4ae00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209e56df7439de37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4ac7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1743426791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0050070 con 0x560069cac1b0 2024-04-25T14:02:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1743426791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4ae00045c0 con 0x560069cac1b0 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad4015070 con 0x560069cac1b0 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1743426791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4ae00047c0 con 0x560069cac1b0 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4af5e1f740 10 client.16248 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560069d90620 con 0x560069cac1b0 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1743426791 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad4015070 con 0x560069cac1b0 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1743426791 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4ae0049040 con 0x560069cac1b0 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4af5e1f740 10 client.16248 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4af5e1f740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4af5e1f740 20 client.16248 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.658+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c6ad80 0x56006a0643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.659+0000 7f4af5e1f740 4 client.16248 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.659+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560069fc0070 con 0x560069c6ad80 2024-04-25T14:02:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.659+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c6ad80 0x56006a0643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.659+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c6ad80 0x56006a0643a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23cf515970b8c52 server_cookie=9509a9f306c4dda9 in_seq=0 out_seq=0 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.659+0000 7f4ac7fff640 10 client.16248 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1743426791 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560069fc0070 con 0x560069c6ad80 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4ac7fff640 10 client.16248 handle_command_reply: tid=0 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 client.16248 shutdown 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069cac1b0 msgr2=0x56006a05dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069cac1b0 0x56006a05dae0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0035000 tx=0x7f4ae00095c0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 shutdown_connections 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069cac1b0 0x56006a05dae0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c6ad80 0x56006a0643a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069c8b4d0 0x56006a056b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1743426791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a056640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4ac7fff640 0 client.16248 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 >> 172.21.15.78:0/1743426791 conn(0x560069ca0300 msgr2=0x56006a047250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 shutdown_connections 2024-04-25T14:02:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.660+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1743426791 wait complete. 2024-04-25T14:02:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.661+0000 7f4af5e1f740 20 client.16248 trim_cache size 0 max 16384 2024-04-25T14:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 Processor -- start 2024-04-25T14:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 -- start start 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a047cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a048200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81ada50 con 0x560069c8b4d0 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81ab320 con 0x560069c832f0 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81a9170 con 0x560069cac1b0 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44284/0 (socket says 172.21.15.78:44284) 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3346924874 learned_addr learned my addr 172.21.15.78:0/3346924874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4aecd16640 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a047cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3346924874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 msgr2=0x56006a048200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a048200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3346924874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 msgr2=0x56006a047cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a047cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 -- 172.21.15.78:0/3346924874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56006a05df40 con 0x560069c8b4d0 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4aecd16640 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a047cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05da00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4018000 tx=0x7f4ad4004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed244bd81f5388b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.676+0000 7f4ad27fc640 1 -- 172.21.15.78:0/3346924874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4029070 con 0x560069c8b4d0 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4ad27fc640 1 -- 172.21.15.78:0/3346924874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad4015040 con 0x560069c8b4d0 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4ad27fc640 1 -- 172.21.15.78:0/3346924874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4019420 con 0x560069c8b4d0 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3346924874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 msgr2=0x56006a05da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05da00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4018000 tx=0x7f4ad4004120 comp rx=0 tx=0).stop 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3346924874 shutdown_connections 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a048200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05da00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/3346924874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a047cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3346924874 >> 172.21.15.78:0/3346924874 conn(0x560069ca0300 msgr2=0x56006a050ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3346924874 shutdown_connections 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.677+0000 7f4af5e1f740 1 -- 172.21.15.78:0/3346924874 wait complete. 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 -- start start 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 Processor -- start 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a05af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a05c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81a97e0 con 0x560069c8b4d0 2024-04-25T14:02:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81aac50 con 0x560069c832f0 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81a9eb0 con 0x560069cac1b0 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4af5e1f740 10 client.0 _ll_register_callbacks cb 0x7f4ae0009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44296/0 (socket says 172.21.15.78:44296) 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4ae7fff640 1 -- 172.21.15.78:0/1104934081 learned_addr learned my addr 172.21.15.78:0/1104934081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.678+0000 7f4aed517640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a05c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a05af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ae7fff640 1 -- 172.21.15.78:0/1104934081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 msgr2=0x56006a05c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a05c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ae7fff640 1 -- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 msgr2=0x56006a05af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a05af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ae7fff640 1 -- 172.21.15.78:0/1104934081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56006a05c930 con 0x560069c8b4d0 2024-04-25T14:02:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a05af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ae7fff640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05b440 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad4003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0325fca95c944b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ac7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1104934081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4032060 con 0x560069c8b4d0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1104934081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4ad4011640 con 0x560069c8b4d0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad4015040 con 0x560069c8b4d0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1104934081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4ad4029040 con 0x560069c8b4d0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4af5e1f740 10 client.16254 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56006a05dbf0 con 0x560069c8b4d0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1104934081 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad4015040 con 0x560069c8b4d0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1104934081 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4ad402c070 con 0x560069c8b4d0 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4af5e1f740 10 client.16254 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.679+0000 7f4af5e1f740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.680+0000 7f4af5e1f740 20 client.16254 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.680+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c401c0 0x56006a0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.680+0000 7f4af5e1f740 4 client.16254 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.680+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560069fb9ee0 con 0x560069c401c0 2024-04-25T14:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.680+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c401c0 0x56006a0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.680+0000 7f4aecd16640 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c401c0 0x56006a0cec20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba29eaa743345711 server_cookie=73354c7b938d0c9e in_seq=0 out_seq=0 2024-04-25T14:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.680+0000 7f4ac7fff640 10 client.16254 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4ac7fff640 1 -- 172.21.15.78:0/1104934081 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560069fb9ee0 con 0x560069c401c0 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4ac7fff640 10 client.16254 handle_command_reply: tid=0 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 client.16254 shutdown 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 msgr2=0x56006a05b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05b440 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad4003e80 comp rx=0 tx=0).stop 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 shutdown_connections 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x560069c401c0 0x56006a0cec20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560069cac1b0 0x56006a05c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4ac7fff640 0 client.16254 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560069c8b4d0 0x56006a05b440 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 --2- 172.21.15.78:0/1104934081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560069c832f0 0x56006a05af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 >> 172.21.15.78:0/1104934081 conn(0x560069ca0300 msgr2=0x56006a047ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 shutdown_connections 2024-04-25T14:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 1 -- 172.21.15.78:0/1104934081 wait complete. 2024-04-25T14:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.694+0000 7f4af5e1f740 20 client.16254 trim_cache size 0 max 16384 2024-04-25T14:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4ad80afae0 msgr2=0x7f4ad80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4ad80afae0 0x7f4ad80b1ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56006a04db40 tx=0x7f4ae0005fd0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8102a00 msgr2=0x7f4ae819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8102a00 0x7f4ae819a250 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c970 tx=0x7f4adc00ce30 comp rx=0 tx=0).stop 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 shutdown_connections 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ae8108940 0x7f4ae81a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8103c40 0x7f4ae819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4ad80afae0 0x7f4ad80b1ff0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 --2- 172.21.15.78:0/466079190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae8102a00 0x7f4ae819a250 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 >> 172.21.15.78:0/466079190 conn(0x7f4ae80fdbd0 msgr2=0x7f4ae8109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 shutdown_connections 2024-04-25T14:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:21.695+0000 7f4aeef7a640 1 -- 172.21.15.78:0/466079190 wait complete. 2024-04-25T14:02:22.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:22 smithi112 ceph-mon[32617]: pgmap v434: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:22 smithi129 ceph-mon[32569]: pgmap v434: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:22 smithi078 ceph-mon[29163]: pgmap v434: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:24 smithi129 ceph-mon[32569]: pgmap v435: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:02:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:24 smithi078 ceph-mon[29163]: pgmap v435: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:02:24.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:24 smithi112 ceph-mon[32617]: pgmap v435: 97 pgs: 97 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:02:26.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:26 smithi129 ceph-mon[32569]: pgmap v436: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:26 smithi078 ceph-mon[29163]: pgmap v436: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:26.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:26 smithi112 ceph-mon[32617]: pgmap v436: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:28 smithi129 ceph-mon[32569]: pgmap v437: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:28.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:28 smithi078 ceph-mon[29163]: pgmap v437: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:28.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:28 smithi112 ceph-mon[32617]: pgmap v437: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:29.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:02:29.679 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:02:29.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 -- 172.21.15.78:0/3252357077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 msgr2=0x7f101810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 --2- 172.21.15.78:0/3252357077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f101810ad40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1010009ca0 tx=0x7f101002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 -- 172.21.15.78:0/3252357077 shutdown_connections 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 --2- 172.21.15.78:0/3252357077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f101810ad40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 --2- 172.21.15.78:0/3252357077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1018103c20 0x7f1018104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 --2- 172.21.15.78:0/3252357077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10181029e0 0x7f10181036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 -- 172.21.15.78:0/3252357077 >> 172.21.15.78:0/3252357077 conn(0x7f10180fdbd0 msgr2=0x7f1018100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 -- 172.21.15.78:0/3252357077 shutdown_connections 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.802+0000 7f101fb14640 1 -- 172.21.15.78:0/3252357077 wait complete. 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 Processor -- start 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 -- start start 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10181029e0 0x7f101806a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1018103c20 0x7f101819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f10181a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f101810e510 con 0x7f1018108920 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f101810e390 con 0x7f10181029e0 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101fb14640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f101810e690 con 0x7f1018103c20 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f10181a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f10181a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57176/0 (socket says 172.21.15.78:57176) 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 -- 172.21.15.78:0/3754280196 learned_addr learned my addr 172.21.15.78:0/3754280196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101d0af640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1018103c20 0x7f101819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101d8b0640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10181029e0 0x7f101806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 -- 172.21.15.78:0/3754280196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1018103c20 msgr2=0x7f101819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1018103c20 0x7f101819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 -- 172.21.15.78:0/3754280196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10181029e0 msgr2=0x7f101806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10181029e0 0x7f101806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 -- 172.21.15.78:0/3754280196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10181a2030 con 0x7f1018108920 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.803+0000 7f101e0b1640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f10181a1810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f10100097b0 tx=0x7f1010003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.804+0000 7f100effd640 1 -- 172.21.15.78:0/3754280196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010004240 con 0x7f1018108920 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.804+0000 7f100effd640 1 -- 172.21.15.78:0/3754280196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f101002f380 con 0x7f1018108920 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.804+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10181a22c0 con 0x7f1018108920 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.804+0000 7f100effd640 1 -- 172.21.15.78:0/3754280196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101002fc30 con 0x7f1018108920 2024-04-25T14:02:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.804+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10181a2860 con 0x7f1018108920 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.806+0000 7f100effd640 1 -- 172.21.15.78:0/3754280196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f10100043e0 con 0x7f1018108920 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.806+0000 7f10269c8740 1 Processor -- start 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.806+0000 7f10269c8740 1 -- start start 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f10269c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f10269c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 0x55bdc4df1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f10269c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f10269c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdc4d56d80 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f10269c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bdc4d108b0 con 0x55bdc4a274d0 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f10269c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bdc4d52d40 con 0x55bdc4a50390 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57178/0 (socket says 172.21.15.78:57178) 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 -- 172.21.15.78:0/2225316820 learned_addr learned my addr 172.21.15.78:0/2225316820 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d8b0640 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 0x55bdc4df1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f100effd640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10040afb70 0x7f10040b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101e0b1640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10040afb70 0x7f10040b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 -- 172.21.15.78:0/2225316820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 msgr2=0x55bdc4df1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 -- 172.21.15.78:0/2225316820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 msgr2=0x55bdc4df1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 0x55bdc4df1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 -- 172.21.15.78:0/2225316820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc4ce1a40 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f100effd640 1 -- 172.21.15.78:0/3754280196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1010100370 con 0x7f1018108920 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.807+0000 7f101d0af640 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df2c70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x55bdc4df2ab0 tx=0x7f100800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf84c2c4ec32682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.808+0000 7f1003fff640 1 -- 172.21.15.78:0/2225316820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008010740 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.808+0000 7f101e0b1640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10040afb70 0x7f10040b2080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10100c8540 tx=0x7f101008f0c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.808+0000 7f1003fff640 1 -- 172.21.15.78:0/2225316820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008010d90 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.808+0000 7f1003fff640 1 -- 172.21.15.78:0/2225316820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100801fc30 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 -- 172.21.15.78:0/2225316820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 msgr2=0x55bdc4df2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df2c70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x55bdc4df2ab0 tx=0x7f100800b870 comp rx=0 tx=0).stop 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 -- 172.21.15.78:0/2225316820 shutdown_connections 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df2c70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 0x55bdc4df1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 --2- 172.21.15.78:0/2225316820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 -- 172.21.15.78:0/2225316820 >> 172.21.15.78:0/2225316820 conn(0x55bdc4a444e0 msgr2=0x55bdc4ded270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 -- 172.21.15.78:0/2225316820 shutdown_connections 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 -- 172.21.15.78:0/2225316820 wait complete. 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 -- start start 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.809+0000 7f10269c8740 1 Processor -- start 2024-04-25T14:02:29.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dfb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dfbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e02e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bdc4df3780 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bdc4cddd10 con 0x55bdc4a50390 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bdc4df3900 con 0x55bdc4a274d0 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 10 client.0 _ll_register_callbacks cb 0x55bdc4d0a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dfbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dfbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57188/0 (socket says 172.21.15.78:57188) 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 -- 172.21.15.78:0/3114856788 learned_addr learned my addr 172.21.15.78:0/3114856788 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d0af640 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dfb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 -- 172.21.15.78:0/3114856788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 msgr2=0x55bdc4dfb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d0af640 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e02e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dfb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 -- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 msgr2=0x55bdc4e02e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e02e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 -- 172.21.15.78:0/3114856788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc4b327d0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f101d8b0640 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dfbeb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f101400be40 tx=0x7f1014012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da06857b5c343854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10017fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10017fa640 1 -- 172.21.15.78:0/3114856788 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140035f0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10017fa640 1 -- 172.21.15.78:0/3114856788 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1014003790 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.810+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1008010d90 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10017fa640 1 -- 172.21.15.78:0/3114856788 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1014003990 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 10 client.16266 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bdc4a683f0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10017fa640 1 -- 172.21.15.78:0/3114856788 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008010d90 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10017fa640 1 -- 172.21.15.78:0/3114856788 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1014024050 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 10 client.16266 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 20 client.16266 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc4a0ef60 0x55bdc4e09370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 4 client.16266 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdc4d64990 con 0x55bdc4a0ef60 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.811+0000 7f101d0af640 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc4a0ef60 0x55bdc4e09370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.812+0000 7f10017fa640 10 client.16266 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.812+0000 7f101d0af640 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc4a0ef60 0x55bdc4e09370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fac5a0699e5663b server_cookie=fcb1c9ec05bbf78e in_seq=0 out_seq=0 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.812+0000 7f10017fa640 1 -- 172.21.15.78:0/3114856788 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bdc4d64990 con 0x55bdc4a0ef60 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.812+0000 7f10017fa640 10 client.16266 handle_command_reply: tid=0 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 client.16266 shutdown 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 msgr2=0x55bdc4dfbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dfbeb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f101400be40 tx=0x7f1014012820 comp rx=0 tx=0).stop 2024-04-25T14:02:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 shutdown_connections 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e02e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dfbeb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc4a0ef60 0x55bdc4e09370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 --2- 172.21.15.78:0/3114856788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dfb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 >> 172.21.15.78:0/3114856788 conn(0x55bdc4a444e0 msgr2=0x55bdc4df0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10017fa640 0 client.16266 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 shutdown_connections 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 1 -- 172.21.15.78:0/3114856788 wait complete. 2024-04-25T14:02:29.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.813+0000 7f10269c8740 20 client.16266 trim_cache size 0 max 16384 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.830+0000 7f10269c8740 1 Processor -- start 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.830+0000 7f10269c8740 1 -- start start 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.830+0000 7f10269c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 0x55bdc4df6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10269c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10269c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10269c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10181adaa0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10269c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10181ab370 con 0x55bdc4a274d0 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10269c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10181a91c0 con 0x55bdc4a50390 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57194/0 (socket says 172.21.15.78:57194) 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 -- 172.21.15.78:0/3910272320 learned_addr learned my addr 172.21.15.78:0/3910272320 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d0af640 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 -- 172.21.15.78:0/3910272320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 msgr2=0x55bdc4df0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 -- 172.21.15.78:0/3910272320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 msgr2=0x55bdc4df6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 0x55bdc4df6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 -- 172.21.15.78:0/3910272320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc4a280f0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d0af640 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f101d8b0640 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df19e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f101400b990 tx=0x7f1014005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b669206688c02b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10037fe640 1 -- 172.21.15.78:0/3910272320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101401b4c0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10037fe640 1 -- 172.21.15.78:0/3910272320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10140035f0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.831+0000 7f10037fe640 1 -- 172.21.15.78:0/3910272320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014010490 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 -- 172.21.15.78:0/3910272320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 msgr2=0x55bdc4df19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df19e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f101400b990 tx=0x7f1014005850 comp rx=0 tx=0).stop 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 -- 172.21.15.78:0/3910272320 shutdown_connections 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a50390 0x55bdc4df0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4df19e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 --2- 172.21.15.78:0/3910272320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a274d0 0x55bdc4df6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 -- 172.21.15.78:0/3910272320 >> 172.21.15.78:0/3910272320 conn(0x55bdc4a444e0 msgr2=0x55bdc4df0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 -- 172.21.15.78:0/3910272320 shutdown_connections 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 -- 172.21.15.78:0/3910272320 wait complete. 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 -- start start 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.832+0000 7f10269c8740 1 Processor -- start 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10269c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10269c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10269c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e73b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10269c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10181a9830 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10269c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10181aaca0 con 0x55bdc4a50390 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10269c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10181a9f00 con 0x55bdc4a274d0 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10269c8740 10 client.0 _ll_register_callbacks cb 0x7f101400b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57198/0 (socket says 172.21.15.78:57198) 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 -- 172.21.15.78:0/2580198987 learned_addr learned my addr 172.21.15.78:0/2580198987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d0af640 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d0af640 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e73b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 -- 172.21.15.78:0/2580198987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 msgr2=0x55bdc4dff4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dff4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 -- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 msgr2=0x55bdc4e73b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e73b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 -- 172.21.15.78:0/2580198987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc4d174b0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f101d8b0640 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dffa20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f101400c010 tx=0x7f101400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f3724899de5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.833+0000 7f10017fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10017fa640 1 -- 172.21.15.78:0/2580198987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014003d30 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10017fa640 1 -- 172.21.15.78:0/2580198987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1014010490 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10140035f0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10017fa640 1 -- 172.21.15.78:0/2580198987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1014042050 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 10 client.16272 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bdc4e750e0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10017fa640 1 -- 172.21.15.78:0/2580198987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10140035f0 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10017fa640 1 -- 172.21.15.78:0/2580198987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1014024070 con 0x55bdc4a2f6b0 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 10 client.16272 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 20 client.16272 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc49e43a0 0x55bdc4e77af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 4 client.16272 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdc4d5e800 con 0x55bdc49e43a0 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.834+0000 7f101d0af640 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc49e43a0 0x55bdc4e77af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10017fa640 10 client.16272 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f101d0af640 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc49e43a0 0x55bdc4e77af0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d622bd01acd7019 server_cookie=faa1272569a1548f in_seq=0 out_seq=0 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10017fa640 1 -- 172.21.15.78:0/2580198987 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bdc4d5e800 con 0x55bdc49e43a0 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10017fa640 10 client.16272 handle_command_reply: tid=0 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 client.16272 shutdown 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 msgr2=0x55bdc4dffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dffa20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f101400c010 tx=0x7f101400b9c0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 shutdown_connections 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bdc4a50390 0x55bdc4e73b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bdc4a2f6b0 0x55bdc4dffa20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bdc49e43a0 0x55bdc4e77af0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 --2- 172.21.15.78:0/2580198987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bdc4a274d0 0x55bdc4dff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10017fa640 0 client.16272 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 >> 172.21.15.78:0/2580198987 conn(0x55bdc4a444e0 msgr2=0x55bdc4df0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 shutdown_connections 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 1 -- 172.21.15.78:0/2580198987 wait complete. 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.835+0000 7f10269c8740 20 client.16272 trim_cache size 0 max 16384 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:02:29.842 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:02:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10040afb70 msgr2=0x7f10040b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10040afb70 0x7f10040b2080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10100c8540 tx=0x7f101008f0c0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 msgr2=0x7f10181a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f10181a1810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f10100097b0 tx=0x7f1010003c20 comp rx=0 tx=0).stop 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 shutdown_connections 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f10040afb70 0x7f10040b2080 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1018108920 0x7f10181a1810 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1018103c20 0x7f101819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 --2- 172.21.15.78:0/3754280196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10181029e0 0x7f101806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 >> 172.21.15.78:0/3754280196 conn(0x7f10180fdbd0 msgr2=0x7f1018104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 shutdown_connections 2024-04-25T14:02:29.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.836+0000 7f101fb14640 1 -- 172.21.15.78:0/3754280196 wait complete. 2024-04-25T14:02:29.846 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:02:29.846 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:02:29.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 -- 172.21.15.78:0/447891480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c1029c0 msgr2=0x7f2b0c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 --2- 172.21.15.78:0/447891480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c1029c0 0x7f2b0c1036c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009900 tx=0x7f2b0002a410 comp rx=0 tx=0).stop 2024-04-25T14:02:29.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 -- 172.21.15.78:0/447891480 shutdown_connections 2024-04-25T14:02:29.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 --2- 172.21.15.78:0/447891480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c108900 0x7f2b0c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 --2- 172.21.15.78:0/447891480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b0c103c00 0x7f2b0c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 --2- 172.21.15.78:0/447891480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c1029c0 0x7f2b0c1036c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 -- 172.21.15.78:0/447891480 >> 172.21.15.78:0/447891480 conn(0x7f2b0c0fdbd0 msgr2=0x7f2b0c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:29.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 -- 172.21.15.78:0/447891480 shutdown_connections 2024-04-25T14:02:29.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 -- 172.21.15.78:0/447891480 wait complete. 2024-04-25T14:02:29.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.955+0000 7f2b11092640 1 Processor -- start 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b11092640 1 -- start start 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b11092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c1029c0 0x7f2b0c06a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b11092640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c103c00 0x7f2b0c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b11092640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b0c108900 0x7f2b0c1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b11092640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c10e4c0 con 0x7f2b0c103c00 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b11092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c10e340 con 0x7f2b0c1029c0 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b11092640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c10e640 con 0x7f2b0c108900 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c103c00 0x7f2b0c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c103c00 0x7f2b0c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57226/0 (socket says 172.21.15.78:57226) 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 -- 172.21.15.78:0/2787873922 learned_addr learned my addr 172.21.15.78:0/2787873922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0b59e640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b0c108900 0x7f2b0c1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0ad9d640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c1029c0 0x7f2b0c06a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 -- 172.21.15.78:0/2787873922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b0c108900 msgr2=0x7f2b0c1a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b0c108900 0x7f2b0c1a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 -- 172.21.15.78:0/2787873922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c1029c0 msgr2=0x7f2b0c06a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c1029c0 0x7f2b0c06a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.956+0000 7f2b0a59c640 1 -- 172.21.15.78:0/2787873922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c1a1d50 con 0x7f2b0c103c00 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.957+0000 7f2b0a59c640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c103c00 0x7f2b0c19efe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d770 tx=0x7f2afc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.957+0000 7f2af3fff640 1 -- 172.21.15.78:0/2787873922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc017070 con 0x7f2b0c103c00 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.957+0000 7f2af3fff640 1 -- 172.21.15.78:0/2787873922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0054d0 con 0x7f2b0c103c00 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.957+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c1a1fe0 con 0x7f2b0c103c00 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.957+0000 7f2af3fff640 1 -- 172.21.15.78:0/2787873922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc004c30 con 0x7f2b0c103c00 2024-04-25T14:02:29.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.957+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1a2450 con 0x7f2b0c103c00 2024-04-25T14:02:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.958+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c107710 con 0x7f2b0c103c00 2024-04-25T14:02:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.960+0000 7f2af3fff640 1 -- 172.21.15.78:0/2787873922 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2afc004dd0 con 0x7f2b0c103c00 2024-04-25T14:02:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.961+0000 7f2af3fff640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2af40afaf0 0x7f2af40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.961+0000 7f2b0ad9d640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2af40afaf0 0x7f2af40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:29.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.961+0000 7f2af3fff640 1 -- 172.21.15.78:0/2787873922 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2afc0dea00 con 0x7f2b0c103c00 2024-04-25T14:02:29.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.961+0000 7f2b0ad9d640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2af40afaf0 0x7f2af40b2000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005ab0 tx=0x7f2b00031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:29.965+0000 7f2af3fff640 1 -- 172.21.15.78:0/2787873922 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2afc0a6f80 con 0x7f2b0c103c00 2024-04-25T14:02:30.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.182+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b0c02db50 con 0x7f2b0c103c00 2024-04-25T14:02:30.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.183+0000 7f2af3fff640 1 -- 172.21.15.78:0/2787873922 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f2afc0ab280 con 0x7f2b0c103c00 2024-04-25T14:02:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:02:30.182 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:02:30.182 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2af40afaf0 msgr2=0x7f2af40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2af40afaf0 0x7f2af40b2000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005ab0 tx=0x7f2b00031400 comp rx=0 tx=0).stop 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c103c00 msgr2=0x7f2b0c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c103c00 0x7f2b0c19efe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00d770 tx=0x7f2afc00dc30 comp rx=0 tx=0).stop 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 shutdown_connections 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b0c108900 0x7f2b0c1a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2af40afaf0 0x7f2af40b2000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b0c103c00 0x7f2b0c19efe0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 --2- 172.21.15.78:0/2787873922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c1029c0 0x7f2b0c06a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 >> 172.21.15.78:0/2787873922 conn(0x7f2b0c0fdbd0 msgr2=0x7f2b0c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 shutdown_connections 2024-04-25T14:02:30.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.185+0000 7f2b11092640 1 -- 172.21.15.78:0/2787873922 wait complete. 2024-04-25T14:02:30.195 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:02:30.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 -- 172.21.15.78:0/1117114287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0109850 msgr2=0x7f72f010bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 --2- 172.21.15.78:0/1117114287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0109850 0x7f72f010bc70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009ca0 tx=0x7f72e802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 -- 172.21.15.78:0/1117114287 shutdown_connections 2024-04-25T14:02:30.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 --2- 172.21.15.78:0/1117114287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0109850 0x7f72f010bc70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 --2- 172.21.15.78:0/1117114287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72f0104b50 0x7f72f0104f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 --2- 172.21.15.78:0/1117114287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72f0103910 0x7f72f0104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 -- 172.21.15.78:0/1117114287 >> 172.21.15.78:0/1117114287 conn(0x7f72f00feb20 msgr2=0x7f72f0100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 -- 172.21.15.78:0/1117114287 shutdown_connections 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.308+0000 7f72f739d640 1 -- 172.21.15.78:0/1117114287 wait complete. 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 Processor -- start 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 -- start start 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0103910 0x7f72f019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72f0104b50 0x7f72f019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72f0109850 0x7f72f01a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f010f440 con 0x7f72f0103910 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72f010f2c0 con 0x7f72f0104b50 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f739d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72f010f5c0 con 0x7f72f0109850 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0103910 0x7f72f019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0103910 0x7f72f019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57242/0 (socket says 172.21.15.78:57242) 2024-04-25T14:02:30.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 -- 172.21.15.78:0/528493966 learned_addr learned my addr 172.21.15.78:0/528493966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f6b9c640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72f0109850 0x7f72f01a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72f0104b50 0x7f72f019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 -- 172.21.15.78:0/528493966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72f0109850 msgr2=0x7f72f01a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72f0109850 0x7f72f01a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 -- 172.21.15.78:0/528493966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72f0104b50 msgr2=0x7f72f019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72f0104b50 0x7f72f019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f639b640 1 -- 172.21.15.78:0/528493966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f01a1b50 con 0x7f72f0103910 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.309+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72f0104b50 0x7f72f019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.310+0000 7f72f639b640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0103910 0x7f72f019a010 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e9f0 tx=0x7f72ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.310+0000 7f72e77fe640 1 -- 172.21.15.78:0/528493966 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00ccc0 con 0x7f72f0103910 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.310+0000 7f72e77fe640 1 -- 172.21.15.78:0/528493966 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ec0035f0 con 0x7f72f0103910 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.310+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f01a1e40 con 0x7f72f0103910 2024-04-25T14:02:30.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.310+0000 7f72e77fe640 1 -- 172.21.15.78:0/528493966 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00ce60 con 0x7f72f0103910 2024-04-25T14:02:30.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.310+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f01a22a0 con 0x7f72f0103910 2024-04-25T14:02:30.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 Processor -- start 2024-04-25T14:02:30.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 -- start start 2024-04-25T14:02:30.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72e77fe640 1 -- 172.21.15.78:0/528493966 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f72ec003aa0 con 0x7f72f0103910 2024-04-25T14:02:30.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c298f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c299bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dd8c1fcd80 con 0x55dd8becd4d0 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dd8c1b6a20 con 0x55dd8bed56b0 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.313+0000 7f72ff584740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dd8c1f8d40 con 0x55dd8bef6390 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c298f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c298f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57248/0 (socket says 172.21.15.78:57248) 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 -- 172.21.15.78:0/3254666024 learned_addr learned my addr 172.21.15.78:0/3254666024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f6b9c640 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c299bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 -- 172.21.15.78:0/3254666024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 msgr2=0x55dd8c2989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 -- 172.21.15.78:0/3254666024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 msgr2=0x55dd8c299bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c299bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 -- 172.21.15.78:0/3254666024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8c1c1c30 con 0x55dd8becd4d0 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72e77fe640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f72dc0afae0 0x7f72dc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f639b640 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c298f60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0e7160 tx=0x7f72ec0e2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d38113343157bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72f6b9c640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f72dc0afae0 0x7f72dc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72e77fe640 1 -- 172.21.15.78:0/528493966 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f72ec0df000 con 0x7f72f0103910 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72e4bfe640 1 -- 172.21.15.78:0/3254666024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0a3430 con 0x55dd8becd4d0 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72e4bfe640 1 -- 172.21.15.78:0/3254666024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ec0a7080 con 0x55dd8becd4d0 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.314+0000 7f72e4bfe640 1 -- 172.21.15.78:0/3254666024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0a6ce0 con 0x55dd8becd4d0 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 -- 172.21.15.78:0/3254666024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 msgr2=0x55dd8c298f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72f6b9c640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f72dc0afae0 0x7f72dc0b1ff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f72e8031b50 tx=0x7f72e8009450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c298f60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0e7160 tx=0x7f72ec0e2f90 comp rx=0 tx=0).stop 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 -- 172.21.15.78:0/3254666024 shutdown_connections 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c299bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c298f60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 --2- 172.21.15.78:0/3254666024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 -- 172.21.15.78:0/3254666024 >> 172.21.15.78:0/3254666024 conn(0x55dd8beea4e0 msgr2=0x55dd8c2941f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 -- 172.21.15.78:0/3254666024 shutdown_connections 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.315+0000 7f72ff584740 1 -- 172.21.15.78:0/3254666024 wait complete. 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 -- start start 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 Processor -- start 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dd8c29a700 con 0x55dd8bed56b0 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dd8c183dc0 con 0x55dd8becd4d0 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dd8c29a880 con 0x55dd8bef6390 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72ff584740 10 client.0 _ll_register_callbacks cb 0x55dd8c19f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72f5b9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72f5b9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57252/0 (socket says 172.21.15.78:57252) 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72f5b9a640 1 -- 172.21.15.78:0/3385203435 learned_addr learned my addr 172.21.15.78:0/3385203435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72f639b640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72f6b9c640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.316+0000 7f72f5b9a640 1 -- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 msgr2=0x55dd8c2a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2a9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72f5b9a640 1 -- 172.21.15.78:0/3385203435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8becd4d0 msgr2=0x55dd8c2a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72f5b9a640 1 -- 172.21.15.78:0/3385203435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8bf0e3f0 con 0x55dd8bed56b0 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72f639b640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a2cc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f72e000f760 tx=0x7f72e000fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fe9686aca05a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72d9ffb640 1 -- 172.21.15.78:0/3385203435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0016070 con 0x55dd8bed56b0 2024-04-25T14:02:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72d9ffb640 1 -- 172.21.15.78:0/3385203435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f72e000bcb0 con 0x55dd8bed56b0 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ec0a7080 con 0x55dd8bed56b0 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72d9ffb640 1 -- 172.21.15.78:0/3385203435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f72e000fe70 con 0x55dd8bed56b0 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 10 client.16290 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55dd8bfd87d0 con 0x55dd8bed56b0 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72d9ffb640 1 -- 172.21.15.78:0/3385203435 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ec0a7080 con 0x55dd8bed56b0 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72d9ffb640 1 -- 172.21.15.78:0/3385203435 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f72e001a400 con 0x55dd8bed56b0 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 10 client.16290 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 10 client.16290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 20 client.16290 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8beb4f60 0x55dd8c2b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 4 client.16290 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.317+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd8c20a990 con 0x55dd8beb4f60 2024-04-25T14:02:30.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.318+0000 7f72f639b640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8beb4f60 0x55dd8c2b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.318+0000 7f72d9ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:02:30.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.318+0000 7f72f639b640 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8beb4f60 0x55dd8c2b0570 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f89c9a0600334f6 server_cookie=a0a3a71dadd518fe in_seq=0 out_seq=0 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72d9ffb640 1 -- 172.21.15.78:0/3385203435 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd8c20a990 con 0x55dd8beb4f60 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72d9ffb640 10 client.16290 handle_command_reply: tid=0 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 client.16290 shutdown 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 msgr2=0x55dd8c2a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a2cc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f72e000f760 tx=0x7f72e000fc20 comp rx=0 tx=0).stop 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 shutdown_connections 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8beb4f60 0x55dd8c2b0570 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72d9ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:02:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a2cc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 --2- 172.21.15.78:0/3385203435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 >> 172.21.15.78:0/3385203435 conn(0x55dd8beea4e0 msgr2=0x55dd8c2933f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 shutdown_connections 2024-04-25T14:02:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.319+0000 7f72ff584740 1 -- 172.21.15.78:0/3385203435 wait complete. 2024-04-25T14:02:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.320+0000 7f72ff584740 20 client.16290 trim_cache size 0 max 16384 2024-04-25T14:02:30.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 Processor -- start 2024-04-25T14:02:30.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 -- start start 2024-04-25T14:02:30.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8becd4d0 0x55dd8c2b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bef6390 0x55dd8c2b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01ada10 con 0x55dd8bed56b0 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01ab2e0 con 0x55dd8bef6390 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72ff584740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01a9130 con 0x55dd8becd4d0 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72f5b9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72f5b9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57262/0 (socket says 172.21.15.78:57262) 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72f5b9a640 1 -- 172.21.15.78:0/1610460080 learned_addr learned my addr 172.21.15.78:0/1610460080 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.333+0000 7f72f6b9c640 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bef6390 0x55dd8c2b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72f5b9a640 1 -- 172.21.15.78:0/1610460080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8becd4d0 msgr2=0x55dd8c2b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8becd4d0 0x55dd8c2b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72f5b9a640 1 -- 172.21.15.78:0/1610460080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bef6390 msgr2=0x55dd8c2b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bef6390 0x55dd8c2b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72f5b9a640 1 -- 172.21.15.78:0/1610460080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8c285ba0 con 0x55dd8bed56b0 2024-04-25T14:02:30.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72f6b9c640 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bef6390 0x55dd8c2b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2b5260 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f72e001a8c0 tx=0x7f72e0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5dc2ab61b8baef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72dbfff640 1 -- 172.21.15.78:0/1610460080 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0034070 con 0x55dd8bed56b0 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72dbfff640 1 -- 172.21.15.78:0/1610460080 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e001acb0 con 0x55dd8bed56b0 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.334+0000 7f72dbfff640 1 -- 172.21.15.78:0/1610460080 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0029070 con 0x55dd8bed56b0 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- 172.21.15.78:0/1610460080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 msgr2=0x55dd8c2b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2b5260 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f72e001a8c0 tx=0x7f72e0005f30 comp rx=0 tx=0).stop 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- 172.21.15.78:0/1610460080 shutdown_connections 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bef6390 0x55dd8c2b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8bed56b0 0x55dd8c2b5260 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 --2- 172.21.15.78:0/1610460080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8becd4d0 0x55dd8c2b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- 172.21.15.78:0/1610460080 >> 172.21.15.78:0/1610460080 conn(0x55dd8beea4e0 msgr2=0x55dd8c29d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- 172.21.15.78:0/1610460080 shutdown_connections 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- 172.21.15.78:0/1610460080 wait complete. 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- start start 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 Processor -- start 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01a97a0 con 0x55dd8becd4d0 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01aac10 con 0x55dd8bed56b0 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72f01a9e70 con 0x55dd8bef6390 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72ff584740 10 client.0 _ll_register_callbacks cb 0x7f72e000d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72f639b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72f639b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57276/0 (socket says 172.21.15.78:57276) 2024-04-25T14:02:30.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72f639b640 1 -- 172.21.15.78:0/636855943 learned_addr learned my addr 172.21.15.78:0/636855943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.335+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72f639b640 1 -- 172.21.15.78:0/636855943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 msgr2=0x55dd8c2a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72f639b640 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72f639b640 1 -- 172.21.15.78:0/636855943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 msgr2=0x55dd8c2a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72f639b640 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72f639b640 1 -- 172.21.15.78:0/636855943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8c2a8560 con 0x55dd8becd4d0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72f639b640 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a6b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0e76e0 tx=0x7f72ec0e7810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3194de0182dddd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72d9ffb640 1 -- 172.21.15.78:0/636855943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec06c060 con 0x55dd8becd4d0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e001acb0 con 0x55dd8becd4d0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72d9ffb640 1 -- 172.21.15.78:0/636855943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f72ec0a77d0 con 0x55dd8becd4d0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72d9ffb640 1 -- 172.21.15.78:0/636855943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f72ec0a6ec0 con 0x55dd8becd4d0 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72ff584740 10 client.16296 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:30.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55dd8c318950 con 0x55dd8becd4d0 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72d9ffb640 1 -- 172.21.15.78:0/636855943 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e001acb0 con 0x55dd8becd4d0 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72d9ffb640 1 -- 172.21.15.78:0/636855943 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f72ec072020 con 0x55dd8becd4d0 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72ff584740 10 client.16296 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72ff584740 10 client.16296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72ff584740 20 client.16296 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.336+0000 7f72ff584740 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8be8a3a0 0x55dd8c31aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.337+0000 7f72ff584740 4 client.16296 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.337+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd8c204800 con 0x55dd8be8a3a0 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.337+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8be8a3a0 0x55dd8c31aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72d9ffb640 10 client.16296 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:02:30.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72f5b9a640 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8be8a3a0 0x55dd8c31aef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b656a5590bb3d70 server_cookie=e156ed9d9ebc014e in_seq=0 out_seq=0 2024-04-25T14:02:30.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72d9ffb640 1 -- 172.21.15.78:0/636855943 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd8c204800 con 0x55dd8be8a3a0 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72d9ffb640 10 client.16296 handle_command_reply: tid=0 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 client.16296 shutdown 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 msgr2=0x55dd8c2a6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a6b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0e76e0 tx=0x7f72ec0e7810 comp rx=0 tx=0).stop 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 shutdown_connections 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55dd8be8a3a0 0x55dd8c31aef0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dd8bef6390 0x55dd8c2a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dd8bed56b0 0x55dd8c2a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72d9ffb640 0 client.16296 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 --2- 172.21.15.78:0/636855943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dd8becd4d0 0x55dd8c2a6b80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 >> 172.21.15.78:0/636855943 conn(0x55dd8beea4e0 msgr2=0x55dd8c296ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 shutdown_connections 2024-04-25T14:02:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.338+0000 7f72ff584740 1 -- 172.21.15.78:0/636855943 wait complete. 2024-04-25T14:02:30.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72ff584740 20 client.16296 trim_cache size 0 max 16384 2024-04-25T14:02:30.344 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f72dc0afae0 msgr2=0x7f72dc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f72dc0afae0 0x7f72dc0b1ff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f72e8031b50 tx=0x7f72e8009450 comp rx=0 tx=0).stop 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0103910 msgr2=0x7f72f019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0103910 0x7f72f019a010 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e9f0 tx=0x7f72ec00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 shutdown_connections 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72f0109850 0x7f72f01a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72f0104b50 0x7f72f019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f72dc0afae0 0x7f72dc0b1ff0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 --2- 172.21.15.78:0/528493966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0103910 0x7f72f019a010 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 >> 172.21.15.78:0/528493966 conn(0x7f72f00feb20 msgr2=0x7f72f0105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 shutdown_connections 2024-04-25T14:02:30.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.339+0000 7f72f739d640 1 -- 172.21.15.78:0/528493966 wait complete. 2024-04-25T14:02:30.349 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:02:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 -- 172.21.15.78:0/502573575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4102f50 msgr2=0x7f5bf4103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/502573575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4102f50 0x7f5bf4103360 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009900 tx=0x7f5bf002a410 comp rx=0 tx=0).stop 2024-04-25T14:02:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 -- 172.21.15.78:0/502573575 shutdown_connections 2024-04-25T14:02:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/502573575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf4104470 0x7f5bf410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/502573575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf4103930 0x7f5bf4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/502573575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4102f50 0x7f5bf4103360 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 -- 172.21.15.78:0/502573575 >> 172.21.15.78:0/502573575 conn(0x7f5bf40fd350 msgr2=0x7f5bf40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 -- 172.21.15.78:0/502573575 shutdown_connections 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.459+0000 7f5bfbe20640 1 -- 172.21.15.78:0/502573575 wait complete. 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 Processor -- start 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 -- start start 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf4102f50 0x7f5bf4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf4103930 0x7f5bf41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4104470 0x7f5bf41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410e500 con 0x7f5bf4104470 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410e380 con 0x7f5bf4102f50 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfbe20640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410e680 con 0x7f5bf4103930 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4104470 0x7f5bf41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4104470 0x7f5bf41a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57300/0 (socket says 172.21.15.78:57300) 2024-04-25T14:02:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 -- 172.21.15.78:0/760644079 learned_addr learned my addr 172.21.15.78:0/760644079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf4103930 0x7f5bf41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf4102f50 0x7f5bf4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 -- 172.21.15.78:0/760644079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf4103930 msgr2=0x7f5bf41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf4103930 0x7f5bf41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 -- 172.21.15.78:0/760644079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf4102f50 msgr2=0x7f5bf4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf4102f50 0x7f5bf4199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.460+0000 7f5bfa3bd640 1 -- 172.21.15.78:0/760644079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf41a0ca0 con 0x7f5bf4104470 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.461+0000 7f5bfa3bd640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4104470 0x7f5bf41a0630 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009fd0 tx=0x7f5bec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.461+0000 7f5beaffd640 1 -- 172.21.15.78:0/760644079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec017070 con 0x7f5bf4104470 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.461+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf41a0f90 con 0x7f5bf4104470 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.461+0000 7f5beaffd640 1 -- 172.21.15.78:0/760644079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bec003e20 con 0x7f5bf4104470 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.461+0000 7f5beaffd640 1 -- 172.21.15.78:0/760644079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec010460 con 0x7f5bf4104470 2024-04-25T14:02:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.461+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf41a13f0 con 0x7f5bf4104470 2024-04-25T14:02:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.463+0000 7f5beaffd640 1 -- 172.21.15.78:0/760644079 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5bec003970 con 0x7f5bf4104470 2024-04-25T14:02:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 Processor -- start 2024-04-25T14:02:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 -- start start 2024-04-25T14:02:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 0x55e41ffde810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc134d0 0x55e41ffded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffdfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e41ff42d80 con 0x55e41fc1b6b0 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e41fefca20 con 0x55e41fc3c390 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5c02dae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e41ff3ed40 con 0x55e41fc134d0 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffdfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffdfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57310/0 (socket says 172.21.15.78:57310) 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5bf93bb640 1 -- 172.21.15.78:0/266445729 learned_addr learned my addr 172.21.15.78:0/266445729 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc134d0 0x55e41ffded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 0x55e41ffde810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.464+0000 7f5bf93bb640 1 -- 172.21.15.78:0/266445729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc134d0 msgr2=0x55e41ffded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc134d0 0x55e41ffded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5beaffd640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5be00afaf0 0x7f5be00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bf93bb640 1 -- 172.21.15.78:0/266445729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 msgr2=0x55e41ffde810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 0x55e41ffde810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bf93bb640 1 -- 172.21.15.78:0/266445729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41ff07c30 con 0x55e41fc1b6b0 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bfa3bd640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5be00afaf0 0x7f5be00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffdfa20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5be4004640 tx=0x7f5be400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1921a83efcd33aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5beaffd640 1 -- 172.21.15.78:0/760644079 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5bec0dea30 con 0x7f5bf4104470 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bdffff640 1 -- 172.21.15.78:0/266445729 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4004220 con 0x55e41fc1b6b0 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bdffff640 1 -- 172.21.15.78:0/266445729 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be40043c0 con 0x55e41fc1b6b0 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.465+0000 7f5bdffff640 1 -- 172.21.15.78:0/266445729 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40035f0 con 0x55e41fc1b6b0 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 -- 172.21.15.78:0/266445729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 msgr2=0x55e41ffdfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5bfa3bd640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5be00afaf0 0x7f5be00b2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5bec003fd0 tx=0x7f5bec06d3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffdfa20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5be4004640 tx=0x7f5be400b870 comp rx=0 tx=0).stop 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 -- 172.21.15.78:0/266445729 shutdown_connections 2024-04-25T14:02:30.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffdfa20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc134d0 0x55e41ffded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 --2- 172.21.15.78:0/266445729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 0x55e41ffde810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 -- 172.21.15.78:0/266445729 >> 172.21.15.78:0/266445729 conn(0x55e41fc304e0 msgr2=0x55e41ffda020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 -- 172.21.15.78:0/266445729 shutdown_connections 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 -- 172.21.15.78:0/266445729 wait complete. 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 -- start start 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.466+0000 7f5c02dae740 1 Processor -- start 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc134d0 0x55e41ffe84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 0x55e41ffef950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5c02dae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e41ffe0530 con 0x55e41fc134d0 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5c02dae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e41fec9dc0 con 0x55e41fc3c390 2024-04-25T14:02:30.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5c02dae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e41ffe06b0 con 0x55e41fc1b6b0 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5c02dae740 10 client.0 _ll_register_callbacks cb 0x55e41fee5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc134d0 0x55e41ffe84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc134d0 0x55e41ffe84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57326/0 (socket says 172.21.15.78:57326) 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 -- 172.21.15.78:0/328415087 learned_addr learned my addr 172.21.15.78:0/328415087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 -- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 msgr2=0x55e41ffe89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 -- 172.21.15.78:0/328415087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 msgr2=0x55e41ffef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 0x55e41ffef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 -- 172.21.15.78:0/328415087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e41fc543f0 con 0x55e41fc134d0 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.467+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc134d0 0x55e41ffe84b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5be400bc20 tx=0x7f5be4004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48f39eafc0dcbec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:30.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/328415087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4005430 con 0x55e41fc134d0 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/328415087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5be4028470 con 0x55e41fc134d0 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be40043c0 con 0x55e41fc134d0 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/328415087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5be40055d0 con 0x55e41fc134d0 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 10 client.16308 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e41fd1e7d0 con 0x55e41fc134d0 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/328415087 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be40043c0 con 0x55e41fc134d0 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/328415087 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5be4033610 con 0x55e41fc134d0 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 10 client.16308 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 10 client.16308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 20 client.16308 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fbfaf60 0x55e41fff5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 4 client.16308 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.468+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e41ff50990 con 0x55e41fbfaf60 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.469+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fbfaf60 0x55e41fff5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.469+0000 7f5bdd7fa640 10 client.16308 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.469+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fbfaf60 0x55e41fff5f40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5398f08f6d505454 server_cookie=394ee37fef390115 in_seq=0 out_seq=0 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/328415087 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e41ff50990 con 0x55e41fbfaf60 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5bdd7fa640 10 client.16308 handle_command_reply: tid=0 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 client.16308 shutdown 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc134d0 msgr2=0x55e41ffe84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc134d0 0x55e41ffe84b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5be400bc20 tx=0x7f5be4004960 comp rx=0 tx=0).stop 2024-04-25T14:02:30.475 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:30 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2787873922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:30.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 shutdown_connections 2024-04-25T14:02:30.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc3c390 0x55e41ffef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fbfaf60 0x55e41fff5f40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5bdd7fa640 0 client.16308 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:02:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 --2- 172.21.15.78:0/328415087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc134d0 0x55e41ffe84b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 >> 172.21.15.78:0/328415087 conn(0x55e41fc304e0 msgr2=0x55e41ffdcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 shutdown_connections 2024-04-25T14:02:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 1 -- 172.21.15.78:0/328415087 wait complete. 2024-04-25T14:02:30.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.470+0000 7f5c02dae740 20 client.16308 trim_cache size 0 max 16384 2024-04-25T14:02:30.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 Processor -- start 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 -- start start 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fbfaf60 0x55e41ffd9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffd9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41aca50 con 0x55e41fc1b6b0 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41aa320 con 0x55e41fc134d0 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5c02dae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41a8170 con 0x55e41fbfaf60 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57330/0 (socket says 172.21.15.78:57330) 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5bf93bb640 1 -- 172.21.15.78:0/1725450888 learned_addr learned my addr 172.21.15.78:0/1725450888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fbfaf60 0x55e41ffd9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.484+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffd9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bf93bb640 1 -- 172.21.15.78:0/1725450888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fbfaf60 msgr2=0x55e41ffd9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fbfaf60 0x55e41ffd9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bf93bb640 1 -- 172.21.15.78:0/1725450888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 msgr2=0x55e41ffd9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffd9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bf93bb640 1 -- 172.21.15.78:0/1725450888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e41ffe65d0 con 0x55e41fc1b6b0 2024-04-25T14:02:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffd9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe8080 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c010 tx=0x7f5be4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a066332029b6d8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bdf7fe640 1 -- 172.21.15.78:0/1725450888 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4005430 con 0x55e41fc1b6b0 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bdf7fe640 1 -- 172.21.15.78:0/1725450888 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be4039030 con 0x55e41fc1b6b0 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.485+0000 7f5bdf7fe640 1 -- 172.21.15.78:0/1725450888 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4028470 con 0x55e41fc1b6b0 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 -- 172.21.15.78:0/1725450888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 msgr2=0x55e41ffe8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe8080 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c010 tx=0x7f5be4000bc0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 -- 172.21.15.78:0/1725450888 shutdown_connections 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fc1b6b0 0x55e41ffe8080 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffd9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 --2- 172.21.15.78:0/1725450888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fbfaf60 0x55e41ffd9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 -- 172.21.15.78:0/1725450888 >> 172.21.15.78:0/1725450888 conn(0x55e41fc304e0 msgr2=0x55e41ffe3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 -- 172.21.15.78:0/1725450888 shutdown_connections 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 -- 172.21.15.78:0/1725450888 wait complete. 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 -- start start 2024-04-25T14:02:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 Processor -- start 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.486+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fbfaf60 0x55e41ffee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e42005e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41a87e0 con 0x55e41fbfaf60 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41a9c50 con 0x55e41fc134d0 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41a8eb0 con 0x55e41fc1b6b0 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 10 client.0 _ll_register_callbacks cb 0x7f5bf0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fbfaf60 0x55e41ffee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fbfaf60 0x55e41ffee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57334/0 (socket says 172.21.15.78:57334) 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 -- 172.21.15.78:0/2633454947 learned_addr learned my addr 172.21.15.78:0/2633454947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e42005e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 -- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 msgr2=0x55e42005e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e42005e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 -- 172.21.15.78:0/2633454947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 msgr2=0x55e41ffee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffee9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 -- 172.21.15.78:0/2633454947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42005ec20 con 0x55e41fbfaf60 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bf93bb640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fbfaf60 0x55e41ffee490 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005c90 tx=0x7f5be400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c867db2b13464a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/2633454947 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40336d0 con 0x55e41fbfaf60 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/2633454947 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5be4004050 con 0x55e41fbfaf60 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be4039030 con 0x55e41fbfaf60 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/2633454947 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5be4004460 con 0x55e41fbfaf60 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 10 client.16314 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.487+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e42005fea0 con 0x55e41fbfaf60 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/2633454947 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be4039030 con 0x55e41fbfaf60 2024-04-25T14:02:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/2633454947 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5be403e030 con 0x55e41fbfaf60 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5c02dae740 10 client.16314 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5c02dae740 10 client.16314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5c02dae740 20 client.16314 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5c02dae740 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fc3c390 0x55e4200627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5c02dae740 4 client.16314 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e41ff4a800 con 0x55e41fc3c390 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.488+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fc3c390 0x55e4200627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5bdd7fa640 10 client.16314 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5bf9bbc640 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fc3c390 0x55e4200627d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6572d74b175f4cfc server_cookie=413deb240b3aaded in_seq=0 out_seq=0 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5bdd7fa640 1 -- 172.21.15.78:0/2633454947 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e41ff4a800 con 0x55e41fc3c390 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5bdd7fa640 10 client.16314 handle_command_reply: tid=0 2024-04-25T14:02:30.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 client.16314 shutdown 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fbfaf60 msgr2=0x55e41ffee490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fbfaf60 0x55e41ffee490 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5be4005c90 tx=0x7f5be400bd70 comp rx=0 tx=0).stop 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 shutdown_connections 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55e41fc3c390 0x55e4200627d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e41fc1b6b0 0x55e42005e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e41fc134d0 0x55e41ffee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 --2- 172.21.15.78:0/2633454947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e41fbfaf60 0x55e41ffee490 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5bdd7fa640 0 client.16314 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 >> 172.21.15.78:0/2633454947 conn(0x55e41fc304e0 msgr2=0x55e41ffe8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 shutdown_connections 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 1 -- 172.21.15.78:0/2633454947 wait complete. 2024-04-25T14:02:30.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.489+0000 7f5c02dae740 20 client.16314 trim_cache size 0 max 16384 2024-04-25T14:02:30.500 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:02:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5be00afaf0 msgr2=0x7f5be00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5be00afaf0 0x7f5be00b2000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5bec003fd0 tx=0x7f5bec06d3c0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4104470 msgr2=0x7f5bf41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4104470 0x7f5bf41a0630 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009fd0 tx=0x7f5bec00b260 comp rx=0 tx=0).stop 2024-04-25T14:02:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 shutdown_connections 2024-04-25T14:02:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5be00afaf0 0x7f5be00b2000 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bf4104470 0x7f5bf41a0630 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf4103930 0x7f5bf41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 --2- 172.21.15.78:0/760644079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf4102f50 0x7f5bf4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 >> 172.21.15.78:0/760644079 conn(0x7f5bf40fd350 msgr2=0x7f5bf4104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 shutdown_connections 2024-04-25T14:02:30.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.490+0000 7f5bfbe20640 1 -- 172.21.15.78:0/760644079 wait complete. 2024-04-25T14:02:30.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:02:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:30 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2787873922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 -- 172.21.15.78:0/4196152366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0104690 msgr2=0x7f9ac010ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/4196152366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0104690 0x7f9ac010ade0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009ca0 tx=0x7f9ab802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 -- 172.21.15.78:0/4196152366 shutdown_connections 2024-04-25T14:02:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/4196152366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0104690 0x7f9ac010ade0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/4196152366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ac0103b30 0x7f9ac0104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/4196152366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ac0102fd0 0x7f9ac01033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 -- 172.21.15.78:0/4196152366 >> 172.21.15.78:0/4196152366 conn(0x7f9ac00fd350 msgr2=0x7f9ac00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 -- 172.21.15.78:0/4196152366 shutdown_connections 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.613+0000 7f9ac4a49640 1 -- 172.21.15.78:0/4196152366 wait complete. 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 Processor -- start 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 -- start start 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ac0102fd0 0x7f9ac0198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0103b30 0x7f9ac0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ac019e3a0 0x7f9ac01a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac010e760 con 0x7f9ac0103b30 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac010e5e0 con 0x7f9ac0102fd0 2024-04-25T14:02:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9ac4a49640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac010e8e0 con 0x7f9ac019e3a0 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0103b30 0x7f9ac0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0103b30 0x7f9ac0199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57356/0 (socket says 172.21.15.78:57356) 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 -- 172.21.15.78:0/481175980 learned_addr learned my addr 172.21.15.78:0/481175980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abed9d640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ac019e3a0 0x7f9ac01a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abe59c640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ac0102fd0 0x7f9ac0198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 -- 172.21.15.78:0/481175980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ac019e3a0 msgr2=0x7f9ac01a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ac019e3a0 0x7f9ac01a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 -- 172.21.15.78:0/481175980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ac0102fd0 msgr2=0x7f9ac0198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ac0102fd0 0x7f9ac0198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abdd9b640 1 -- 172.21.15.78:0/481175980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac01a0d80 con 0x7f9ac0103b30 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abed9d640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ac019e3a0 0x7f9ac01a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.614+0000 7f9abe59c640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ac0102fd0 0x7f9ac0198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.615+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0103b30 0x7f9ac0199420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e9f0 tx=0x7f9ab000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.615+0000 7f9aaf7fe640 1 -- 172.21.15.78:0/481175980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000cd80 con 0x7f9ac0103b30 2024-04-25T14:02:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.615+0000 7f9aaf7fe640 1 -- 172.21.15.78:0/481175980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab00054d0 con 0x7f9ac0103b30 2024-04-25T14:02:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.615+0000 7f9aaf7fe640 1 -- 172.21.15.78:0/481175980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0010690 con 0x7f9ac0103b30 2024-04-25T14:02:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.615+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac01a1070 con 0x7f9ac0103b30 2024-04-25T14:02:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.615+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac01a1570 con 0x7f9ac0103b30 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.617+0000 7f9acb9d4740 1 Processor -- start 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.617+0000 7f9acb9d4740 1 -- start start 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.617+0000 7f9aaf7fe640 1 -- 172.21.15.78:0/481175980 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9ab0010830 con 0x7f9ac0103b30 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.617+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700b0390 0x557b70454a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b70454fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b70455c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b703b6d80 con 0x557b700874d0 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b70370a20 con 0x557b7008f6b0 2024-04-25T14:02:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557b703b2d40 con 0x557b700b0390 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b70454fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b70454fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57358/0 (socket says 172.21.15.78:57358) 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 -- 172.21.15.78:0/1088480895 learned_addr learned my addr 172.21.15.78:0/1088480895 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abed9d640 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700b0390 0x557b70454a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b70455c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 -- 172.21.15.78:0/1088480895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700b0390 msgr2=0x557b70454a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700b0390 0x557b70454a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 -- 172.21.15.78:0/1088480895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 msgr2=0x557b70455c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b70455c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 -- 172.21.15.78:0/1088480895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b7037bc30 con 0x557b700874d0 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b70455c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abe59c640 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b70454fa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400bbe0 tx=0x7f9ab400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38be3c6fc0171e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9aaf7fe640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9aa80afaf0 0x7f9aa80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9aacff9640 1 -- 172.21.15.78:0/1088480895 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4017620 con 0x557b700874d0 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9abed9d640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9aa80afaf0 0x7f9aa80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9aacff9640 1 -- 172.21.15.78:0/1088480895 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab4017c70 con 0x557b700874d0 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9aaf7fe640 1 -- 172.21.15.78:0/481175980 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9ab00df160 con 0x7f9ac0103b30 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.618+0000 7f9aacff9640 1 -- 172.21.15.78:0/1088480895 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400fc60 con 0x557b700874d0 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9acb9d4740 1 -- 172.21.15.78:0/1088480895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 msgr2=0x557b70454fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b70454fa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400bbe0 tx=0x7f9ab400d710 comp rx=0 tx=0).stop 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9abed9d640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9aa80afaf0 0x7f9aa80b2000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8031b50 tx=0x7f9ab80098c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9acb9d4740 1 -- 172.21.15.78:0/1088480895 shutdown_connections 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b70455c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b70454fa0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/1088480895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700b0390 0x557b70454a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.619+0000 7f9acb9d4740 1 -- 172.21.15.78:0/1088480895 >> 172.21.15.78:0/1088480895 conn(0x557b700a44e0 msgr2=0x557b7044e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 -- 172.21.15.78:0/1088480895 shutdown_connections 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 -- 172.21.15.78:0/1088480895 wait complete. 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 -- start start 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 Processor -- start 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7045e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 0x557b70465bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557b70456740 con 0x557b700874d0 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557b7033ddc0 con 0x557b700b0390 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557b704568c0 con 0x557b7008f6b0 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9acb9d4740 10 client.0 _ll_register_callbacks cb 0x557b70359600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7045e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7045e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57362/0 (socket says 172.21.15.78:57362) 2024-04-25T14:02:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9abe59c640 1 -- 172.21.15.78:0/220755232 learned_addr learned my addr 172.21.15.78:0/220755232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.620+0000 7f9abed9d640 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 0x557b70465bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9abe59c640 1 -- 172.21.15.78:0/220755232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 msgr2=0x557b7045ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9abe59c640 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9abe59c640 1 -- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 msgr2=0x557b70465bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9abe59c640 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 0x557b70465bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9abe59c640 1 -- 172.21.15.78:0/220755232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b700c83f0 con 0x557b700874d0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9abe59c640 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7045e750 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4019040 tx=0x7f9ab400fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57570bdb4a3c2023 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/220755232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4026020 con 0x557b700874d0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/220755232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9ab4017630 con 0x557b700874d0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab4017c70 con 0x557b700874d0 2024-04-25T14:02:30.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/220755232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9ab4031050 con 0x557b700874d0 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9acb9d4740 10 client.16326 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557b701927d0 con 0x557b700874d0 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/220755232 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab4017c70 con 0x557b700874d0 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/220755232 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9ab4021900 con 0x557b700874d0 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9acb9d4740 10 client.16326 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9acb9d4740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.621+0000 7f9acb9d4740 20 client.16326 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.622+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b7006ef60 0x557b7046c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.622+0000 7f9acb9d4740 4 client.16326 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.622+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b703c4990 con 0x557b7006ef60 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.622+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b7006ef60 0x557b7046c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.622+0000 7f9aa5ffb640 10 client.16326 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.622+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b7006ef60 0x557b7046c120 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b5c5d3a2c9d13c server_cookie=160899ab451f8654 in_seq=0 out_seq=0 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/220755232 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b703c4990 con 0x557b7006ef60 2024-04-25T14:02:30.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9aa5ffb640 10 client.16326 handle_command_reply: tid=0 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 client.16326 shutdown 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 msgr2=0x557b7045e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7045e750 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4019040 tx=0x7f9ab400fef0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 shutdown_connections 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 0x557b70465bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b7006ef60 0x557b7046c120 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9aa5ffb640 0 client.16326 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/220755232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7045e750 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 >> 172.21.15.78:0/220755232 conn(0x557b700a44e0 msgr2=0x557b70450f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 shutdown_connections 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 1 -- 172.21.15.78:0/220755232 wait complete. 2024-04-25T14:02:30.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.623+0000 7f9acb9d4740 20 client.16326 trim_cache size 0 max 16384 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.636+0000 7f9acb9d4740 1 Processor -- start 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.636+0000 7f9acb9d4740 1 -- start start 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7044e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 0x557b7045d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac01ab8a0 con 0x557b700874d0 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac01a9170 con 0x557b700b0390 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac01a6fc0 con 0x557b7008f6b0 2024-04-25T14:02:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7044e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7044e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57370/0 (socket says 172.21.15.78:57370) 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 -- 172.21.15.78:0/957764804 learned_addr learned my addr 172.21.15.78:0/957764804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 -- 172.21.15.78:0/957764804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 msgr2=0x557b7045cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 -- 172.21.15.78:0/957764804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 msgr2=0x557b7045d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 0x557b7045d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 -- 172.21.15.78:0/957764804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b704569a0 con 0x557b700874d0 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9abe59c640 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7044e160 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4022010 tx=0x7f9ab400fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06c9bfc933a0ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9aa7fff640 1 -- 172.21.15.78:0/957764804 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4026040 con 0x557b700874d0 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9aa7fff640 1 -- 172.21.15.78:0/957764804 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab4005070 con 0x557b700874d0 2024-04-25T14:02:30.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.637+0000 7f9aa7fff640 1 -- 172.21.15.78:0/957764804 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4022a90 con 0x557b700874d0 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 -- 172.21.15.78:0/957764804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 msgr2=0x557b7044e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7044e160 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4022010 tx=0x7f9ab400fd90 comp rx=0 tx=0).stop 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 -- 172.21.15.78:0/957764804 shutdown_connections 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b700b0390 0x557b7045d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b7008f6b0 0x557b7045cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/957764804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700874d0 0x557b7044e160 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 -- 172.21.15.78:0/957764804 >> 172.21.15.78:0/957764804 conn(0x557b700a44e0 msgr2=0x557b7044da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 -- 172.21.15.78:0/957764804 shutdown_connections 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 -- 172.21.15.78:0/957764804 wait complete. 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 -- start start 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.638+0000 7f9acb9d4740 1 Processor -- start 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700874d0 0x557b70461f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b704624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700b0390 0x557b704d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac01a7630 con 0x557b700b0390 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac01a8aa0 con 0x557b7008f6b0 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac01a7d00 con 0x557b700874d0 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 10 client.0 _ll_register_callbacks cb 0x7f9ab400ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700b0390 0x557b704d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700b0390 0x557b704d2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57380/0 (socket says 172.21.15.78:57380) 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 -- 172.21.15.78:0/3774051398 learned_addr learned my addr 172.21.15.78:0/3774051398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abe59c640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700874d0 0x557b70461f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b704624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 -- 172.21.15.78:0/3774051398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700874d0 msgr2=0x557b70461f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700874d0 0x557b70461f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 -- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 msgr2=0x557b704624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b704624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 -- 172.21.15.78:0/3774051398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b70465460 con 0x557b700b0390 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abe59c640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700874d0 0x557b70461f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abdd9b640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b704624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9abed9d640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700b0390 0x557b704d2440 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8003bf0 tx=0x7f9ab802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a231317e0ea1f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/3774051398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8004310 con 0x557b700b0390 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/3774051398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9ab80044b0 con 0x557b700b0390 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.639+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab4005070 con 0x557b700b0390 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/3774051398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9ab80046b0 con 0x557b700b0390 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 10 client.16332 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557b704d38c0 con 0x557b700b0390 2024-04-25T14:02:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/3774051398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab4005070 con 0x557b700b0390 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/3774051398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9ab8044040 con 0x557b700b0390 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 10 client.16332 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 20 client.16332 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b700443a0 0x557b704d6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 4 client.16332 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b703be800 con 0x557b700443a0 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.640+0000 7f9abe59c640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b700443a0 0x557b704d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9aa5ffb640 10 client.16332 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9abe59c640 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b700443a0 0x557b704d6310 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c24d2381a3cfc2 server_cookie=df23e412de191bde in_seq=0 out_seq=0 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9aa5ffb640 1 -- 172.21.15.78:0/3774051398 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b703be800 con 0x557b700443a0 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9aa5ffb640 10 client.16332 handle_command_reply: tid=0 2024-04-25T14:02:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 client.16332 shutdown 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700b0390 msgr2=0x557b704d2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700b0390 0x557b704d2440 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8003bf0 tx=0x7f9ab802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 shutdown_connections 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b700b0390 0x557b704d2440 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557b7008f6b0 0x557b704624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557b700443a0 0x557b704d6310 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 --2- 172.21.15.78:0/3774051398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b700874d0 0x557b70461f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9aa5ffb640 0 client.16332 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 >> 172.21.15.78:0/3774051398 conn(0x557b700a44e0 msgr2=0x557b70450e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 shutdown_connections 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 1 -- 172.21.15.78:0/3774051398 wait complete. 2024-04-25T14:02:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9acb9d4740 20 client.16332 trim_cache size 0 max 16384 2024-04-25T14:02:30.648 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:02:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9aa80afaf0 msgr2=0x7f9aa80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9aa80afaf0 0x7f9aa80b2000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8031b50 tx=0x7f9ab80098c0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0103b30 msgr2=0x7f9ac0199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.641+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0103b30 0x7f9ac0199420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000e9f0 tx=0x7f9ab000edf0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 shutdown_connections 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ac019e3a0 0x7f9ac01a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9aa80afaf0 0x7f9aa80b2000 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ac0103b30 0x7f9ac0199420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 --2- 172.21.15.78:0/481175980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ac0102fd0 0x7f9ac0198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 >> 172.21.15.78:0/481175980 conn(0x7f9ac00fd350 msgr2=0x7f9ac0106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 shutdown_connections 2024-04-25T14:02:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:30.642+0000 7f9ac4a49640 1 -- 172.21.15.78:0/481175980 wait complete. 2024-04-25T14:02:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:30 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2787873922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:02:31.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:31 smithi112 ceph-mon[32617]: pgmap v438: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:31.652 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:02:31.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:02:31.653 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:02:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:31 smithi129 ceph-mon[32569]: pgmap v438: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:31.677 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:31 smithi078 ceph-mon[29163]: pgmap v438: 97 pgs: 97 active+clean; 1.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:31.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 -- 172.21.15.78:0/4234888415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44103c20 msgr2=0x7f8d44104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/4234888415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44103c20 0x7f8d44104030 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009900 tx=0x7f8d3802a410 comp rx=0 tx=0).stop 2024-04-25T14:02:31.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 -- 172.21.15.78:0/4234888415 shutdown_connections 2024-04-25T14:02:31.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/4234888415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d44108920 0x7f8d4410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/4234888415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44103c20 0x7f8d44104030 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/4234888415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d441029e0 0x7f8d441036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 -- 172.21.15.78:0/4234888415 >> 172.21.15.78:0/4234888415 conn(0x7f8d440fdbd0 msgr2=0x7f8d44100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 -- 172.21.15.78:0/4234888415 shutdown_connections 2024-04-25T14:02:31.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.783+0000 7f8d4a06e640 1 -- 172.21.15.78:0/4234888415 wait complete. 2024-04-25T14:02:31.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 Processor -- start 2024-04-25T14:02:31.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 -- start start 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d441029e0 0x7f8d4419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d44103c20 0x7f8d4419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44108920 0x7f8d441a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4410f610 con 0x7f8d44108920 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4410f490 con 0x7f8d441029e0 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d4a06e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4410f790 con 0x7f8d44103c20 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.784+0000 7f8d43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44108920 0x7f8d441a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44108920 0x7f8d441a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57400/0 (socket says 172.21.15.78:57400) 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 -- 172.21.15.78:0/2022948396 learned_addr learned my addr 172.21.15.78:0/2022948396 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d437fe640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d441029e0 0x7f8d4419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d44103c20 0x7f8d4419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 -- 172.21.15.78:0/2022948396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d44103c20 msgr2=0x7f8d4419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d44103c20 0x7f8d4419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 -- 172.21.15.78:0/2022948396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d441029e0 msgr2=0x7f8d4419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d441029e0 0x7f8d4419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 -- 172.21.15.78:0/2022948396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d441a2080 con 0x7f8d44108920 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d43fff640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44108920 0x7f8d441a1a10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00eb10 tx=0x7f8d3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d40ff9640 1 -- 172.21.15.78:0/2022948396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c004b20 con 0x7f8d44108920 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d40ff9640 1 -- 172.21.15.78:0/2022948396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d3c004cc0 con 0x7f8d44108920 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d441a2370 con 0x7f8d44108920 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d40ff9640 1 -- 172.21.15.78:0/2022948396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c010670 con 0x7f8d44108920 2024-04-25T14:02:31.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.785+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d441a28c0 con 0x7f8d44108920 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d40ff9640 1 -- 172.21.15.78:0/2022948396 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8d3c005020 con 0x7f8d44108920 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 Processor -- start 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 -- start start 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff98006550 0x55ff983d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9800e730 0x55ff983d7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ff98335b00 con 0x55ff9802f410 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ff982d1920 con 0x55ff9800e730 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d50fff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ff98331ac0 con 0x55ff98006550 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57412/0 (socket says 172.21.15.78:57412) 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d42ffd640 1 -- 172.21.15.78:0/174904165 learned_addr learned my addr 172.21.15.78:0/174904165 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d40ff9640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8d2c0afaf0 0x7f8d2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.788+0000 7f8d437fe640 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff98006550 0x55ff983d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9800e730 0x55ff983d7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d43fff640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8d2c0afaf0 0x7f8d2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d42ffd640 1 -- 172.21.15.78:0/174904165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff98006550 msgr2=0x55ff983d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff98006550 0x55ff983d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d42ffd640 1 -- 172.21.15.78:0/174904165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9800e730 msgr2=0x55ff983d7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9800e730 0x55ff983d7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d42ffd640 1 -- 172.21.15.78:0/174904165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9810f7e0 con 0x55ff9802f410 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d40ff9640 1 -- 172.21.15.78:0/2022948396 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8d3c0df250 con 0x7f8d44108920 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983d04a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8d3802a930 tx=0x7f8d38005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce222ccc12d0bae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d29ffb640 1 -- 172.21.15.78:0/174904165 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38035050 con 0x55ff9802f410 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d29ffb640 1 -- 172.21.15.78:0/174904165 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d38041030 con 0x55ff9802f410 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.789+0000 7f8d29ffb640 1 -- 172.21.15.78:0/174904165 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38005570 con 0x55ff9802f410 2024-04-25T14:02:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 -- 172.21.15.78:0/174904165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 msgr2=0x55ff983d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d43fff640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8d2c0afaf0 0x7f8d2c0b2000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c020000 tx=0x7f8d3c076040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983d04a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8d3802a930 tx=0x7f8d38005a40 comp rx=0 tx=0).stop 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 -- 172.21.15.78:0/174904165 shutdown_connections 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9800e730 0x55ff983d7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff98006550 0x55ff983d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 --2- 172.21.15.78:0/174904165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983d04a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 -- 172.21.15.78:0/174904165 >> 172.21.15.78:0/174904165 conn(0x55ff98023560 msgr2=0x55ff983cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 -- 172.21.15.78:0/174904165 shutdown_connections 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 -- 172.21.15.78:0/174904165 wait complete. 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.790+0000 7f8d50fff740 1 -- start start 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 1 Processor -- start 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ff983d8310 con 0x55ff9802f410 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ff982cf5d0 con 0x55ff98006550 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ff983d8490 con 0x55ff9800e730 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d50fff740 10 client.0 _ll_register_callbacks cb 0x55ff982bd080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57420/0 (socket says 172.21.15.78:57420) 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d42ffd640 1 -- 172.21.15.78:0/365183124 learned_addr learned my addr 172.21.15.78:0/365183124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d437fe640 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.791+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d42ffd640 1 -- 172.21.15.78:0/365183124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 msgr2=0x55ff983da810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983da810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d42ffd640 1 -- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 msgr2=0x55ff983da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d42ffd640 1 -- 172.21.15.78:0/365183124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9816ff70 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983e1770 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8d38031440 tx=0x7f8d38005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae580fef75c939c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d22ffd640 1 -- 172.21.15.78:0/365183124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38035040 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d22ffd640 1 -- 172.21.15.78:0/365183124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8d380315a0 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d38041030 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d22ffd640 1 -- 172.21.15.78:0/365183124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8d380317a0 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d50fff740 10 client.16344 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ff98007170 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d22ffd640 1 -- 172.21.15.78:0/365183124 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d38041030 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d22ffd640 1 -- 172.21.15.78:0/365183124 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8d38004cc0 con 0x55ff9802f410 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d50fff740 10 client.16344 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d50fff740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:31.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.792+0000 7f8d50fff740 20 client.16344 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.793+0000 7f8d50fff740 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fee1f0 0x55ff983e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.793+0000 7f8d50fff740 4 client.16344 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.793+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff98343710 con 0x55ff97fee1f0 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.793+0000 7f8d437fe640 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fee1f0 0x55ff983e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.793+0000 7f8d22ffd640 10 client.16344 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.793+0000 7f8d437fe640 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fee1f0 0x55ff983e7830 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6903d4f153dbb0 server_cookie=b7c0b8fd3c61fb74 in_seq=0 out_seq=0 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d22ffd640 1 -- 172.21.15.78:0/365183124 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff98343710 con 0x55ff97fee1f0 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d22ffd640 10 client.16344 handle_command_reply: tid=0 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 client.16344 shutdown 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 msgr2=0x55ff983e1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983e1770 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8d38031440 tx=0x7f8d38005040 comp rx=0 tx=0).stop 2024-04-25T14:02:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 shutdown_connections 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9802f410 0x55ff983e1770 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fee1f0 0x55ff983e7830 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 --2- 172.21.15.78:0/365183124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d22ffd640 0 client.16344 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.794+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 >> 172.21.15.78:0/365183124 conn(0x55ff98023560 msgr2=0x55ff983caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.795+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 shutdown_connections 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.795+0000 7f8d50fff740 1 -- 172.21.15.78:0/365183124 wait complete. 2024-04-25T14:02:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.795+0000 7f8d50fff740 20 client.16344 trim_cache size 0 max 16384 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 Processor -- start 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 -- start start 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff98006550 0x55ff983d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9802f410 0x55ff983cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441adb10 con 0x55ff98006550 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441ab3e0 con 0x55ff9802f410 2024-04-25T14:02:31.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d50fff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441a9230 con 0x55ff9800e730 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff98006550 0x55ff983d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff98006550 0x55ff983d8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57430/0 (socket says 172.21.15.78:57430) 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1200439338 learned_addr learned my addr 172.21.15.78:0/1200439338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d437fe640 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1200439338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 msgr2=0x55ff983cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1200439338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9802f410 msgr2=0x55ff983cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9802f410 0x55ff983cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 -- 172.21.15.78:0/1200439338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff983d51b0 con 0x55ff98006550 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff98006550 0x55ff983d8f90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8d380092a0 tx=0x7f8d380319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76278ec2657e8d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.813+0000 7f8d297fa640 1 -- 172.21.15.78:0/1200439338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38035040 con 0x55ff98006550 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d297fa640 1 -- 172.21.15.78:0/1200439338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d38041070 con 0x55ff98006550 2024-04-25T14:02:31.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d297fa640 1 -- 172.21.15.78:0/1200439338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38004dd0 con 0x55ff98006550 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 -- 172.21.15.78:0/1200439338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff98006550 msgr2=0x55ff983d8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff98006550 0x55ff983d8f90 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8d380092a0 tx=0x7f8d380319c0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 -- 172.21.15.78:0/1200439338 shutdown_connections 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff9802f410 0x55ff983cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9800e730 0x55ff983cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 --2- 172.21.15.78:0/1200439338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff98006550 0x55ff983d8f90 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 -- 172.21.15.78:0/1200439338 >> 172.21.15.78:0/1200439338 conn(0x55ff98023560 msgr2=0x55ff983cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 -- 172.21.15.78:0/1200439338 shutdown_connections 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.814+0000 7f8d50fff740 1 -- 172.21.15.78:0/1200439338 wait complete. 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 -- start start 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 Processor -- start 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983e14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9800e730 0x55ff983e1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9802f410 0x55ff9844dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441a98a0 con 0x55ff9800e730 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441aad10 con 0x55ff98006550 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441a9f70 con 0x55ff9802f410 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d50fff740 10 client.0 _ll_register_callbacks cb 0x7f8d38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9800e730 0x55ff983e1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9800e730 0x55ff983e1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57442/0 (socket says 172.21.15.78:57442) 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d437fe640 1 -- 172.21.15.78:0/2692669085 learned_addr learned my addr 172.21.15.78:0/2692669085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983e14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9802f410 0x55ff9844dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.815+0000 7f8d437fe640 1 -- 172.21.15.78:0/2692669085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9802f410 msgr2=0x55ff9844dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d437fe640 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9802f410 0x55ff9844dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d437fe640 1 -- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 msgr2=0x55ff983e14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d437fe640 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983e14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d437fe640 1 -- 172.21.15.78:0/2692669085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9844e360 con 0x55ff9800e730 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d437fe640 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9800e730 0x55ff983e1a30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55ff983d1880 tx=0x7f8d3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d02917a135a9d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d22ffd640 1 -- 172.21.15.78:0/2692669085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340200e0 con 0x55ff9800e730 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d22ffd640 1 -- 172.21.15.78:0/2692669085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8d34020280 con 0x55ff9800e730 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d38041070 con 0x55ff9800e730 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d22ffd640 1 -- 172.21.15.78:0/2692669085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8d340204a0 con 0x55ff9800e730 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 10 client.16350 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:31.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ff9844f6f0 con 0x55ff9800e730 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d22ffd640 1 -- 172.21.15.78:0/2692669085 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d38041070 con 0x55ff9800e730 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d22ffd640 1 -- 172.21.15.78:0/2692669085 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8d34011b20 con 0x55ff9800e730 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 10 client.16350 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 20 client.16350 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fc3630 0x55ff98452160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 4 client.16350 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.816+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ff9833d580 con 0x55ff97fc3630 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.817+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fc3630 0x55ff98452160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.817+0000 7f8d42ffd640 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fc3630 0x55ff98452160 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf846ea2c1254e8 server_cookie=c54d46bd138c8481 in_seq=0 out_seq=0 2024-04-25T14:02:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.817+0000 7f8d22ffd640 10 client.16350 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d22ffd640 1 -- 172.21.15.78:0/2692669085 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ff9833d580 con 0x55ff97fc3630 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d22ffd640 10 client.16350 handle_command_reply: tid=0 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 client.16350 shutdown 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9800e730 msgr2=0x55ff983e1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9800e730 0x55ff983e1a30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55ff983d1880 tx=0x7f8d3400be30 comp rx=0 tx=0).stop 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 shutdown_connections 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ff97fc3630 0x55ff98452160 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff9802f410 0x55ff9844dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff9800e730 0x55ff983e1a30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 --2- 172.21.15.78:0/2692669085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ff98006550 0x55ff983e14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d22ffd640 0 client.16350 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 >> 172.21.15.78:0/2692669085 conn(0x55ff98023560 msgr2=0x55ff983cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 shutdown_connections 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.818+0000 7f8d50fff740 1 -- 172.21.15.78:0/2692669085 wait complete. 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d50fff740 20 client.16350 trim_cache size 0 max 16384 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "f43bf553-f510-4376-9007-1ce24a7f5db1", 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:02:31.825 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:02:31.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8d2c0afaf0 msgr2=0x7f8d2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8d2c0afaf0 0x7f8d2c0b2000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c020000 tx=0x7f8d3c076040 comp rx=0 tx=0).stop 2024-04-25T14:02:31.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44108920 msgr2=0x7f8d441a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44108920 0x7f8d441a1a10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00eb10 tx=0x7f8d3c00c2d0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 shutdown_connections 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8d2c0afaf0 0x7f8d2c0b2000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d44108920 0x7f8d441a1a10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d44103c20 0x7f8d4419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 --2- 172.21.15.78:0/2022948396 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d441029e0 0x7f8d4419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 >> 172.21.15.78:0/2022948396 conn(0x7f8d440fdbd0 msgr2=0x7f8d44104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 shutdown_connections 2024-04-25T14:02:31.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.819+0000 7f8d4a06e640 1 -- 172.21.15.78:0/2022948396 wait complete. 2024-04-25T14:02:31.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:02:31.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 -- 172.21.15.78:0/3625529081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c101750 msgr2=0x7f395c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 --2- 172.21.15.78:0/3625529081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c101750 0x7f395c101bc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3950009900 tx=0x7f395002a410 comp rx=0 tx=0).stop 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 -- 172.21.15.78:0/3625529081 shutdown_connections 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 --2- 172.21.15.78:0/3625529081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f395c102250 0x7f395c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 --2- 172.21.15.78:0/3625529081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c101750 0x7f395c101bc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 --2- 172.21.15.78:0/3625529081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395c1050a0 0x7f395c1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 -- 172.21.15.78:0/3625529081 >> 172.21.15.78:0/3625529081 conn(0x7f395c0fd350 msgr2=0x7f395c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 -- 172.21.15.78:0/3625529081 shutdown_connections 2024-04-25T14:02:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 -- 172.21.15.78:0/3625529081 wait complete. 2024-04-25T14:02:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 Processor -- start 2024-04-25T14:02:31.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.981+0000 7f3962a42640 1 -- start start 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f3962a42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395c101750 0x7f395c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f3962a42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c102250 0x7f395c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f3962a42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f395c1050a0 0x7f395c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f3962a42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10f610 con 0x7f395c102250 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f3962a42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10f490 con 0x7f395c1050a0 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f3962a42640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10f790 con 0x7f395c101750 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c102250 0x7f395c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c102250 0x7f395c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57460/0 (socket says 172.21.15.78:57460) 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 -- 172.21.15.78:0/2907792577 learned_addr learned my addr 172.21.15.78:0/2907792577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f3960fdf640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f395c1050a0 0x7f395c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395bfff640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395c101750 0x7f395c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 -- 172.21.15.78:0/2907792577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395c101750 msgr2=0x7f395c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395c101750 0x7f395c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 -- 172.21.15.78:0/2907792577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f395c1050a0 msgr2=0x7f395c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f395c1050a0 0x7f395c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 -- 172.21.15.78:0/2907792577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395c1a1d50 con 0x7f395c102250 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c102250 0x7f395c19a750 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3950006010 tx=0x7f3950004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.982+0000 7f39597fa640 1 -- 172.21.15.78:0/2907792577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395003f030 con 0x7f395c102250 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.983+0000 7f39597fa640 1 -- 172.21.15.78:0/2907792577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3950003cb0 con 0x7f395c102250 2024-04-25T14:02:31.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.983+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395c1a1fe0 con 0x7f395c102250 2024-04-25T14:02:31.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.983+0000 7f39597fa640 1 -- 172.21.15.78:0/2907792577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950035070 con 0x7f395c102250 2024-04-25T14:02:31.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.983+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395c1a23f0 con 0x7f395c102250 2024-04-25T14:02:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 Processor -- start 2024-04-25T14:02:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39597fa640 1 -- 172.21.15.78:0/2907792577 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3950042030 con 0x7f395c102250 2024-04-25T14:02:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 -- start start 2024-04-25T14:02:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2083da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca204734d0 0x55ca2083dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca2047b6b0 0x55ca2083ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca207a2d80 con 0x55ca2047b6b0 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ca2075c8b0 con 0x55ca204734d0 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.985+0000 7f39699c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ca2079ed40 con 0x55ca2049c390 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca2047b6b0 0x55ca2083ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca2047b6b0 0x55ca2083ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57462/0 (socket says 172.21.15.78:57462) 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 -- 172.21.15.78:0/2470381216 learned_addr learned my addr 172.21.15.78:0/2470381216 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395bfff640 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca204734d0 0x55ca2083dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f3960fdf640 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2083da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 -- 172.21.15.78:0/2470381216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 msgr2=0x55ca2083da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2083da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 -- 172.21.15.78:0/2470381216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca204734d0 msgr2=0x55ca2083dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca204734d0 0x55ca2083dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 -- 172.21.15.78:0/2470381216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2072da40 con 0x55ca2047b6b0 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f39597fa640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39440afaf0 0x7f39440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca2047b6b0 0x55ca2083ec70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3950107560 tx=0x7f3950107650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edb9edb6dc7f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f3960fdf640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39440afaf0 0x7f39440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f39597fa640 1 -- 172.21.15.78:0/2907792577 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f39500ff370 con 0x7f395c102250 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f39427fc640 1 -- 172.21.15.78:0/2470381216 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500c3700 con 0x55ca2047b6b0 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f39427fc640 1 -- 172.21.15.78:0/2470381216 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39500c7350 con 0x55ca2047b6b0 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.986+0000 7f39427fc640 1 -- 172.21.15.78:0/2470381216 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500c6fb0 con 0x55ca2047b6b0 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 -- 172.21.15.78:0/2470381216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca2047b6b0 msgr2=0x55ca2083ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f3960fdf640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39440afaf0 0x7f39440b2000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55ca2083eab0 tx=0x7f3954009660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca2047b6b0 0x55ca2083ec70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3950107560 tx=0x7f3950107650 comp rx=0 tx=0).stop 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 -- 172.21.15.78:0/2470381216 shutdown_connections 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca2047b6b0 0x55ca2083ec70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca204734d0 0x55ca2083dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 --2- 172.21.15.78:0/2470381216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2083da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 -- 172.21.15.78:0/2470381216 >> 172.21.15.78:0/2470381216 conn(0x55ca204904e0 msgr2=0x55ca20839270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 -- 172.21.15.78:0/2470381216 shutdown_connections 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 -- 172.21.15.78:0/2470381216 wait complete. 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.987+0000 7f39699c8740 1 -- start start 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 1 Processor -- start 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca208476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca20847c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2084eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca2083f540 con 0x55ca204734d0 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ca20729d10 con 0x55ca2047b6b0 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ca2083f6c0 con 0x55ca2049c390 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f39699c8740 10 client.0 _ll_register_callbacks cb 0x55ca20756210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f395bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca208476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.988+0000 7f395bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca208476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57464/0 (socket says 172.21.15.78:57464) 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395bfff640 1 -- 172.21.15.78:0/796717988 learned_addr learned my addr 172.21.15.78:0/796717988 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f3960fdf640 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2084eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395b7fe640 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca20847c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395bfff640 1 -- 172.21.15.78:0/796717988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 msgr2=0x55ca2084eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395bfff640 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2084eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395bfff640 1 -- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 msgr2=0x55ca20847c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395bfff640 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca20847c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395bfff640 1 -- 172.21.15.78:0/796717988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2057e7d0 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f395bfff640 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca208476c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f394c009cc0 tx=0x7f394c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfc28995fad16e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f393b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f393b7fe640 1 -- 172.21.15.78:0/796717988 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c012860 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f393b7fe640 1 -- 172.21.15.78:0/796717988 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f394c012a00 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39500c7350 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f393b7fe640 1 -- 172.21.15.78:0/796717988 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f394c012c00 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f39699c8740 10 client.16362 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ca204b43f0 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.989+0000 7f393b7fe640 1 -- 172.21.15.78:0/796717988 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39500c7350 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f393b7fe640 1 -- 172.21.15.78:0/796717988 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f394c00f040 con 0x55ca204734d0 2024-04-25T14:02:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f39699c8740 10 client.16362 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f39699c8740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f39699c8740 20 client.16362 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f39699c8740 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2045af60 0x55ca20855100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f39699c8740 4 client.16362 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca207b0990 con 0x55ca2045af60 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f395b7fe640 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2045af60 0x55ca20855100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f393b7fe640 10 client.16362 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.990+0000 7f395b7fe640 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2045af60 0x55ca20855100 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5723f1f4980648 server_cookie=93ce66695813fa27 in_seq=0 out_seq=0 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f393b7fe640 1 -- 172.21.15.78:0/796717988 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca207b0990 con 0x55ca2045af60 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f393b7fe640 10 client.16362 handle_command_reply: tid=0 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 client.16362 shutdown 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 msgr2=0x55ca208476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca208476c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f394c009cc0 tx=0x7f394c0117b0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 shutdown_connections 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2045af60 0x55ca20855100 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2049c390 0x55ca2084eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca20847c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f393b7fe640 0 client.16362 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 --2- 172.21.15.78:0/796717988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca208476c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.991+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 >> 172.21.15.78:0/796717988 conn(0x55ca204904e0 msgr2=0x55ca208383f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.992+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 shutdown_connections 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.992+0000 7f39699c8740 1 -- 172.21.15.78:0/796717988 wait complete. 2024-04-25T14:02:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:31.992+0000 7f39699c8740 20 client.16362 trim_cache size 0 max 16384 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.009+0000 7f39699c8740 1 Processor -- start 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.009+0000 7f39699c8740 1 -- start start 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f39699c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2045af60 0x55ca20846350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f39699c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f39699c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca2083c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f39699c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1adbc0 con 0x55ca204734d0 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f39699c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1ab490 con 0x55ca2047b6b0 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f39699c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1a92e0 con 0x55ca2045af60 2024-04-25T14:02:32.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57476/0 (socket says 172.21.15.78:57476) 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 -- 172.21.15.78:0/3098484137 learned_addr learned my addr 172.21.15.78:0/3098484137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 -- 172.21.15.78:0/3098484137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2045af60 msgr2=0x55ca20846350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f3960fdf640 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca2083c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2045af60 0x55ca20846350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 -- 172.21.15.78:0/3098484137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 msgr2=0x55ca2083c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca2083c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 -- 172.21.15.78:0/3098484137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca204740f0 con 0x55ca204734d0 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f3960fdf640 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca2083c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f395b7fe640 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084ea80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f39500c4ab0 tx=0x7f395003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31449ea1db0376aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f3941ffb640 1 -- 172.21.15.78:0/3098484137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950049070 con 0x55ca204734d0 2024-04-25T14:02:32.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f3941ffb640 1 -- 172.21.15.78:0/3098484137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39500c7350 con 0x55ca204734d0 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.010+0000 7f3941ffb640 1 -- 172.21.15.78:0/3098484137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39500c7190 con 0x55ca204734d0 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 -- 172.21.15.78:0/3098484137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 msgr2=0x55ca2084ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084ea80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f39500c4ab0 tx=0x7f395003b6b0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 -- 172.21.15.78:0/3098484137 shutdown_connections 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2047b6b0 0x55ca2083c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084ea80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 --2- 172.21.15.78:0/3098484137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2045af60 0x55ca20846350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 -- 172.21.15.78:0/3098484137 >> 172.21.15.78:0/3098484137 conn(0x55ca204904e0 msgr2=0x55ca2083c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 -- 172.21.15.78:0/3098484137 shutdown_connections 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.011+0000 7f39699c8740 1 -- 172.21.15.78:0/3098484137 wait complete. 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 -- start start 2024-04-25T14:02:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 Processor -- start 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2045af60 0x55ca2084c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2047b6b0 0x55ca2084d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1a9950 con 0x55ca204734d0 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1aadc0 con 0x55ca2045af60 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f395c1aa020 con 0x55ca2047b6b0 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f39699c8740 10 client.0 _ll_register_callbacks cb 0x7f394c009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57482/0 (socket says 172.21.15.78:57482) 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 -- 172.21.15.78:0/2834688384 learned_addr learned my addr 172.21.15.78:0/2834688384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f3960fdf640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2047b6b0 0x55ca2084d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395bfff640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2045af60 0x55ca2084c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:32.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 -- 172.21.15.78:0/2834688384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2047b6b0 msgr2=0x55ca2084d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2047b6b0 0x55ca2084d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 -- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2045af60 msgr2=0x55ca2084c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2045af60 0x55ca2084c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395b7fe640 1 -- 172.21.15.78:0/2834688384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca207634b0 con 0x55ca204734d0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.012+0000 7f395bfff640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2045af60 0x55ca2084c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f395b7fe640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084c730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f39500057f0 tx=0x7f395002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434e6376918faf64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f393b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f393b7fe640 1 -- 172.21.15.78:0/2834688384 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950049070 con 0x55ca204734d0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f393b7fe640 1 -- 172.21.15.78:0/2834688384 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f39500c3e30 con 0x55ca204734d0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39500c7350 con 0x55ca204734d0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f393b7fe640 1 -- 172.21.15.78:0/2834688384 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f39500c7610 con 0x55ca204734d0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 10 client.16368 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ca2084ebe0 con 0x55ca204734d0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f393b7fe640 1 -- 172.21.15.78:0/2834688384 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39500e4080 con 0x55ca204734d0 2024-04-25T14:02:32.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f393b7fe640 1 -- 172.21.15.78:0/2834688384 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f39500aece0 con 0x55ca204734d0 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 10 client.16368 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 20 client.16368 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2049c390 0x55ca208bf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 4 client.16368 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.013+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca2083e3e0 con 0x55ca2049c390 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f395bfff640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2049c390 0x55ca208bf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f393b7fe640 10 client.16368 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f395bfff640 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2049c390 0x55ca208bf650 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b761b012ff5967c9 server_cookie=fe5ff3de218b30e1 in_seq=0 out_seq=0 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f393b7fe640 1 -- 172.21.15.78:0/2834688384 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ca2083e3e0 con 0x55ca2049c390 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f393b7fe640 10 client.16368 handle_command_reply: tid=0 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 client.16368 shutdown 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 msgr2=0x55ca2084c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:32.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084c730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f39500057f0 tx=0x7f395002a410 comp rx=0 tx=0).stop 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 shutdown_connections 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ca2049c390 0x55ca208bf650 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca2047b6b0 0x55ca2084d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca204734d0 0x55ca2084c730 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f393b7fe640 0 client.16368 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 --2- 172.21.15.78:0/2834688384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ca2045af60 0x55ca2084c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 >> 172.21.15.78:0/2834688384 conn(0x55ca204904e0 msgr2=0x55ca20847c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 shutdown_connections 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.014+0000 7f39699c8740 1 -- 172.21.15.78:0/2834688384 wait complete. 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f39699c8740 20 client.16368 trim_cache size 0 max 16384 2024-04-25T14:02:32.021 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: "f43bf553-f510-4376-9007-1ce24a7f5db1": { 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "f43bf553-f510-4376-9007-1ce24a7f5db1", 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:02:32.022 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:02:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39440afaf0 msgr2=0x7f39440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39440afaf0 0x7f39440b2000 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55ca2083eab0 tx=0x7f3954009660 comp rx=0 tx=0).stop 2024-04-25T14:02:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c102250 msgr2=0x7f395c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c102250 0x7f395c19a750 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3950006010 tx=0x7f3950004340 comp rx=0 tx=0).stop 2024-04-25T14:02:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 shutdown_connections 2024-04-25T14:02:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f395c1050a0 0x7f395c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39440afaf0 0x7f39440b2000 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c102250 0x7f395c19a750 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 --2- 172.21.15.78:0/2907792577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395c101750 0x7f395c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 >> 172.21.15.78:0/2907792577 conn(0x7f395c0fd350 msgr2=0x7f395c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 shutdown_connections 2024-04-25T14:02:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:02:32.015+0000 7f3962a42640 1 -- 172.21.15.78:0/2907792577 wait complete. 2024-04-25T14:02:32.026 INFO:tasks.cephfs.filesystem:scrub status for tag:f43bf553-f510-4376-9007-1ce24a7f5db1 - {'path': '/', 'tag': 'f43bf553-f510-4376-9007-1ce24a7f5db1', 'options': 'recursive,force'} 2024-04-25T14:02:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:32 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:02:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:32 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:32 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:02:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:32 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:32 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:02:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:32 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:32 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:02:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:32 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:32 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:02:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:32 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:32 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:02:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:32 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:02:33.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:33 smithi112 ceph-mon[32617]: pgmap v439: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:33.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:33 smithi129 ceph-mon[32569]: pgmap v439: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:33 smithi078 ceph-mon[29163]: pgmap v439: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:34 smithi112 ceph-mon[32617]: pgmap v440: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:34 smithi129 ceph-mon[32569]: pgmap v440: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:34 smithi078 ceph-mon[29163]: pgmap v440: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 -- 172.21.15.78:0/1855967649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4109a60 msgr2=0x7f90a410be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 --2- 172.21.15.78:0/1855967649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4109a60 0x7f90a410be80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f909c009ca0 tx=0x7f909c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 -- 172.21.15.78:0/1855967649 shutdown_connections 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 --2- 172.21.15.78:0/1855967649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4109a60 0x7f90a410be80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 --2- 172.21.15.78:0/1855967649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4104d60 0x7f90a4105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 --2- 172.21.15.78:0/1855967649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a4103b20 0x7f90a4104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 -- 172.21.15.78:0/1855967649 >> 172.21.15.78:0/1855967649 conn(0x7f90a40fed10 msgr2=0x7f90a41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 -- 172.21.15.78:0/1855967649 shutdown_connections 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.799+0000 7f90ac538640 1 -- 172.21.15.78:0/1855967649 wait complete. 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 Processor -- start 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 -- start start 2024-04-25T14:02:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4103b20 0x7f90a419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4104d60 0x7f90a419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a4109a60 0x7f90a41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f650 con 0x7f90a4104d60 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f4d0 con 0x7f90a4109a60 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90ac538640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f7d0 con 0x7f90a4103b20 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90a9ad3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4104d60 0x7f90a419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90a9ad3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4104d60 0x7f90a419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57496/0 (socket says 172.21.15.78:57496) 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.800+0000 7f90a9ad3640 1 -- 172.21.15.78:0/2351671702 learned_addr learned my addr 172.21.15.78:0/2351671702 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90aaad5640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a4109a60 0x7f90a41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4103b20 0x7f90a419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90a9ad3640 1 -- 172.21.15.78:0/2351671702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4103b20 msgr2=0x7f90a419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4103b20 0x7f90a419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90a9ad3640 1 -- 172.21.15.78:0/2351671702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a4109a60 msgr2=0x7f90a41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a4109a60 0x7f90a41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90a9ad3640 1 -- 172.21.15.78:0/2351671702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a41a1d00 con 0x7f90a4104d60 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4103b20 0x7f90a419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4104d60 0x7f90a419a700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f909400d770 tx=0x7f909400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f909b7fe640 1 -- 172.21.15.78:0/2351671702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094017070 con 0x7f90a4104d60 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f909b7fe640 1 -- 172.21.15.78:0/2351671702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90940054d0 con 0x7f90a4104d60 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a41a1ff0 con 0x7f90a4104d60 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f909b7fe640 1 -- 172.21.15.78:0/2351671702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094004c30 con 0x7f90a4104d60 2024-04-25T14:02:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.801+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a41a2460 con 0x7f90a4104d60 2024-04-25T14:02:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 Processor -- start 2024-04-25T14:02:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f909b7fe640 1 -- 172.21.15.78:0/2351671702 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9094004e10 con 0x7f90a4104d60 2024-04-25T14:02:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 -- start start 2024-04-25T14:02:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0291970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afec82f0 0x5633b0291ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b0298df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5633b01f7460 con 0x5633afef11b0 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633b01b1590 con 0x5633afed04d0 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90b33ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5633b01f3420 con 0x5633afec82f0 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90aaad5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0291970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90aaad5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0291970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57504/0 (socket says 172.21.15.78:57504) 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90aaad5640 1 -- 172.21.15.78:0/2047135342 learned_addr learned my addr 172.21.15.78:0/2047135342 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.804+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afec82f0 0x5633b0291ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b0298df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90aaad5640 1 -- 172.21.15.78:0/2047135342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afec82f0 msgr2=0x5633b0291ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90aaad5640 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afec82f0 0x5633b0291ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90aaad5640 1 -- 172.21.15.78:0/2047135342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 msgr2=0x5633b0298df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90aaad5640 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b0298df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90aaad5640 1 -- 172.21.15.78:0/2047135342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633b01be0d0 con 0x5633afef11b0 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f909b7fe640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f90900afb60 0x7f90900b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90aaad5640 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0291970 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f909c035000 tx=0x7f909c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8911cb15ba48e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f90900afb60 0x7f90900b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f909b7fe640 1 -- 172.21.15.78:0/2351671702 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f90940de670 con 0x7f90a4104d60 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f9098bfe640 1 -- 172.21.15.78:0/2047135342 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c031710 con 0x5633afef11b0 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f9098bfe640 1 -- 172.21.15.78:0/2047135342 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f909c036030 con 0x5633afef11b0 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.805+0000 7f9098bfe640 1 -- 172.21.15.78:0/2047135342 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c02f660 con 0x5633afef11b0 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 -- 172.21.15.78:0/2047135342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 msgr2=0x5633b0291970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0291970 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f909c035000 tx=0x7f909c0098b0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f90900afb60 0x7f90900b2070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5633b0292d50 tx=0x7f90a00094c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 -- 172.21.15.78:0/2047135342 shutdown_connections 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b0298df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afec82f0 0x5633b0291ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2047135342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0291970 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 -- 172.21.15.78:0/2047135342 >> 172.21.15.78:0/2047135342 conn(0x5633afee5300 msgr2=0x5633b028d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 -- 172.21.15.78:0/2047135342 shutdown_connections 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 -- 172.21.15.78:0/2047135342 wait complete. 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.806+0000 7f90b33ec740 1 -- start start 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 1 Processor -- start 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b029b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afed04d0 0x5633b029b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b02a2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5633b0299660 con 0x5633afed04d0 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633b017eb60 con 0x5633afec82f0 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5633b02997e0 con 0x5633afef11b0 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90b33ec740 10 client.0 _ll_register_callbacks cb 0x5633b01a32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afed04d0 0x5633b029b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afed04d0 0x5633b029b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57512/0 (socket says 172.21.15.78:57512) 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 -- 172.21.15.78:0/290162849 learned_addr learned my addr 172.21.15.78:0/290162849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b029b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90aaad5640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b02a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 -- 172.21.15.78:0/290162849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 msgr2=0x5633b02a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b02a2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 -- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 msgr2=0x5633b029b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b029b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 -- 172.21.15.78:0/290162849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633aff09210 con 0x5633afed04d0 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b029b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90aaad5640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b02a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afed04d0 0x5633b029b9a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9094009610 tx=0x7f90940a6370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60d86ba8bde2fef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.807+0000 7f908dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f908dffb640 1 -- 172.21.15.78:0/290162849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094020070 con 0x5633afed04d0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f908dffb640 1 -- 172.21.15.78:0/290162849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f90940a2dd0 con 0x5633afed04d0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f909c036030 con 0x5633afed04d0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f908dffb640 1 -- 172.21.15.78:0/290162849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f90940a7240 con 0x5633afed04d0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 10 client.16380 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5633affd5620 con 0x5633afed04d0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f908dffb640 1 -- 172.21.15.78:0/290162849 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f909c036030 con 0x5633afed04d0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f908dffb640 1 -- 172.21.15.78:0/290162849 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f90940af510 con 0x5633afed04d0 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 10 client.16380 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 20 client.16380 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afeafd80 0x5633b02a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 4 client.16380 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.808+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633b0205070 con 0x5633afeafd80 2024-04-25T14:02:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.809+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afeafd80 0x5633b02a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.809+0000 7f908dffb640 10 client.16380 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.809+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afeafd80 0x5633b02a91c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae44d8ef9aee529 server_cookie=a3b9cc476742a73 in_seq=0 out_seq=0 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f908dffb640 1 -- 172.21.15.78:0/290162849 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633b0205070 con 0x5633afeafd80 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f908dffb640 10 client.16380 handle_command_reply: tid=0 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 client.16380 shutdown 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afed04d0 msgr2=0x5633b029b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afed04d0 0x5633b029b9a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9094009610 tx=0x7f90940a6370 comp rx=0 tx=0).stop 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 shutdown_connections 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afeafd80 0x5633b02a91c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b02a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afed04d0 0x5633b029b9a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f908dffb640 0 client.16380 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 --2- 172.21.15.78:0/290162849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b029b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 >> 172.21.15.78:0/290162849 conn(0x5633afee5300 msgr2=0x5633b0290210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 shutdown_connections 2024-04-25T14:02:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 1 -- 172.21.15.78:0/290162849 wait complete. 2024-04-25T14:02:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.810+0000 7f90b33ec740 20 client.16380 trim_cache size 0 max 16384 2024-04-25T14:02:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 Processor -- start 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 -- start start 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afec82f0 0x5633b028cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b028d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b0290100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41adaa0 con 0x5633afec82f0 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41ab370 con 0x5633afed04d0 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90b33ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41a91c0 con 0x5633afef11b0 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90aa2d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afec82f0 0x5633b028cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90aa2d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afec82f0 0x5633b028cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57526/0 (socket says 172.21.15.78:57526) 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.825+0000 7f90aa2d4640 1 -- 172.21.15.78:0/3165480753 learned_addr learned my addr 172.21.15.78:0/3165480753 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90aaad5640 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b0290100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b028d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90aa2d4640 1 -- 172.21.15.78:0/3165480753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 msgr2=0x5633b0290100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b0290100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90aa2d4640 1 -- 172.21.15.78:0/3165480753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 msgr2=0x5633b028d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b028d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90aa2d4640 1 -- 172.21.15.78:0/3165480753 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633b02a5ad0 con 0x5633afec82f0 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b028d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afec82f0 0x5633b028cc10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009fd0 tx=0x7f90a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867dc721bd4409ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f908ffff640 1 -- 172.21.15.78:0/3165480753 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0020900 con 0x5633afec82f0 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f908ffff640 1 -- 172.21.15.78:0/3165480753 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a0029330 con 0x5633afec82f0 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.826+0000 7f908ffff640 1 -- 172.21.15.78:0/3165480753 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a001f660 con 0x5633afec82f0 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- 172.21.15.78:0/3165480753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afec82f0 msgr2=0x5633b028cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afec82f0 0x5633b028cc10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009fd0 tx=0x7f90a0014000 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- 172.21.15.78:0/3165480753 shutdown_connections 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afef11b0 0x5633b0290100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afed04d0 0x5633b028d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 --2- 172.21.15.78:0/3165480753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afec82f0 0x5633b028cc10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- 172.21.15.78:0/3165480753 >> 172.21.15.78:0/3165480753 conn(0x5633afee5300 msgr2=0x5633b0295ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- 172.21.15.78:0/3165480753 shutdown_connections 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- 172.21.15.78:0/3165480753 wait complete. 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- start start 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 Processor -- start 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b030cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afed04d0 0x5633b030d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0314150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41a9830 con 0x5633afef11b0 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41aaca0 con 0x5633afec82f0 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90a41a9f00 con 0x5633afed04d0 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90b33ec740 10 client.0 _ll_register_callbacks cb 0x7f909400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.827+0000 7f90aaad5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0314150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0314150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57534/0 (socket says 172.21.15.78:57534) 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 -- 172.21.15.78:0/2703470352 learned_addr learned my addr 172.21.15.78:0/2703470352 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90a9ad3640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afed04d0 0x5633b030d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b030cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 -- 172.21.15.78:0/2703470352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afed04d0 msgr2=0x5633b030d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afed04d0 0x5633b030d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 -- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 msgr2=0x5633b030cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b030cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 -- 172.21.15.78:0/2703470352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633b0314710 con 0x5633afef11b0 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b030cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90aaad5640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0314150 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f909c009920 tx=0x7f909c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95842ece5c2b981a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f908dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f908dffb640 1 -- 172.21.15.78:0/2703470352 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c052070 con 0x5633afef11b0 2024-04-25T14:02:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f908dffb640 1 -- 172.21.15.78:0/2703470352 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f909c02f660 con 0x5633afef11b0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a0029330 con 0x5633afef11b0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f908dffb640 1 -- 172.21.15.78:0/2703470352 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f909c05e050 con 0x5633afef11b0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90b33ec740 10 client.16386 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5633b0315910 con 0x5633afef11b0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.828+0000 7f908dffb640 1 -- 172.21.15.78:0/2703470352 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a0029330 con 0x5633afef11b0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f908dffb640 1 -- 172.21.15.78:0/2703470352 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f909c043070 con 0x5633afef11b0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90b33ec740 10 client.16386 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90b33ec740 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90b33ec740 20 client.16386 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afe851c0 0x5633b0317ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90b33ec740 4 client.16386 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633b01feee0 con 0x5633afe851c0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afe851c0 0x5633b0317ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f908dffb640 10 client.16386 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.829+0000 7f90aa2d4640 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afe851c0 0x5633b0317ff0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef6a28748791254 server_cookie=326aa39b248c52d1 in_seq=0 out_seq=0 2024-04-25T14:02:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f908dffb640 1 -- 172.21.15.78:0/2703470352 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633b01feee0 con 0x5633afe851c0 2024-04-25T14:02:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f908dffb640 10 client.16386 handle_command_reply: tid=0 2024-04-25T14:02:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 client.16386 shutdown 2024-04-25T14:02:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 msgr2=0x5633b0314150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0314150 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f909c009920 tx=0x7f909c003ba0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 shutdown_connections 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633afef11b0 0x5633b0314150 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5633afe851c0 0x5633b0317ff0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633afed04d0 0x5633b030d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 --2- 172.21.15.78:0/2703470352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633afec82f0 0x5633b030cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f908dffb640 0 client.16386 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 >> 172.21.15.78:0/2703470352 conn(0x5633afee5300 msgr2=0x5633b0290100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 shutdown_connections 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 1 -- 172.21.15.78:0/2703470352 wait complete. 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.843+0000 7f90b33ec740 20 client.16386 trim_cache size 0 max 16384 2024-04-25T14:02:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:02:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:02:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:02:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:02:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f90900afb60 msgr2=0x7f90900b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f90900afb60 0x7f90900b2070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5633b0292d50 tx=0x7f90a00094c0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4104d60 msgr2=0x7f90a419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4104d60 0x7f90a419a700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f909400d770 tx=0x7f909400dc30 comp rx=0 tx=0).stop 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 shutdown_connections 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90a4109a60 0x7f90a41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f90900afb60 0x7f90900b2070 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a4104d60 0x7f90a419a700 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 --2- 172.21.15.78:0/2351671702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a4103b20 0x7f90a419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 >> 172.21.15.78:0/2351671702 conn(0x7f90a40fed10 msgr2=0x7f90a4105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 shutdown_connections 2024-04-25T14:02:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:34.844+0000 7f90ac538640 1 -- 172.21.15.78:0/2351671702 wait complete. 2024-04-25T14:02:36.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:36 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:02:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:36 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:02:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:36 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:02:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:37 smithi112 ceph-mon[32617]: pgmap v441: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:02:37.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:37 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:02:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:37 smithi129 ceph-mon[32569]: pgmap v441: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:02:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:37 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:02:37.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:37 smithi078 ceph-mon[29163]: pgmap v441: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:02:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:37 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:02:38.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:02:38.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:02:38.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:02:38.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:02:38.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:02:38.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:02:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:02:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:02:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:02:39.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:39 smithi112 ceph-mon[32617]: pgmap v442: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:39.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:39 smithi129 ceph-mon[32569]: pgmap v442: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:39 smithi078 ceph-mon[29163]: pgmap v442: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:40 smithi129 ceph-mon[32569]: pgmap v443: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:40 smithi078 ceph-mon[29163]: pgmap v443: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:40 smithi112 ceph-mon[32617]: pgmap v443: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:42.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:42 smithi129 ceph-mon[32569]: pgmap v444: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:02:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:42 smithi078 ceph-mon[29163]: pgmap v444: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:02:42.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:42 smithi112 ceph-mon[32617]: pgmap v444: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:02:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:44 smithi129 ceph-mon[32569]: pgmap v445: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:44 smithi078 ceph-mon[29163]: pgmap v445: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:44 smithi112 ceph-mon[32617]: pgmap v445: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:02:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:46 smithi129 ceph-mon[32569]: pgmap v446: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:46 smithi078 ceph-mon[29163]: pgmap v446: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:46 smithi112 ceph-mon[32617]: pgmap v446: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:02:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 -- 172.21.15.78:0/4078936217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 msgr2=0x7fbb1410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 --2- 172.21.15.78:0/4078936217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb1410ad80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009ca0 tx=0x7fbb0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 -- 172.21.15.78:0/4078936217 shutdown_connections 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 --2- 172.21.15.78:0/4078936217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb1410ad80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 --2- 172.21.15.78:0/4078936217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14103c60 0x7fbb14104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 --2- 172.21.15.78:0/4078936217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14102a20 0x7fbb14103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 -- 172.21.15.78:0/4078936217 >> 172.21.15.78:0/4078936217 conn(0x7fbb140fdc10 msgr2=0x7fbb141000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 -- 172.21.15.78:0/4078936217 shutdown_connections 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 -- 172.21.15.78:0/4078936217 wait complete. 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 Processor -- start 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.955+0000 7fbb1936c640 1 -- start start 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb1936c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14102a20 0x7fbb1406ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb1936c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14103c60 0x7fbb1419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb1936c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb141a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb1936c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410e530 con 0x7fbb14108960 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb1936c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410e3b0 con 0x7fbb14103c60 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb1936c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410e6b0 con 0x7fbb14102a20 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb141a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb141a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43724/0 (socket says 172.21.15.78:43724) 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 -- 172.21.15.78:0/365153208 learned_addr learned my addr 172.21.15.78:0/365153208 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14103c60 0x7fbb1419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14102a20 0x7fbb1406ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 -- 172.21.15.78:0/365153208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14102a20 msgr2=0x7fbb1406ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14102a20 0x7fbb1406ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 -- 172.21.15.78:0/365153208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14103c60 msgr2=0x7fbb1419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14103c60 0x7fbb1419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 -- 172.21.15.78:0/365153208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb141a1cd0 con 0x7fbb14108960 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14103c60 0x7fbb1419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.956+0000 7fbb137fe640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb141a14b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097b0 tx=0x7fbb0c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.957+0000 7fbb10ff9640 1 -- 172.21.15.78:0/365153208 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c004240 con 0x7fbb14108960 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.957+0000 7fbb10ff9640 1 -- 172.21.15.78:0/365153208 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c02f380 con 0x7fbb14108960 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.957+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb141a1f60 con 0x7fbb14108960 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.957+0000 7fbb10ff9640 1 -- 172.21.15.78:0/365153208 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c02fc30 con 0x7fbb14108960 2024-04-25T14:02:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.957+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141a23c0 con 0x7fbb14108960 2024-04-25T14:02:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.959+0000 7fbb2021b740 1 Processor -- start 2024-04-25T14:02:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.959+0000 7fbb10ff9640 1 -- 172.21.15.78:0/365153208 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbb0c0043e0 con 0x7fbb14108960 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb2021b740 1 -- start start 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa128cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1293df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb2021b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dfa11f2460 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb2021b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dfa11ac590 con 0x55dfa0eec1b0 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb2021b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dfa11ee420 con 0x55dfa0ec32f0 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1293df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1293df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43730/0 (socket says 172.21.15.78:43730) 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb0bfff640 1 -- 172.21.15.78:0/773802655 learned_addr learned my addr 172.21.15.78:0/773802655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa128cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb10ff9640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbb000afae0 0x7fbb000b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.960+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb137fe640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbb000afae0 0x7fbb000b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb0bfff640 1 -- 172.21.15.78:0/773802655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 msgr2=0x55dfa128cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa128cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb0bfff640 1 -- 172.21.15.78:0/773802655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 msgr2=0x55dfa128c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb0bfff640 1 -- 172.21.15.78:0/773802655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfa11b90d0 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb10ff9640 1 -- 172.21.15.78:0/365153208 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbb0c100440 con 0x7fbb14108960 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1293df0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0139a0 tx=0x7fbafc00c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3d85cb2543002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb097fa640 1 -- 172.21.15.78:0/773802655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0119e0 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb137fe640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbb000afae0 0x7fbb000b1ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0fdf80 tx=0x7fbb0c093fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb097fa640 1 -- 172.21.15.78:0/773802655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc00f480 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.961+0000 7fbb097fa640 1 -- 172.21.15.78:0/773802655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc01e600 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 -- 172.21.15.78:0/773802655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 msgr2=0x55dfa1293df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1293df0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0139a0 tx=0x7fbafc00c6f0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 -- 172.21.15.78:0/773802655 shutdown_connections 2024-04-25T14:02:47.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1293df0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa128cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 --2- 172.21.15.78:0/773802655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 -- 172.21.15.78:0/773802655 >> 172.21.15.78:0/773802655 conn(0x55dfa0ee0300 msgr2=0x55dfa1288300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 -- 172.21.15.78:0/773802655 shutdown_connections 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 -- 172.21.15.78:0/773802655 wait complete. 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.962+0000 7fbb2021b740 1 -- start start 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 1 Processor -- start 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa129dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dfa1294840 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dfa1179b60 con 0x55dfa0eec1b0 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dfa12949c0 con 0x55dfa0ec32f0 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb2021b740 10 client.0 _ll_register_callbacks cb 0x55dfa119e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43734/0 (socket says 172.21.15.78:43734) 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 -- 172.21.15.78:0/386184516 learned_addr learned my addr 172.21.15.78:0/386184516 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa129dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 -- 172.21.15.78:0/386184516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 msgr2=0x55dfa1296620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 -- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 msgr2=0x55dfa129dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa129dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 -- 172.21.15.78:0/386184516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfa0f04210 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400dfc0 tx=0x7fbb040128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1b22402261643a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.963+0000 7fbafa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbafa7fc640 1 -- 172.21.15.78:0/386184516 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04005220 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbafa7fc640 1 -- 172.21.15.78:0/386184516 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbb040053c0 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc00f480 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbafa7fc640 1 -- 172.21.15.78:0/386184516 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbb040055c0 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 10 client.16398 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55dfa0fd0620 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbafa7fc640 1 -- 172.21.15.78:0/386184516 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc00f480 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbafa7fc640 1 -- 172.21.15.78:0/386184516 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbb04019030 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 10 client.16398 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 20 client.16398 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0eaad80 0x55dfa12a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 4 client.16398 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:02:47.974 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfa1200070 con 0x55dfa0eaad80 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.964+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0eaad80 0x55dfa12a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.965+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0eaad80 0x55dfa12a4480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154285afafa40c56 server_cookie=fcedd2744f7448c1 in_seq=0 out_seq=0 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.965+0000 7fbafa7fc640 10 client.16398 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbafa7fc640 1 -- 172.21.15.78:0/386184516 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dfa1200070 con 0x55dfa0eaad80 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbafa7fc640 10 client.16398 handle_command_reply: tid=0 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 client.16398 shutdown 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 msgr2=0x55dfa1296b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400dfc0 tx=0x7fbb040128e0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 shutdown_connections 2024-04-25T14:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0eec1b0 0x55dfa129dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0eaad80 0x55dfa12a4480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 --2- 172.21.15.78:0/386184516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 >> 172.21.15.78:0/386184516 conn(0x55dfa0ee0300 msgr2=0x55dfa128b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbafa7fc640 0 client.16398 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 shutdown_connections 2024-04-25T14:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 1 -- 172.21.15.78:0/386184516 wait complete. 2024-04-25T14:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.966+0000 7fbb2021b740 20 client.16398 trim_cache size 0 max 16384 2024-04-25T14:02:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.981+0000 7fbb2021b740 1 Processor -- start 2024-04-25T14:02:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.981+0000 7fbb2021b740 1 -- start start 2024-04-25T14:02:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb2021b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141adad0 con 0x55dfa0eec1b0 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb2021b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141ab3a0 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb2021b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141a91f0 con 0x55dfa0ec32f0 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43746/0 (socket says 172.21.15.78:43746) 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 -- 172.21.15.78:0/3617591799 learned_addr learned my addr 172.21.15.78:0/3617591799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 -- 172.21.15.78:0/3617591799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 msgr2=0x55dfa1296620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 -- 172.21.15.78:0/3617591799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 msgr2=0x55dfa1296b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 -- 172.21.15.78:0/3617591799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfa12941b0 con 0x55dfa0eec1b0 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c8f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0137e0 tx=0x7fbafc00c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69b6f91d308beb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.982+0000 7fbb08ff9640 1 -- 172.21.15.78:0/3617591799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc013040 con 0x55dfa0eec1b0 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb08ff9640 1 -- 172.21.15.78:0/3617591799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc0131e0 con 0x55dfa0eec1b0 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb08ff9640 1 -- 172.21.15.78:0/3617591799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00f9f0 con 0x55dfa0eec1b0 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 -- 172.21.15.78:0/3617591799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 msgr2=0x55dfa128c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c8f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0137e0 tx=0x7fbafc00c6f0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 -- 172.21.15.78:0/3617591799 shutdown_connections 2024-04-25T14:02:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa128c8f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa1296b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 --2- 172.21.15.78:0/3617591799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa1296620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 -- 172.21.15.78:0/3617591799 >> 172.21.15.78:0/3617591799 conn(0x55dfa0ee0300 msgr2=0x55dfa1282f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 -- 172.21.15.78:0/3617591799 shutdown_connections 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.983+0000 7fbb2021b740 1 -- 172.21.15.78:0/3617591799 wait complete. 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 -- start start 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 Processor -- start 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa129a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa129a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa130a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141a9860 con 0x55dfa0eec1b0 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141aacd0 con 0x55dfa0ecb4d0 2024-04-25T14:02:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141a9f30 con 0x55dfa0ec32f0 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb2021b740 10 client.0 _ll_register_callbacks cb 0x7fbb0400d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa130a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa130a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43760/0 (socket says 172.21.15.78:43760) 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb0bfff640 1 -- 172.21.15.78:0/4247099336 learned_addr learned my addr 172.21.15.78:0/4247099336 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa129a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.984+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa129a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb0bfff640 1 -- 172.21.15.78:0/4247099336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 msgr2=0x55dfa129a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa129a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb0bfff640 1 -- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 msgr2=0x55dfa129a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa129a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb0bfff640 1 -- 172.21.15.78:0/4247099336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfa129ab80 con 0x55dfa0eec1b0 2024-04-25T14:02:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa129a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb0bfff640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa130a820 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0058c0 tx=0x7fbafc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f981bbb782a5f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbafa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbafa7fc640 1 -- 172.21.15.78:0/4247099336 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc013390 con 0x55dfa0eec1b0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbafa7fc640 1 -- 172.21.15.78:0/4247099336 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbafc013530 con 0x55dfa0eec1b0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc0131e0 con 0x55dfa0eec1b0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbafa7fc640 1 -- 172.21.15.78:0/4247099336 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbafc00f770 con 0x55dfa0eec1b0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb2021b740 10 client.16404 fetch_fsmap learned FSMap version 22 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55dfa130be20 con 0x55dfa0eec1b0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.985+0000 7fbafa7fc640 1 -- 172.21.15.78:0/4247099336 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc0131e0 con 0x55dfa0eec1b0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbafa7fc640 1 -- 172.21.15.78:0/4247099336 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbafc041050 con 0x55dfa0eec1b0 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb2021b740 10 client.16404 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb2021b740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:02:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb2021b740 20 client.16404 populate_metadata read hostname 'smithi078' 2024-04-25T14:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb2021b740 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0e801c0 0x55dfa130e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb2021b740 4 client.16404 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfa11f9ee0 con 0x55dfa0e801c0 2024-04-25T14:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0e801c0 0x55dfa130e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbafa7fc640 10 client.16404 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:47.986+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0e801c0 0x55dfa130e650 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a7d40e50076ffa server_cookie=e354b0cf0cf8c428 in_seq=0 out_seq=0 2024-04-25T14:02:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbafa7fc640 1 -- 172.21.15.78:0/4247099336 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfa11f9ee0 con 0x55dfa0e801c0 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbafa7fc640 10 client.16404 handle_command_reply: tid=0 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 client.16404 shutdown 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 msgr2=0x55dfa130a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa130a820 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0058c0 tx=0x7fbafc00cf30 comp rx=0 tx=0).stop 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 shutdown_connections 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dfa0eec1b0 0x55dfa130a820 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dfa0ecb4d0 0x55dfa129a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dfa0e801c0 0x55dfa130e650 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 --2- 172.21.15.78:0/4247099336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfa0ec32f0 0x55dfa129a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbafa7fc640 0 client.16404 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.000+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 >> 172.21.15.78:0/4247099336 conn(0x55dfa0ee0300 msgr2=0x55dfa1282f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 shutdown_connections 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb2021b740 1 -- 172.21.15.78:0/4247099336 wait complete. 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb2021b740 20 client.16404 trim_cache size 0 max 16384 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:02:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:02:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:02:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbb000afae0 msgr2=0x7fbb000b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbb000afae0 0x7fbb000b1ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0fdf80 tx=0x7fbb0c093fd0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 msgr2=0x7fbb141a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb141a14b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097b0 tx=0x7fbb0c003c20 comp rx=0 tx=0).stop 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 shutdown_connections 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbb000afae0 0x7fbb000b1ff0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14108960 0x7fbb141a14b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14103c60 0x7fbb1419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 --2- 172.21.15.78:0/365153208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14102a20 0x7fbb1406ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 >> 172.21.15.78:0/365153208 conn(0x7fbb140fdc10 msgr2=0x7fbb14104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 shutdown_connections 2024-04-25T14:02:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:02:48.001+0000 7fbb1936c640 1 -- 172.21.15.78:0/365153208 wait complete. 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:64/600: open fd = 3 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:65/600: open fd = 3 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.376 INFO:tasks.workunit.client.0.smithi078.stdout:66/600: open fd = 3 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:67/600: open fd = 3 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:68/600: open fd = 3 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:69/600: open fd = 3 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.377 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:70/600: open fd = 3 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:71/600: open fd = 3 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:72/600: open fd = 3 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.378 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:73/600: open fd = 3 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:74/600: open fd = 3 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:75/600: open fd = 3 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:76/600: open fd = 3 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.379 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:77/600: open fd = 3 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:78/600: open fd = 3 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:79/600: open fd = 3 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.380 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:80/600: open fd = 3 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:81/600: open fd = 3 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:82/600: open fd = 3 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:83/600: open fd = 3 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.381 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:84/600: open fd = 3 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:85/600: open fd = 3 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:86/600: open fd = 3 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.382 INFO:tasks.workunit.client.0.smithi078.stdout:87/600: open fd = 3 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:88/600: open fd = 3 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:89/600: open fd = 3 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:90/600: open fd = 3 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.383 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:91/600: open fd = 3 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:92/600: open fd = 3 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:93/600: open fd = 3 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.384 INFO:tasks.workunit.client.0.smithi078.stdout:94/600: open fd = 3 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:95/600: open fd = 3 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:96/600: open fd = 3 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:97/600: open fd = 3 2024-04-25T14:02:48.385 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:98/600: open fd = 3 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:99/600: open fd = 3 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:100/600: open fd = 3 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.386 INFO:tasks.workunit.client.0.smithi078.stdout:101/600: open fd = 3 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:102/600: open fd = 3 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:103/600: open fd = 3 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:104/600: open fd = 3 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.387 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:105/600: open fd = 3 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:106/600: open fd = 3 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:107/600: open fd = 3 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:108/600: open fd = 3 2024-04-25T14:02:48.388 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:109/600: open fd = 3 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:110/600: open fd = 3 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:111/600: open fd = 3 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.389 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:112/600: open fd = 3 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:113/600: open fd = 3 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:114/600: open fd = 3 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.390 INFO:tasks.workunit.client.0.smithi078.stdout:115/600: open fd = 3 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:116/600: open fd = 3 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:117/600: open fd = 3 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:118/600: open fd = 3 2024-04-25T14:02:48.391 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:119/600: open fd = 3 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:120/600: open fd = 3 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:121/600: open fd = 3 2024-04-25T14:02:48.392 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:122/600: open fd = 3 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:123/600: open fd = 3 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:124/600: open fd = 3 2024-04-25T14:02:48.393 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:125/600: open fd = 3 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:126/600: open fd = 3 2024-04-25T14:02:48.394 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:02:48.667 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:48 smithi129 ceph-mon[32569]: pgmap v447: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:02:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:48 smithi078 ceph-mon[29163]: pgmap v447: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:02:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:48 smithi112 ceph-mon[32617]: pgmap v447: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:02:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:50 smithi129 ceph-mon[32569]: pgmap v448: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:02:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:50 smithi078 ceph-mon[29163]: pgmap v448: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:02:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:50 smithi112 ceph-mon[32617]: pgmap v448: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:02:52.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:52 smithi112 ceph-mon[32617]: pgmap v449: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:52 smithi129 ceph-mon[32569]: pgmap v449: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:52 smithi078 ceph-mon[29163]: pgmap v449: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:02:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:54 smithi129 ceph-mon[32569]: pgmap v450: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:54 smithi078 ceph-mon[29163]: pgmap v450: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:54.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:54 smithi112 ceph-mon[32617]: pgmap v450: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:02:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:56 smithi129 ceph-mon[32569]: pgmap v451: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:02:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:56 smithi078 ceph-mon[29163]: pgmap v451: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:02:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:56 smithi112 ceph-mon[32617]: pgmap v451: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:02:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:02:58 smithi129 ceph-mon[32569]: pgmap v452: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:02:58 smithi078 ceph-mon[29163]: pgmap v452: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:02:58.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:02:58 smithi112 ceph-mon[32617]: pgmap v452: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:03:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:00 smithi129 ceph-mon[32569]: pgmap v453: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:03:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:00 smithi078 ceph-mon[29163]: pgmap v453: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:03:00.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:00 smithi112 ceph-mon[32617]: pgmap v453: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/1866237930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14103b50 msgr2=0x7fbb14104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/1866237930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14103b50 0x7fbb14104850 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009900 tx=0x7fbb0402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/1866237930 shutdown_connections 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/1866237930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14109a90 0x7fbb1410beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/1866237930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14104d90 0x7fbb141051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/1866237930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14103b50 0x7fbb14104850 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/1866237930 >> 172.21.15.78:0/1866237930 conn(0x7fbb140fed20 msgr2=0x7fbb141011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/1866237930 shutdown_connections 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/1866237930 wait complete. 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.107+0000 7fbb1a0f9640 1 Processor -- start 2024-04-25T14:03:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb1a0f9640 1 -- start start 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb1a0f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14103b50 0x7fbb1419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb1a0f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14104d90 0x7fbb1419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb1a0f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14109a90 0x7fbb141a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb1a0f9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410f650 con 0x7fbb14109a90 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb1a0f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410f4d0 con 0x7fbb14104d90 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb1a0f9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410f7d0 con 0x7fbb14103b50 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14109a90 0x7fbb141a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14109a90 0x7fbb141a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45086/0 (socket says 172.21.15.78:45086) 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 -- 172.21.15.78:0/3204034800 learned_addr learned my addr 172.21.15.78:0/3204034800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb137fe640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14103b50 0x7fbb1419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14104d90 0x7fbb1419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 -- 172.21.15.78:0/3204034800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14103b50 msgr2=0x7fbb1419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14103b50 0x7fbb1419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 -- 172.21.15.78:0/3204034800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14104d90 msgr2=0x7fbb1419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14104d90 0x7fbb1419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 -- 172.21.15.78:0/3204034800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb141a1d80 con 0x7fbb14109a90 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.108+0000 7fbb13fff640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14109a90 0x7fbb141a1710 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00cad0 tx=0x7fbb0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.109+0000 7fbb10ff9640 1 -- 172.21.15.78:0/3204034800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00ae20 con 0x7fbb14109a90 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.109+0000 7fbb10ff9640 1 -- 172.21.15.78:0/3204034800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c005020 con 0x7fbb14109a90 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.109+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb141a2070 con 0x7fbb14109a90 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.109+0000 7fbb10ff9640 1 -- 172.21.15.78:0/3204034800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00f6b0 con 0x7fbb14109a90 2024-04-25T14:03:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.109+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141a25c0 con 0x7fbb14109a90 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.111+0000 7fbb20fa9740 1 Processor -- start 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.111+0000 7fbb10ff9640 1 -- 172.21.15.78:0/3204034800 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbb0c00f850 con 0x7fbb14109a90 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.111+0000 7fbb20fa9740 1 -- start start 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e7090970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6cc72f0 0x5630e7090ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ccf4d0 0x5630e7097df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630e6ff6460 con 0x5630e6cf01b0 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630e6fb0590 con 0x5630e6cc72f0 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630e6ff2420 con 0x5630e6ccf4d0 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e7090970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e7090970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45102/0 (socket says 172.21.15.78:45102) 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1420650801 learned_addr learned my addr 172.21.15.78:0/1420650801 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb137fe640 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6cc72f0 0x5630e7090ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb10ff9640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbafc0afae0 0x7fbafc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ccf4d0 0x5630e7097df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1420650801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ccf4d0 msgr2=0x5630e7097df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ccf4d0 0x5630e7097df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1420650801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6cc72f0 msgr2=0x5630e7090ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6cc72f0 0x5630e7090ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1420650801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e6fbd0d0 con 0x5630e6cf01b0 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb13fff640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbafc0afae0 0x7fbafc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e7090970 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbb08012470 tx=0x7fbb0800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b70d83eed477f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbb10ff9640 1 -- 172.21.15.78:0/3204034800 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbb0c0dec30 con 0x7fbb14109a90 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.112+0000 7fbaf9ffb640 1 -- 172.21.15.78:0/1420650801 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08018030 con 0x5630e6cf01b0 2024-04-25T14:03:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.113+0000 7fbaf9ffb640 1 -- 172.21.15.78:0/1420650801 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb08010af0 con 0x5630e6cf01b0 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.113+0000 7fbaf9ffb640 1 -- 172.21.15.78:0/1420650801 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0801c9a0 con 0x5630e6cf01b0 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.113+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1420650801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 msgr2=0x5630e7090970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.113+0000 7fbb13fff640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbafc0afae0 0x7fbafc0b1ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c01a000 tx=0x7fbb0c074040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.113+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e7090970 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbb08012470 tx=0x7fbb0800c5a0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1420650801 shutdown_connections 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ccf4d0 0x5630e7097df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6cc72f0 0x5630e7090ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1420650801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e7090970 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1420650801 >> 172.21.15.78:0/1420650801 conn(0x5630e6ce4300 msgr2=0x5630e708c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1420650801 shutdown_connections 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1420650801 wait complete. 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- start start 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 Processor -- start 2024-04-25T14:03:01.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e709a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 0x5630e709ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e70a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630e7098840 con 0x5630e6cf01b0 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5630e6f7db60 con 0x5630e6ccf4d0 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630e70989c0 con 0x5630e6cc72f0 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.114+0000 7fbb20fa9740 10 client.0 _ll_register_callbacks cb 0x5630e6fa22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e70a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e70a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45114/0 (socket says 172.21.15.78:45114) 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1417733943 learned_addr learned my addr 172.21.15.78:0/1417733943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb137fe640 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 0x5630e709ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e709a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1417733943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 msgr2=0x5630e709a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e709a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 msgr2=0x5630e709ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 0x5630e709ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 -- 172.21.15.78:0/1417733943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e6d08210 con 0x5630e6cf01b0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e70a1ac0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800c3e0 tx=0x7fbb0800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d3502c5b1e6907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbaf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/1417733943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08018040 con 0x5630e6cf01b0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/1417733943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbb080108f0 con 0x5630e6cf01b0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb08010af0 con 0x5630e6cf01b0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/1417733943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbb0801c760 con 0x5630e6cf01b0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb20fa9740 10 client.16416 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.115+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5630e6dd4620 con 0x5630e6cf01b0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/1417733943 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb08010af0 con 0x5630e6cf01b0 2024-04-25T14:03:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/1417733943 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbb080359e0 con 0x5630e6cf01b0 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbb20fa9740 10 client.16416 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbb20fa9740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbb20fa9740 20 client.16416 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6caed80 0x5630e70a8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbb20fa9740 4 client.16416 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630e7004070 con 0x5630e6caed80 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.116+0000 7fbb137fe640 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6caed80 0x5630e70a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.117+0000 7fbaf2ffd640 10 client.16416 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.117+0000 7fbb137fe640 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6caed80 0x5630e70a8020 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53be41bb10a30d8d server_cookie=dd45e4ca3a4b357f in_seq=0 out_seq=0 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.117+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/1417733943 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630e7004070 con 0x5630e6caed80 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.117+0000 7fbaf2ffd640 10 client.16416 handle_command_reply: tid=0 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.117+0000 7fbb20fa9740 1 client.16416 shutdown 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.117+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 msgr2=0x5630e70a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.117+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e70a1ac0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800c3e0 tx=0x7fbb0800c7e0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 shutdown_connections 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cf01b0 0x5630e70a1ac0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 0x5630e709ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6caed80 0x5630e70a8020 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/1417733943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e709a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbaf2ffd640 0 client.16416 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 >> 172.21.15.78:0/1417733943 conn(0x5630e6ce4300 msgr2=0x5630e708f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 shutdown_connections 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 1 -- 172.21.15.78:0/1417733943 wait complete. 2024-04-25T14:03:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.118+0000 7fbb20fa9740 20 client.16416 trim_cache size 0 max 16384 2024-04-25T14:03:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.132+0000 7fbb20fa9740 1 Processor -- start 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb20fa9740 1 -- start start 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ce4300 0x5630e70ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cc72f0 0x5630e70add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 0x5630e70a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141adc40 con 0x5630e6cc72f0 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141ab510 con 0x5630e6ccf4d0 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141a9360 con 0x5630e6ce4300 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cc72f0 0x5630e70add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cc72f0 0x5630e70add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45120/0 (socket says 172.21.15.78:45120) 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 -- 172.21.15.78:0/2692338779 learned_addr learned my addr 172.21.15.78:0/2692338779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 -- 172.21.15.78:0/2692338779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ce4300 msgr2=0x5630e70ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 --2- 172.21.15.78:0/2692338779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ce4300 0x5630e70ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 -- 172.21.15.78:0/2692338779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 msgr2=0x5630e70a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 --2- 172.21.15.78:0/2692338779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 0x5630e70a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 -- 172.21.15.78:0/2692338779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e70a1f20 con 0x5630e6cc72f0 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbb137fe640 1 --2- 172.21.15.78:0/2692338779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cc72f0 0x5630e70add40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009900 tx=0x7fbb04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbff8b963c4f7d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbaf97fa640 1 -- 172.21.15.78:0/2692338779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0404b020 con 0x5630e6cc72f0 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbaf97fa640 1 -- 172.21.15.78:0/2692338779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb04004380 con 0x5630e6cc72f0 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.133+0000 7fbaf97fa640 1 -- 172.21.15.78:0/2692338779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04035040 con 0x5630e6cc72f0 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2692338779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cc72f0 msgr2=0x5630e70add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2692338779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cc72f0 0x5630e70add40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009900 tx=0x7fbb04000bc0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2692338779 shutdown_connections 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2692338779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ccf4d0 0x5630e70a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2692338779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6cc72f0 0x5630e70add40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2692338779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6ce4300 0x5630e70ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2692338779 >> 172.21.15.78:0/2692338779 conn(0x5630e7087520 msgr2=0x5630e7094af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2692338779 shutdown_connections 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2692338779 wait complete. 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.134+0000 7fbb20fa9740 1 -- start start 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 1 Processor -- start 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e710a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6ccf4d0 0x5630e710c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ce4300 0x5630e710ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141a99d0 con 0x5630e6ccf4d0 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141aae40 con 0x5630e6ce4300 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141aa0a0 con 0x5630e6cc72f0 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb20fa9740 10 client.0 _ll_register_callbacks cb 0x7fbb04005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6ccf4d0 0x5630e710c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6ccf4d0 0x5630e710c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45124/0 (socket says 172.21.15.78:45124) 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 -- 172.21.15.78:0/2738557278 learned_addr learned my addr 172.21.15.78:0/2738557278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e710a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ce4300 0x5630e710ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 -- 172.21.15.78:0/2738557278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 msgr2=0x5630e710a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e710a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 -- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ce4300 msgr2=0x5630e710ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ce4300 0x5630e710ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 -- 172.21.15.78:0/2738557278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e710f360 con 0x5630e6ccf4d0 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbb137fe640 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6ccf4d0 0x5630e710c8d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbb04005ab0 tx=0x7fbb040042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d3e2477e0bacc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.135+0000 7fbaf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/2738557278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0404b030 con 0x5630e6ccf4d0 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/2738557278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbb04047420 con 0x5630e6ccf4d0 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb04004380 con 0x5630e6ccf4d0 2024-04-25T14:03:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/2738557278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbb04035040 con 0x5630e6ccf4d0 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 10 client.16422 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5630e71105c0 con 0x5630e6ccf4d0 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/2738557278 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb04004380 con 0x5630e6ccf4d0 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/2738557278 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbb0403cac0 con 0x5630e6ccf4d0 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 10 client.16422 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 20 client.16422 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6cf01b0 0x5630e7112f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 4 client.16422 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630e6ffdee0 con 0x5630e6cf01b0 2024-04-25T14:03:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.136+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6cf01b0 0x5630e7112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.137+0000 7fbaf2ffd640 10 client.16422 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.137+0000 7fbb12ffd640 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6cf01b0 0x5630e7112f40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffabbc8cc9248026 server_cookie=33a91968f6e2cba4 in_seq=0 out_seq=0 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.151+0000 7fbaf2ffd640 1 -- 172.21.15.78:0/2738557278 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630e6ffdee0 con 0x5630e6cf01b0 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.151+0000 7fbaf2ffd640 10 client.16422 handle_command_reply: tid=0 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.151+0000 7fbb20fa9740 1 client.16422 shutdown 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6ccf4d0 msgr2=0x5630e710c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6ccf4d0 0x5630e710c8d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbb04005ab0 tx=0x7fbb040042e0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 shutdown_connections 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5630e6ce4300 0x5630e710ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e6ccf4d0 0x5630e710c8d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5630e6cf01b0 0x5630e7112f40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 --2- 172.21.15.78:0/2738557278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630e6cc72f0 0x5630e710a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbaf2ffd640 0 client.16422 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 >> 172.21.15.78:0/2738557278 conn(0x5630e7087520 msgr2=0x5630e708f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 shutdown_connections 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 1 -- 172.21.15.78:0/2738557278 wait complete. 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb20fa9740 20 client.16422 trim_cache size 0 max 16384 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:03:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:03:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:03:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbafc0afae0 msgr2=0x7fbafc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbafc0afae0 0x7fbafc0b1ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c01a000 tx=0x7fbb0c074040 comp rx=0 tx=0).stop 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14109a90 msgr2=0x7fbb141a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14109a90 0x7fbb141a1710 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00cad0 tx=0x7fbb0c00cf90 comp rx=0 tx=0).stop 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 shutdown_connections 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbafc0afae0 0x7fbafc0b1ff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb14109a90 0x7fbb141a1710 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb14104d90 0x7fbb1419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.152+0000 7fbb1a0f9640 1 --2- 172.21.15.78:0/3204034800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb14103b50 0x7fbb1419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.153+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 >> 172.21.15.78:0/3204034800 conn(0x7fbb140fed20 msgr2=0x7fbb14105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.153+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 shutdown_connections 2024-04-25T14:03:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:01.153+0000 7fbb1a0f9640 1 -- 172.21.15.78:0/3204034800 wait complete. 2024-04-25T14:03:02.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:02.027 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 -- 172.21.15.78:0/179149586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c104470 msgr2=0x7fd26c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 --2- 172.21.15.78:0/179149586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c104470 0x7fd26c10abb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd264009ca0 tx=0x7fd26402a8e0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 -- 172.21.15.78:0/179149586 shutdown_connections 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 --2- 172.21.15.78:0/179149586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c104470 0x7fd26c10abb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 --2- 172.21.15.78:0/179149586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd26c103970 0x7fd26c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 --2- 172.21.15.78:0/179149586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd26c102f90 0x7fd26c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 -- 172.21.15.78:0/179149586 >> 172.21.15.78:0/179149586 conn(0x7fd26c0fd350 msgr2=0x7fd26c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 -- 172.21.15.78:0/179149586 shutdown_connections 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 -- 172.21.15.78:0/179149586 wait complete. 2024-04-25T14:03:02.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 Processor -- start 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.150+0000 7fd272700640 1 -- start start 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd272700640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd26c102f90 0x7fd26c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd272700640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c103970 0x7fd26c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd272700640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd26c104470 0x7fd26c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd272700640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c10e590 con 0x7fd26c103970 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd272700640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c10e410 con 0x7fd26c104470 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd272700640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c10e710 con 0x7fd26c102f90 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c103970 0x7fd26c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c103970 0x7fd26c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45154/0 (socket says 172.21.15.78:45154) 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 -- 172.21.15.78:0/2350600375 learned_addr learned my addr 172.21.15.78:0/2350600375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26bfff640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd26c102f90 0x7fd26c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd270c9d640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd26c104470 0x7fd26c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 -- 172.21.15.78:0/2350600375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd26c102f90 msgr2=0x7fd26c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd26c102f90 0x7fd26c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 -- 172.21.15.78:0/2350600375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd26c104470 msgr2=0x7fd26c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd26c104470 0x7fd26c1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 -- 172.21.15.78:0/2350600375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c1a1cc0 con 0x7fd26c103970 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26bfff640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd26c102f90 0x7fd26c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c103970 0x7fd26c19a770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd26000c920 tx=0x7fd26000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd2697fa640 1 -- 172.21.15.78:0/2350600375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260007c70 con 0x7fd26c103970 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd2697fa640 1 -- 172.21.15.78:0/2350600375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd260007e10 con 0x7fd26c103970 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.151+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c1a1fb0 con 0x7fd26c103970 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.152+0000 7fd2697fa640 1 -- 172.21.15.78:0/2350600375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260018820 con 0x7fd26c103970 2024-04-25T14:03:02.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.152+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c1a24a0 con 0x7fd26c103970 2024-04-25T14:03:02.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 Processor -- start 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 -- start start 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2697fa640 1 -- 172.21.15.78:0/2350600375 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd2600189c0 con 0x7fd26c103970 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a0792390 0x55b7a0b33a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07694d0 0x55b7a0b33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b7a0a98d80 con 0x55b7a07694d0 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b7a0a528b0 con 0x55b7a0792390 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd2795b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b7a0a94d40 con 0x55b7a07716b0 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd26bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07694d0 0x55b7a0b33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd26bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07694d0 0x55b7a0b33fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45170/0 (socket says 172.21.15.78:45170) 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.154+0000 7fd26bfff640 1 -- 172.21.15.78:0/82129452 learned_addr learned my addr 172.21.15.78:0/82129452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd270c9d640 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a0792390 0x55b7a0b33a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26bfff640 1 -- 172.21.15.78:0/82129452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 msgr2=0x55b7a0b34c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26bfff640 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b34c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26bfff640 1 -- 172.21.15.78:0/82129452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a0792390 msgr2=0x55b7a0b33a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26bfff640 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a0792390 0x55b7a0b33a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26bfff640 1 -- 172.21.15.78:0/82129452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a0a23a40 con 0x55b7a07694d0 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd270c9d640 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a0792390 0x55b7a0b33a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd2697fa640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd2540afaf0 0x7fd2540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd26bfff640 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07694d0 0x55b7a0b33fe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00a9a0 tx=0x7fd25c00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f25a44d43f68f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd270c9d640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd2540afaf0 0x7fd2540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd2697fa640 1 -- 172.21.15.78:0/2350600375 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd2600dee60 con 0x7fd26c103970 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd2527fc640 1 -- 172.21.15.78:0/82129452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c015070 con 0x55b7a07694d0 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd2527fc640 1 -- 172.21.15.78:0/82129452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd25c00cc00 con 0x55b7a07694d0 2024-04-25T14:03:02.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.155+0000 7fd2527fc640 1 -- 172.21.15.78:0/82129452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0185c0 con 0x55b7a07694d0 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 -- 172.21.15.78:0/82129452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07694d0 msgr2=0x55b7a0b33fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd270c9d640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd2540afaf0 0x7fd2540b2000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd264031930 tx=0x7fd264009810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07694d0 0x55b7a0b33fe0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00a9a0 tx=0x7fd25c00ae60 comp rx=0 tx=0).stop 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 -- 172.21.15.78:0/82129452 shutdown_connections 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07694d0 0x55b7a0b33fe0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 --2- 172.21.15.78:0/82129452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a0792390 0x55b7a0b33a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 -- 172.21.15.78:0/82129452 >> 172.21.15.78:0/82129452 conn(0x55b7a07864e0 msgr2=0x55b7a0b2f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 -- 172.21.15.78:0/82129452 shutdown_connections 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.156+0000 7fd2795b1740 1 -- 172.21.15.78:0/82129452 wait complete. 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 -- start start 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 Processor -- start 2024-04-25T14:03:02.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b3d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b3de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b44d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b7a0b35780 con 0x55b7a07716b0 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b7a0a1fd10 con 0x55b7a07694d0 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b7a0b35900 con 0x55b7a0792390 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd2795b1740 10 client.0 _ll_register_callbacks cb 0x55b7a0a4c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b3de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b3de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45176/0 (socket says 172.21.15.78:45176) 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3283397009 learned_addr learned my addr 172.21.15.78:0/3283397009 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b3d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd270c9d640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b44d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3283397009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 msgr2=0x55b7a0b44d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b44d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 msgr2=0x55b7a0b3d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.157+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b3d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3283397009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a08747d0 con 0x55b7a07716b0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd270c9d640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b44d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b3de10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd2600a7030 tx=0x7fd26000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bf1158991c0889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd24b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3283397009 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600a7330 con 0x55b7a07716b0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3283397009 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd2600aff10 con 0x55b7a07716b0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd25c00cc00 con 0x55b7a07716b0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3283397009 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd2600ab6e0 con 0x55b7a07716b0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 10 client.16434 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b7a07aa3f0 con 0x55b7a07716b0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3283397009 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd25c00cc00 con 0x55b7a07716b0 2024-04-25T14:03:02.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3283397009 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd2600a2dc0 con 0x55b7a07716b0 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 10 client.16434 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 20 client.16434 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a0750f60 0x55b7a0b4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 4 client.16434 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.158+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7a0aa6990 con 0x55b7a0750f60 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.159+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a0750f60 0x55b7a0b4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.159+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a0750f60 0x55b7a0b4b260 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70eff26b6e6ad1aa server_cookie=7938b76b9042cc29 in_seq=0 out_seq=0 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.159+0000 7fd24b7fe640 10 client.16434 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3283397009 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7a0aa6990 con 0x55b7a0750f60 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd24b7fe640 10 client.16434 handle_command_reply: tid=0 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 client.16434 shutdown 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 msgr2=0x55b7a0b3de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b3de10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd2600a7030 tx=0x7fd26000fcc0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 shutdown_connections 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a0750f60 0x55b7a0b4b260 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b44d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b3de10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3283397009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b3d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd24b7fe640 0 client.16434 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.160+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 >> 172.21.15.78:0/3283397009 conn(0x55b7a07864e0 msgr2=0x55b7a0b32180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.161+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 shutdown_connections 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.161+0000 7fd2795b1740 1 -- 172.21.15.78:0/3283397009 wait complete. 2024-04-25T14:03:02.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.161+0000 7fd2795b1740 20 client.16434 trim_cache size 0 max 16384 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 Processor -- start 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 -- start start 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b2eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b2f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a0792390 0x55b7a0b3bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1adb50 con 0x55b7a0792390 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1ab420 con 0x55b7a07694d0 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.178+0000 7fd2795b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1a9270 con 0x55b7a07716b0 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a0792390 0x55b7a0b3bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a0792390 0x55b7a0b3bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45190/0 (socket says 172.21.15.78:45190) 2024-04-25T14:03:02.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 -- 172.21.15.78:0/3910139863 learned_addr learned my addr 172.21.15.78:0/3910139863 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b2eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b2f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 -- 172.21.15.78:0/3910139863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 msgr2=0x55b7a0b2f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b2f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 -- 172.21.15.78:0/3910139863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 msgr2=0x55b7a0b2eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b2eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 -- 172.21.15.78:0/3910139863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a076a0f0 con 0x55b7a0792390 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd270c9d640 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a0792390 0x55b7a0b3bf00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd264035000 tx=0x7fd26403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46791548550f5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd251ffb640 1 -- 172.21.15.78:0/3910139863 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264004240 con 0x55b7a0792390 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd251ffb640 1 -- 172.21.15.78:0/3910139863 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd26404e070 con 0x55b7a0792390 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.179+0000 7fd251ffb640 1 -- 172.21.15.78:0/3910139863 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264030dc0 con 0x55b7a0792390 2024-04-25T14:03:02.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 -- 172.21.15.78:0/3910139863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a0792390 msgr2=0x55b7a0b3bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a0792390 0x55b7a0b3bf00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd264035000 tx=0x7fd26403e000 comp rx=0 tx=0).stop 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 -- 172.21.15.78:0/3910139863 shutdown_connections 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a0792390 0x55b7a0b3bf00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a07716b0 0x55b7a0b2f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3910139863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b2eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 -- 172.21.15.78:0/3910139863 >> 172.21.15.78:0/3910139863 conn(0x55b7a07864e0 msgr2=0x55b7a0b339e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 -- 172.21.15.78:0/3910139863 shutdown_connections 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.180+0000 7fd2795b1740 1 -- 172.21.15.78:0/3910139863 wait complete. 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 -- start start 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 Processor -- start 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b41850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b41d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b42cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1a98e0 con 0x55b7a07716b0 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1aad50 con 0x55b7a07694d0 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c1a9fb0 con 0x55b7a0792390 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd2795b1740 10 client.0 _ll_register_callbacks cb 0x7fd26000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b41d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd26b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b41d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45204/0 (socket says 172.21.15.78:45204) 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3962683335 learned_addr learned my addr 172.21.15.78:0/3962683335 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b41850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.181+0000 7fd270c9d640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b42cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3962683335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 msgr2=0x55b7a0b42cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b42cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 msgr2=0x55b7a0b41850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b41850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd26b7fe640 1 -- 172.21.15.78:0/3962683335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a0a594b0 con 0x55b7a07716b0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b41850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd26b7fe640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b41d90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd260007590 tx=0x7fd2600a7330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8469320c1b06eed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd24b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3962683335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600a67a0 con 0x55b7a07716b0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3962683335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd2600a2ca0 con 0x55b7a07716b0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd26404e070 con 0x55b7a07716b0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3962683335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd2600a2ec0 con 0x55b7a07716b0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 10 client.16440 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b7a0b44340 con 0x55b7a07716b0 2024-04-25T14:03:02.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3962683335 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd26404e070 con 0x55b7a07716b0 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3962683335 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd2600a6100 con 0x55b7a07716b0 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 10 client.16440 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 20 client.16440 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a07263a0 0x55b7a0bb5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 4 client.16440 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.182+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7a0aa0800 con 0x55b7a07263a0 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.183+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a07263a0 0x55b7a0bb5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.183+0000 7fd24b7fe640 10 client.16440 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.183+0000 7fd26bfff640 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a07263a0 0x55b7a0bb5a20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406e9d333a6bd26e server_cookie=ae8ffeca2503c4a3 in_seq=0 out_seq=0 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.183+0000 7fd24b7fe640 1 -- 172.21.15.78:0/3962683335 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7a0aa0800 con 0x55b7a07263a0 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.183+0000 7fd24b7fe640 10 client.16440 handle_command_reply: tid=0 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.183+0000 7fd2795b1740 1 client.16440 shutdown 2024-04-25T14:03:02.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 msgr2=0x55b7a0b41d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b41d90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd260007590 tx=0x7fd2600a7330 comp rx=0 tx=0).stop 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 shutdown_connections 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b7a07263a0 0x55b7a0bb5a20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7a0792390 0x55b7a0b42cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b7a07716b0 0x55b7a0b41d90 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd24b7fe640 0 client.16440 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 --2- 172.21.15.78:0/3962683335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7a07694d0 0x55b7a0b41850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 >> 172.21.15.78:0/3962683335 conn(0x55b7a07864e0 msgr2=0x55b7a0b2ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 shutdown_connections 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 1 -- 172.21.15.78:0/3962683335 wait complete. 2024-04-25T14:03:02.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd2795b1740 20 client.16440 trim_cache size 0 max 16384 2024-04-25T14:03:02.192 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:03:02.192 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:03:02.192 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:03:02.192 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:03:02.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd2540afaf0 msgr2=0x7fd2540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd2540afaf0 0x7fd2540b2000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd264031930 tx=0x7fd264009810 comp rx=0 tx=0).stop 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c103970 msgr2=0x7fd26c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c103970 0x7fd26c19a770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd26000c920 tx=0x7fd26000cde0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 shutdown_connections 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd26c104470 0x7fd26c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd2540afaf0 0x7fd2540b2000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd26c103970 0x7fd26c19a770 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 --2- 172.21.15.78:0/2350600375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd26c102f90 0x7fd26c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.184+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 >> 172.21.15.78:0/2350600375 conn(0x7fd26c0fd350 msgr2=0x7fd26c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.185+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 shutdown_connections 2024-04-25T14:03:02.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.185+0000 7fd272700640 1 -- 172.21.15.78:0/2350600375 wait complete. 2024-04-25T14:03:02.195 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:03:02.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:02.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 -- 172.21.15.78:0/2376132787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 msgr2=0x7f9edc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/2376132787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc10ad40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802a410 comp rx=0 tx=0).stop 2024-04-25T14:03:02.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 -- 172.21.15.78:0/2376132787 shutdown_connections 2024-04-25T14:03:02.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/2376132787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc10ad40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/2376132787 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9edc103c20 0x7f9edc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/2376132787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9edc1029e0 0x7f9edc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 -- 172.21.15.78:0/2376132787 >> 172.21.15.78:0/2376132787 conn(0x7f9edc0fdbd0 msgr2=0x7f9edc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 -- 172.21.15.78:0/2376132787 shutdown_connections 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.305+0000 7f9ee47ee640 1 -- 172.21.15.78:0/2376132787 wait complete. 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 Processor -- start 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 -- start start 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9edc1029e0 0x7f9edc199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9edc103c20 0x7f9edc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10e4f0 con 0x7f9edc108920 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10e370 con 0x7f9edc1029e0 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee47ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10e670 con 0x7f9edc103c20 2024-04-25T14:03:02.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc1a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45214/0 (socket says 172.21.15.78:45214) 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 -- 172.21.15.78:0/1643334874 learned_addr learned my addr 172.21.15.78:0/1643334874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee1d89640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9edc103c20 0x7f9edc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee258a640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9edc1029e0 0x7f9edc199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 -- 172.21.15.78:0/1643334874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9edc103c20 msgr2=0x7f9edc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9edc103c20 0x7f9edc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 -- 172.21.15.78:0/1643334874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9edc1029e0 msgr2=0x7f9edc199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9edc1029e0 0x7f9edc199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee2d8b640 1 -- 172.21.15.78:0/1643334874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc1a0ba0 con 0x7f9edc108920 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.306+0000 7f9ee258a640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9edc1029e0 0x7f9edc199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.307+0000 7f9ee2d8b640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc1a0530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009230 tx=0x7f9ed80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.307+0000 7f9ecf7fe640 1 -- 172.21.15.78:0/1643334874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed803f030 con 0x7f9edc108920 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.307+0000 7f9ecf7fe640 1 -- 172.21.15.78:0/1643334874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed8031910 con 0x7f9edc108920 2024-04-25T14:03:02.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.307+0000 7f9ecf7fe640 1 -- 172.21.15.78:0/1643334874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8035070 con 0x7f9edc108920 2024-04-25T14:03:02.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.307+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc1a0e30 con 0x7f9edc108920 2024-04-25T14:03:02.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.307+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc1a1240 con 0x7f9edc108920 2024-04-25T14:03:02.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.308+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc107730 con 0x7f9edc108920 2024-04-25T14:03:02.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.310+0000 7f9ecf7fe640 1 -- 172.21.15.78:0/1643334874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9ed8042030 con 0x7f9edc108920 2024-04-25T14:03:02.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.311+0000 7f9ecf7fe640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ec80afaf0 0x7f9ec80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.311+0000 7f9ee258a640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ec80afaf0 0x7f9ec80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.311+0000 7f9ecf7fe640 1 -- 172.21.15.78:0/1643334874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9ed80ff560 con 0x7f9edc108920 2024-04-25T14:03:02.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.311+0000 7f9ee258a640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ec80afaf0 0x7f9ec80b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9edc1035e0 tx=0x7f9ed4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.315+0000 7f9ecf7fe640 1 -- 172.21.15.78:0/1643334874 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ed80c8130 con 0x7f9edc108920 2024-04-25T14:03:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:02 smithi078 ceph-mon[29163]: pgmap v454: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:02.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.530+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9edc02db50 con 0x7f9edc108920 2024-04-25T14:03:02.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.530+0000 7f9ecf7fe640 1 -- 172.21.15.78:0/1643334874 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f9ed80d01b0 con 0x7f9edc108920 2024-04-25T14:03:02.530 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:03:02.531 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:03:02.531 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:03:02.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ec80afaf0 msgr2=0x7f9ec80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ec80afaf0 0x7f9ec80b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9edc1035e0 tx=0x7f9ed4005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 msgr2=0x7f9edc1a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc1a0530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009230 tx=0x7f9ed80042f0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 shutdown_connections 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9ec80afaf0 0x7f9ec80b2000 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9edc108920 0x7f9edc1a0530 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9edc103c20 0x7f9edc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 --2- 172.21.15.78:0/1643334874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9edc1029e0 0x7f9edc199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 >> 172.21.15.78:0/1643334874 conn(0x7f9edc0fdbd0 msgr2=0x7f9edc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 shutdown_connections 2024-04-25T14:03:02.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.533+0000 7f9ee47ee640 1 -- 172.21.15.78:0/1643334874 wait complete. 2024-04-25T14:03:02.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:03:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 -- 172.21.15.78:0/605266194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 msgr2=0x7f4d2c103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/605266194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c103380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009900 tx=0x7f4d1c02a410 comp rx=0 tx=0).stop 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 -- 172.21.15.78:0/605266194 shutdown_connections 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/605266194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d2c104470 0x7f4d2c10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/605266194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c103950 0x7f4d2c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/605266194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c103380 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 -- 172.21.15.78:0/605266194 >> 172.21.15.78:0/605266194 conn(0x7f4d2c0fd350 msgr2=0x7f4d2c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 -- 172.21.15.78:0/605266194 shutdown_connections 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.654+0000 7f4d31b1f640 1 -- 172.21.15.78:0/605266194 wait complete. 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 Processor -- start 2024-04-25T14:03:02.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 -- start start 2024-04-25T14:03:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c103950 0x7f4d2c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d2c104470 0x7f4d2c1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10e510 con 0x7f4d2c102f70 2024-04-25T14:03:02.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10e390 con 0x7f4d2c104470 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d31b1f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10e690 con 0x7f4d2c103950 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45244/0 (socket says 172.21.15.78:45244) 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/3668465363 learned_addr learned my addr 172.21.15.78:0/3668465363 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2affd640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c103950 0x7f4d2c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d2c104470 0x7f4d2c1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/3668465363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c103950 msgr2=0x7f4d2c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c103950 0x7f4d2c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.655+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/3668465363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d2c104470 msgr2=0x7f4d2c1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d2c104470 0x7f4d2c1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/3668465363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c1a1d00 con 0x7f4d2c102f70 2024-04-25T14:03:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c102210 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c006010 tx=0x7f4d1c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d28ff9640 1 -- 172.21.15.78:0/3668465363 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c03f030 con 0x7f4d2c102f70 2024-04-25T14:03:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d28ff9640 1 -- 172.21.15.78:0/3668465363 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d1c003aa0 con 0x7f4d2c102f70 2024-04-25T14:03:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c1a1f90 con 0x7f4d2c102f70 2024-04-25T14:03:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d28ff9640 1 -- 172.21.15.78:0/3668465363 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c035070 con 0x7f4d2c102f70 2024-04-25T14:03:02.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.656+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c1a23a0 con 0x7f4d2c102f70 2024-04-25T14:03:02.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:02 smithi129 ceph-mon[32569]: pgmap v454: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.658+0000 7f4d28ff9640 1 -- 172.21.15.78:0/3668465363 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4d1c042030 con 0x7f4d2c102f70 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.658+0000 7f4d389d2740 1 Processor -- start 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d389d2740 1 -- start start 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c4751bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474df76b0 0x55c4751bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d389d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c47511ed80 con 0x55c474def4d0 2024-04-25T14:03:02.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d389d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c4750d8a20 con 0x55c474e18390 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d389d2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c47511ad40 con 0x55c474df76b0 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c4751bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c4751bad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45260/0 (socket says 172.21.15.78:45260) 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/2015070161 learned_addr learned my addr 172.21.15.78:0/2015070161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2affd640 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474df76b0 0x55c4751bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d28ff9640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d140afaf0 0x7f4d140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/2015070161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474df76b0 msgr2=0x55c4751bba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474df76b0 0x55c4751bba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/2015070161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 msgr2=0x55c4751ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751ba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2b7fe640 1 -- 172.21.15.78:0/2015070161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4750e3c30 con 0x55c474def4d0 2024-04-25T14:03:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.659+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d140afaf0 0x7f4d140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.660+0000 7f4d28ff9640 1 -- 172.21.15.78:0/3668465363 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4d1c0ff300 con 0x7f4d2c102f70 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.660+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c4751bad90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0e5ec0 tx=0x7f4d1c0f5ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bacaae9db9b022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.660+0000 7f4d11ffb640 1 -- 172.21.15.78:0/2015070161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c049070 con 0x55c474def4d0 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.660+0000 7f4d11ffb640 1 -- 172.21.15.78:0/2015070161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d1c0c7280 con 0x55c474def4d0 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.660+0000 7f4d11ffb640 1 -- 172.21.15.78:0/2015070161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0cbd80 con 0x55c474def4d0 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 -- 172.21.15.78:0/2015070161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 msgr2=0x55c4751bad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d140afaf0 0x7f4d140b2000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55c4751bb860 tx=0x7f4d24004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c4751bad90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0e5ec0 tx=0x7f4d1c0f5ec0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 -- 172.21.15.78:0/2015070161 shutdown_connections 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474df76b0 0x55c4751bba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c4751bad90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2015070161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 -- 172.21.15.78:0/2015070161 >> 172.21.15.78:0/2015070161 conn(0x55c474e0c4e0 msgr2=0x55c4751b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 -- 172.21.15.78:0/2015070161 shutdown_connections 2024-04-25T14:03:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 -- 172.21.15.78:0/2015070161 wait complete. 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 -- start start 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 Processor -- start 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.661+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c47511ed80 con 0x55c474df76b0 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c4750a5dc0 con 0x55c474e18390 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c4751bc2f0 con 0x55c474def4d0 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 10 client.0 _ll_register_callbacks cb 0x55c4750c1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45262/0 (socket says 172.21.15.78:45262) 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 -- 172.21.15.78:0/2107081999 learned_addr learned my addr 172.21.15.78:0/2107081999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 -- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 msgr2=0x55c4751c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 -- 172.21.15.78:0/2107081999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 msgr2=0x55c4751cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 -- 172.21.15.78:0/2107081999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c474e303f0 con 0x55c474df76b0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d2affd640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751c4680 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000bec0 tx=0x7f4d20012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece0d49885cb42b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d0affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d0affd640 1 -- 172.21.15.78:0/2107081999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20005220 con 0x55c474df76b0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d0affd640 1 -- 172.21.15.78:0/2107081999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4d200053c0 con 0x55c474df76b0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d1c0c7280 con 0x55c474df76b0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d0affd640 1 -- 172.21.15.78:0/2107081999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4d200055c0 con 0x55c474df76b0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 10 client.16458 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.662+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c474efa7d0 con 0x55c474df76b0 2024-04-25T14:03:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d0affd640 1 -- 172.21.15.78:0/2107081999 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d1c0c7280 con 0x55c474df76b0 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d0affd640 1 -- 172.21.15.78:0/2107081999 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4d20023050 con 0x55c474df76b0 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d389d2740 10 client.16458 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d389d2740 10 client.16458 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d389d2740 20 client.16458 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474dd6f60 0x55c4751d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d389d2740 4 client.16458 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c47512c990 con 0x55c474dd6f60 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.663+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474dd6f60 0x55c4751d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.664+0000 7f4d0affd640 10 client.16458 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.664+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474dd6f60 0x55c4751d1e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af2048dfa4a22fee server_cookie=14227ff879757da3 in_seq=0 out_seq=0 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.664+0000 7f4d0affd640 1 -- 172.21.15.78:0/2107081999 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c47512c990 con 0x55c474dd6f60 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.664+0000 7f4d0affd640 10 client.16458 handle_command_reply: tid=0 2024-04-25T14:03:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.664+0000 7f4d389d2740 1 client.16458 shutdown 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 msgr2=0x55c4751c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751c4680 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000bec0 tx=0x7f4d20012820 comp rx=0 tx=0).stop 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 shutdown_connections 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474dd6f60 0x55c4751d1e80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474e18390 0x55c4751cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d0affd640 0 client.16458 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751c4680 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2107081999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 >> 172.21.15.78:0/2107081999 conn(0x55c474e0c4e0 msgr2=0x55c4751bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 shutdown_connections 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 1 -- 172.21.15.78:0/2107081999 wait complete. 2024-04-25T14:03:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.665+0000 7f4d389d2740 20 client.16458 trim_cache size 0 max 16384 2024-04-25T14:03:02.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 Processor -- start 2024-04-25T14:03:02.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 -- start start 2024-04-25T14:03:02.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474dd6f60 0x55c4751c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1adb30 con 0x55c474df76b0 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1ab400 con 0x55c474dd6f60 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.678+0000 7f4d389d2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1a9250 con 0x55c474def4d0 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751bcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45274/0 (socket says 172.21.15.78:45274) 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 -- 172.21.15.78:0/2554853696 learned_addr learned my addr 172.21.15.78:0/2554853696 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474dd6f60 0x55c4751c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2affd640 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 -- 172.21.15.78:0/2554853696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 msgr2=0x55c4751c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 -- 172.21.15.78:0/2554853696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474dd6f60 msgr2=0x55c4751c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474dd6f60 0x55c4751c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 -- 172.21.15.78:0/2554853696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4751c23e0 con 0x55c474df76b0 2024-04-25T14:03:02.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751bcb80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009fd0 tx=0x7f4d24015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2122f4d27762dffd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d117fa640 1 -- 172.21.15.78:0/2554853696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2401f9d0 con 0x55c474df76b0 2024-04-25T14:03:02.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d117fa640 1 -- 172.21.15.78:0/2554853696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d24017070 con 0x55c474df76b0 2024-04-25T14:03:02.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.679+0000 7f4d117fa640 1 -- 172.21.15.78:0/2554853696 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d240273e0 con 0x55c474df76b0 2024-04-25T14:03:02.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 -- 172.21.15.78:0/2554853696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 msgr2=0x55c4751bcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751bcb80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009fd0 tx=0x7f4d24015000 comp rx=0 tx=0).stop 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 -- 172.21.15.78:0/2554853696 shutdown_connections 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474df76b0 0x55c4751bcb80 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474def4d0 0x55c4751c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 --2- 172.21.15.78:0/2554853696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474dd6f60 0x55c4751c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 -- 172.21.15.78:0/2554853696 >> 172.21.15.78:0/2554853696 conn(0x55c474e0c4e0 msgr2=0x55c4751bf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 -- 172.21.15.78:0/2554853696 shutdown_connections 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 -- 172.21.15.78:0/2554853696 wait complete. 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.680+0000 7f4d389d2740 1 -- start start 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 1 Processor -- start 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474dd6f60 0x55c4752351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c475235730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474df76b0 0x55c47523c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1a98c0 con 0x55c474def4d0 2024-04-25T14:03:02.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1aad30 con 0x55c474df76b0 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c1a9f90 con 0x55c474dd6f60 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d389d2740 10 client.0 _ll_register_callbacks cb 0x7f4d2000b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c475235730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c475235730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45282/0 (socket says 172.21.15.78:45282) 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 -- 172.21.15.78:0/3947105491 learned_addr learned my addr 172.21.15.78:0/3947105491 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2bfff640 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474df76b0 0x55c47523c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474dd6f60 0x55c4752351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 -- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474dd6f60 msgr2=0x55c4752351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474dd6f60 0x55c4752351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 -- 172.21.15.78:0/3947105491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474df76b0 msgr2=0x55c47523c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474df76b0 0x55c47523c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 -- 172.21.15.78:0/3947105491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47523cbe0 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d2affd640 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c475235730 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d20000c00 tx=0x7f4d20025690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc46855682a1a369 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d0affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.681+0000 7f4d0affd640 1 -- 172.21.15.78:0/3947105491 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20025c90 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d0affd640 1 -- 172.21.15.78:0/3947105491 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4d2002d420 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d24017070 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d0affd640 1 -- 172.21.15.78:0/3947105491 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4d20012d20 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 10 client.16464 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c47523def0 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d0affd640 1 -- 172.21.15.78:0/3947105491 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d24017070 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d0affd640 1 -- 172.21.15.78:0/3947105491 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4d20023040 con 0x55c474def4d0 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 10 client.16464 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 10 client.16464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 20 client.16464 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:02.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474e18390 0x55c475240910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 4 client.16464 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c475126800 con 0x55c474e18390 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.682+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474e18390 0x55c475240910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d0affd640 10 client.16464 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d2b7fe640 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474e18390 0x55c475240910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38a8a14cab49c4ad server_cookie=696f8a8c0e2564dd in_seq=0 out_seq=0 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d0affd640 1 -- 172.21.15.78:0/3947105491 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c475126800 con 0x55c474e18390 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d0affd640 10 client.16464 handle_command_reply: tid=0 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 client.16464 shutdown 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 msgr2=0x55c475235730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c475235730 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4d20000c00 tx=0x7f4d20025690 comp rx=0 tx=0).stop 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 shutdown_connections 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55c474e18390 0x55c475240910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c474df76b0 0x55c47523c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c474def4d0 0x55c475235730 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d0affd640 0 client.16464 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 --2- 172.21.15.78:0/3947105491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c474dd6f60 0x55c4752351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 >> 172.21.15.78:0/3947105491 conn(0x55c474e0c4e0 msgr2=0x55c4751c2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 shutdown_connections 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 1 -- 172.21.15.78:0/3947105491 wait complete. 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.683+0000 7f4d389d2740 20 client.16464 trim_cache size 0 max 16384 2024-04-25T14:03:02.691 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d140afaf0 msgr2=0x7f4d140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d140afaf0 0x7f4d140b2000 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55c4751bb860 tx=0x7f4d24004a40 comp rx=0 tx=0).stop 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 msgr2=0x7f4d2c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c102210 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c006010 tx=0x7f4d1c031a50 comp rx=0 tx=0).stop 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 shutdown_connections 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d2c104470 0x7f4d2c1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d2c103950 0x7f4d2c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d140afaf0 0x7f4d140b2000 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 --2- 172.21.15.78:0/3668465363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d2c102f70 0x7f4d2c102210 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 >> 172.21.15.78:0/3668465363 conn(0x7f4d2c0fd350 msgr2=0x7f4d2c104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 shutdown_connections 2024-04-25T14:03:02.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.684+0000 7f4d31b1f640 1 -- 172.21.15.78:0/3668465363 wait complete. 2024-04-25T14:03:02.695 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:03:02.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:02 smithi112 ceph-mon[32617]: pgmap v454: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:02.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 -- 172.21.15.78:0/978735921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 msgr2=0x7fb4f81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 --2- 172.21.15.78:0/978735921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f81047f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 -- 172.21.15.78:0/978735921 shutdown_connections 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 --2- 172.21.15.78:0/978735921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4f8109a30 0x7fb4f810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 --2- 172.21.15.78:0/978735921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4f8104d30 0x7fb4f8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 --2- 172.21.15.78:0/978735921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f81047f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 -- 172.21.15.78:0/978735921 >> 172.21.15.78:0/978735921 conn(0x7fb4f80fece0 msgr2=0x7fb4f8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 -- 172.21.15.78:0/978735921 shutdown_connections 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.805+0000 7fb50077d640 1 -- 172.21.15.78:0/978735921 wait complete. 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 Processor -- start 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 -- start start 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4f8104d30 0x7fb4f819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4f8109a30 0x7fb4f81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810f5d0 con 0x7fb4f8103af0 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810f450 con 0x7fb4f8109a30 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb50077d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f810f750 con 0x7fb4f8104d30 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb4fe519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb4fe519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45300/0 (socket says 172.21.15.78:45300) 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb4fe519640 1 -- 172.21.15.78:0/2952799401 learned_addr learned my addr 172.21.15.78:0/2952799401 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4f8109a30 0x7fb4f81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.806+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4f8104d30 0x7fb4f819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4fe519640 1 -- 172.21.15.78:0/2952799401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4f8104d30 msgr2=0x7fb4f819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4fe519640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4f8104d30 0x7fb4f819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4fe519640 1 -- 172.21.15.78:0/2952799401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4f8109a30 msgr2=0x7fb4f81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4fe519640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4f8109a30 0x7fb4f81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4fe519640 1 -- 172.21.15.78:0/2952799401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f81a1db0 con 0x7fb4f8103af0 2024-04-25T14:03:02.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4fe519640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f819a270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4032fd0 tx=0x7fb4f40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/2952799401 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f403f070 con 0x7fb4f8103af0 2024-04-25T14:03:02.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/2952799401 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f40044d0 con 0x7fb4f8103af0 2024-04-25T14:03:02.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f81a2040 con 0x7fb4f8103af0 2024-04-25T14:03:02.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/2952799401 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4035070 con 0x7fb4f8103af0 2024-04-25T14:03:02.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.807+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f81a2530 con 0x7fb4f8103af0 2024-04-25T14:03:02.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.809+0000 7fb50762c740 1 Processor -- start 2024-04-25T14:03:02.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.809+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/2952799401 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb4f403b5f0 con 0x7fb4f8103af0 2024-04-25T14:03:02.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.809+0000 7fb50762c740 1 -- start start 2024-04-25T14:03:02.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb50762c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb50762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d04ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb50762c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb50762c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c1cfb2d80 con 0x562c1cc8b6b0 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb50762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c1cf6ca20 con 0x562c1cc834d0 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb50762c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c1cfaed40 con 0x562c1ccac390 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45314/0 (socket says 172.21.15.78:45314) 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 -- 172.21.15.78:0/2335951163 learned_addr learned my addr 172.21.15.78:0/2335951163 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fe519640 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d04ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4ef7fe640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4e40afaf0 0x7fb4e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 -- 172.21.15.78:0/2335951163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 msgr2=0x562c1d04e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 -- 172.21.15.78:0/2335951163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 msgr2=0x562c1d04ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d04ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 -- 172.21.15.78:0/2335951163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1cf77c30 con 0x562c1cc8b6b0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4e40afaf0 0x7fb4e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/2952799401 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb4f40ff1f0 con 0x7fb4f8103af0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.810+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04fa20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800cbb0 tx=0x7fb4e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d0bc7e5cce968b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.811+0000 7fb4ecff9640 1 -- 172.21.15.78:0/2335951163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8005220 con 0x562c1cc8b6b0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.811+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4e40afaf0 0x7fb4e40b2000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x562c1d04f860 tx=0x7fb4f0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.811+0000 7fb4ecff9640 1 -- 172.21.15.78:0/2335951163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e8004c50 con 0x562c1cc8b6b0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.811+0000 7fb4ecff9640 1 -- 172.21.15.78:0/2335951163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8010720 con 0x562c1cc8b6b0 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.811+0000 7fb50762c740 1 -- 172.21.15.78:0/2335951163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 msgr2=0x562c1d04fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.811+0000 7fb50762c740 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04fa20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800cbb0 tx=0x7fb4e800e720 comp rx=0 tx=0).stop 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- 172.21.15.78:0/2335951163 shutdown_connections 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04fa20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d04ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 --2- 172.21.15.78:0/2335951163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- 172.21.15.78:0/2335951163 >> 172.21.15.78:0/2335951163 conn(0x562c1cca04e0 msgr2=0x562c1d04a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- 172.21.15.78:0/2335951163 shutdown_connections 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- 172.21.15.78:0/2335951163 wait complete. 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- start start 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 Processor -- start 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d058540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d058a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d05f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c1d050530 con 0x562c1cc834d0 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562c1cf39dc0 con 0x562c1ccac390 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c1d0506b0 con 0x562c1cc8b6b0 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb50762c740 10 client.0 _ll_register_callbacks cb 0x562c1cf55600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb4fe519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d058540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb4fe519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d058540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45316/0 (socket says 172.21.15.78:45316) 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.812+0000 7fb4fe519640 1 -- 172.21.15.78:0/4045132514 learned_addr learned my addr 172.21.15.78:0/4045132514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d05f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d058a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fe519640 1 -- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 msgr2=0x562c1d058a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fe519640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d058a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fe519640 1 -- 172.21.15.78:0/4045132514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 msgr2=0x562c1d05f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fe519640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d05f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fe519640 1 -- 172.21.15.78:0/4045132514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1ccc43f0 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d05f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4fe519640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d058540 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f4009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c040969ea205cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/4045132514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4048050 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/4045132514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb4f40c7920 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e8004c50 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/4045132514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb4f40c7490 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb50762c740 10 client.16476 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562c1cd8e7d0 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.813+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/4045132514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e8004c50 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/4045132514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb4f404b060 con 0x562c1cc834d0 2024-04-25T14:03:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb50762c740 10 client.16476 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb50762c740 10 client.16476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb50762c740 20 client.16476 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb50762c740 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc6af60 0x562c1d065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb50762c740 4 client.16476 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c1cfc0990 con 0x562c1cc6af60 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc6af60 0x562c1d065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb4e1ffb640 10 client.16476 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.814+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc6af60 0x562c1d065fe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=deea1f353da8db1c server_cookie=f0efea2b6a31bc5a in_seq=0 out_seq=0 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.815+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/4045132514 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c1cfc0990 con 0x562c1cc6af60 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.815+0000 7fb4e1ffb640 10 client.16476 handle_command_reply: tid=0 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.815+0000 7fb50762c740 1 client.16476 shutdown 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.815+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 msgr2=0x562c1d058540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.815+0000 7fb50762c740 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d058540 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f4009290 comp rx=0 tx=0).stop 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 shutdown_connections 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d05f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc6af60 0x562c1d065fe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d058a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 --2- 172.21.15.78:0/4045132514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d058540 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb4e1ffb640 0 client.16476 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 >> 172.21.15.78:0/4045132514 conn(0x562c1cca04e0 msgr2=0x562c1d04cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 shutdown_connections 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 1 -- 172.21.15.78:0/4045132514 wait complete. 2024-04-25T14:03:02.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.816+0000 7fb50762c740 20 client.16476 trim_cache size 0 max 16384 2024-04-25T14:03:02.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 Processor -- start 2024-04-25T14:03:02.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 -- start start 2024-04-25T14:03:02.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d05ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81adb90 con 0x562c1cc8b6b0 2024-04-25T14:03:02.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81ab460 con 0x562c1cc834d0 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.829+0000 7fb50762c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a92b0 con 0x562c1ccac390 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45318/0 (socket says 172.21.15.78:45318) 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 -- 172.21.15.78:0/1828842649 learned_addr learned my addr 172.21.15.78:0/1828842649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fe519640 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d05ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 -- 172.21.15.78:0/1828842649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 msgr2=0x562c1d04d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 -- 172.21.15.78:0/1828842649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 msgr2=0x562c1d05ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d05ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 -- 172.21.15.78:0/1828842649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1d04e3d0 con 0x562c1cc8b6b0 2024-04-25T14:03:02.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04cea0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8005fd0 tx=0x7fb4e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ae5d8512427163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4e3fff640 1 -- 172.21.15.78:0/1828842649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e801c0d0 con 0x562c1cc8b6b0 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4e3fff640 1 -- 172.21.15.78:0/1828842649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e801c270 con 0x562c1cc8b6b0 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.830+0000 7fb4e3fff640 1 -- 172.21.15.78:0/1828842649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80278e0 con 0x562c1cc8b6b0 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 -- 172.21.15.78:0/1828842649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 msgr2=0x562c1d04cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04cea0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8005fd0 tx=0x7fb4e8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 -- 172.21.15.78:0/1828842649 shutdown_connections 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1ccac390 0x562c1d04d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc8b6b0 0x562c1d04cea0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 --2- 172.21.15.78:0/1828842649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1cc834d0 0x562c1d05ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 -- 172.21.15.78:0/1828842649 >> 172.21.15.78:0/1828842649 conn(0x562c1cca04e0 msgr2=0x562c1d0532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 -- 172.21.15.78:0/1828842649 shutdown_connections 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 -- 172.21.15.78:0/1828842649 wait complete. 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 -- start start 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.831+0000 7fb50762c740 1 Processor -- start 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a9920 con 0x562c1cc834d0 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81aad90 con 0x562c1ccac390 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81a9ff0 con 0x562c1cc8b6b0 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 10 client.0 _ll_register_callbacks cb 0x7fb4f4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d0c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45330/0 (socket says 172.21.15.78:45330) 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 -- 172.21.15.78:0/3811455124 learned_addr learned my addr 172.21.15.78:0/3811455124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 -- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 msgr2=0x562c1d0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d0c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 -- 172.21.15.78:0/3811455124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 msgr2=0x562c1d0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d0d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 -- 172.21.15.78:0/3811455124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1d0d1110 con 0x562c1cc834d0 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fed1a640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d0d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4fe519640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d0c96b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005ab0 tx=0x7fb4f40c3580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8aef74acb5ee0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/3811455124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4048070 con 0x562c1cc834d0 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/3811455124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb4f40aeb60 con 0x562c1cc834d0 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c1cfba800 con 0x562c1cc834d0 2024-04-25T14:03:02.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.832+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/3811455124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb4f40c6e30 con 0x562c1cc834d0 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 10 client.16482 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562c1d0d1110 con 0x562c1cc834d0 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/3811455124 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f40e7070 con 0x562c1cc834d0 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/3811455124 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb4f4102b20 con 0x562c1cc834d0 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 10 client.16482 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 10 client.16482 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 20 client.16482 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc403a0 0x562c1d0d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 4 client.16482 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c1cf35610 con 0x562c1cc403a0 2024-04-25T14:03:02.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.833+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc403a0 0x562c1d0d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb4e1ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:02.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb4fdd18640 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc403a0 0x562c1d0d49a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75f6398e98a24fef server_cookie=abcb1b8597ce027f in_seq=0 out_seq=0 2024-04-25T14:03:02.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb4e1ffb640 1 -- 172.21.15.78:0/3811455124 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c1cf35610 con 0x562c1cc403a0 2024-04-25T14:03:02.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb4e1ffb640 10 client.16482 handle_command_reply: tid=0 2024-04-25T14:03:02.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 client.16482 shutdown 2024-04-25T14:03:02.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 msgr2=0x562c1d0c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d0c96b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005ab0 tx=0x7fb4f40c3580 comp rx=0 tx=0).stop 2024-04-25T14:03:02.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 shutdown_connections 2024-04-25T14:03:02.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562c1ccac390 0x562c1d0d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562c1cc403a0 0x562c1d0d49a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c1cc8b6b0 0x562c1d0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb4e1ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:02.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 --2- 172.21.15.78:0/3811455124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c1cc834d0 0x562c1d0c96b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 >> 172.21.15.78:0/3811455124 conn(0x562c1cca04e0 msgr2=0x562c1d04cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 shutdown_connections 2024-04-25T14:03:02.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 1 -- 172.21.15.78:0/3811455124 wait complete. 2024-04-25T14:03:02.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50762c740 20 client.16482 trim_cache size 0 max 16384 2024-04-25T14:03:02.846 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4e40afaf0 msgr2=0x7fb4e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.834+0000 7fb50077d640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4e40afaf0 0x7fb4e40b2000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x562c1d04f860 tx=0x7fb4f0004a40 comp rx=0 tx=0).stop 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 msgr2=0x7fb4f819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f819a270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4032fd0 tx=0x7fb4f40042e0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 shutdown_connections 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4f8109a30 0x7fb4f81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4f8104d30 0x7fb4f819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4e40afaf0 0x7fb4e40b2000 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 --2- 172.21.15.78:0/2952799401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f8103af0 0x7fb4f819a270 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 >> 172.21.15.78:0/2952799401 conn(0x7fb4f80fece0 msgr2=0x7fb4f8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 shutdown_connections 2024-04-25T14:03:02.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.835+0000 7fb50077d640 1 -- 172.21.15.78:0/2952799401 wait complete. 2024-04-25T14:03:02.849 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:03:02.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 -- 172.21.15.78:0/4275614682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1098b0 msgr2=0x7f9c1c10bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 --2- 172.21.15.78:0/4275614682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1098b0 0x7f9c1c10bcd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009ca0 tx=0x7f9c1402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 -- 172.21.15.78:0/4275614682 shutdown_connections 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 --2- 172.21.15.78:0/4275614682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1098b0 0x7f9c1c10bcd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 --2- 172.21.15.78:0/4275614682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c106f50 0x7f9c1c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 --2- 172.21.15.78:0/4275614682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c1045f0 0x7f9c1c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 -- 172.21.15.78:0/4275614682 >> 172.21.15.78:0/4275614682 conn(0x7f9c1c0fdbf0 msgr2=0x7f9c1c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 -- 172.21.15.78:0/4275614682 shutdown_connections 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.959+0000 7f9c227b2640 1 -- 172.21.15.78:0/4275614682 wait complete. 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 Processor -- start 2024-04-25T14:03:02.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 -- start start 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1045f0 0x7f9c1c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c106f50 0x7f9c1c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c1098b0 0x7f9c1c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10f630 con 0x7f9c1c1045f0 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10f4b0 con 0x7f9c1c1098b0 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c227b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10f7b0 con 0x7f9c1c106f50 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1045f0 0x7f9c1c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1045f0 0x7f9c1c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45346/0 (socket says 172.21.15.78:45346) 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c1bfff640 1 -- 172.21.15.78:0/2855255511 learned_addr learned my addr 172.21.15.78:0/2855255511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.960+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c1098b0 0x7f9c1c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c106f50 0x7f9c1c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c1bfff640 1 -- 172.21.15.78:0/2855255511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c106f50 msgr2=0x7f9c1c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c106f50 0x7f9c1c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c1bfff640 1 -- 172.21.15.78:0/2855255511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c1098b0 msgr2=0x7f9c1c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c1098b0 0x7f9c1c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c1bfff640 1 -- 172.21.15.78:0/2855255511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c1a1d70 con 0x7f9c1c1045f0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1045f0 0x7f9c1c19a230 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00d740 tx=0x7f9c0c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c197fa640 1 -- 172.21.15.78:0/2855255511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c017070 con 0x7f9c1c1045f0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c1a2060 con 0x7f9c1c1045f0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c197fa640 1 -- 172.21.15.78:0/2855255511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0c00de50 con 0x7f9c1c1045f0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c197fa640 1 -- 172.21.15.78:0/2855255511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c004ce0 con 0x7f9c1c1045f0 2024-04-25T14:03:02.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.961+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c1a2560 con 0x7f9c1c1045f0 2024-04-25T14:03:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 Processor -- start 2024-04-25T14:03:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 -- start start 2024-04-25T14:03:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c3001a390 0x559c303bc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff14d0 0x559c303bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c197fa640 1 -- 172.21.15.78:0/2855255511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9c0c004e80 con 0x7f9c1c1045f0 2024-04-25T14:03:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c30320d80 con 0x559c2fff14d0 2024-04-25T14:03:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559c302daa20 con 0x559c2fff96b0 2024-04-25T14:03:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c29664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c3031cd40 con 0x559c3001a390 2024-04-25T14:03:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff14d0 0x559c303bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff14d0 0x559c303bcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45348/0 (socket says 172.21.15.78:45348) 2024-04-25T14:03:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.964+0000 7f9c1bfff640 1 -- 172.21.15.78:0/278088278 learned_addr learned my addr 172.21.15.78:0/278088278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c3001a390 0x559c303bc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1bfff640 1 -- 172.21.15.78:0/278088278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c3001a390 msgr2=0x559c303bc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c3001a390 0x559c303bc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1bfff640 1 -- 172.21.15.78:0/278088278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 msgr2=0x559c303bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303bda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1bfff640 1 -- 172.21.15.78:0/278088278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c302e5c30 con 0x559c2fff14d0 2024-04-25T14:03:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c3001a390 0x559c303bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c197fa640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9c040afaf0 0x7f9c040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9c040afaf0 0x7f9c040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff14d0 0x559c303bcd90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0e4c10 tx=0x7f9c0c0e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a5762e574e5e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c197fa640 1 -- 172.21.15.78:0/2855255511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9c0c0de690 con 0x7f9c1c1045f0 2024-04-25T14:03:02.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c027fc640 1 -- 172.21.15.78:0/278088278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c020020 con 0x559c2fff14d0 2024-04-25T14:03:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c027fc640 1 -- 172.21.15.78:0/278088278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0c0af310 con 0x559c2fff14d0 2024-04-25T14:03:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.965+0000 7f9c027fc640 1 -- 172.21.15.78:0/278088278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0a6930 con 0x559c2fff14d0 2024-04-25T14:03:02.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 -- 172.21.15.78:0/278088278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff14d0 msgr2=0x559c303bcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9c040afaf0 0x7f9c040b2000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9c14031b50 tx=0x7f9c140094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff14d0 0x559c303bcd90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0e4c10 tx=0x7f9c0c0e6690 comp rx=0 tx=0).stop 2024-04-25T14:03:02.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 -- 172.21.15.78:0/278088278 shutdown_connections 2024-04-25T14:03:02.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff14d0 0x559c303bcd90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 --2- 172.21.15.78:0/278088278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c3001a390 0x559c303bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 -- 172.21.15.78:0/278088278 >> 172.21.15.78:0/278088278 conn(0x559c3000e4e0 msgr2=0x559c303b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 -- 172.21.15.78:0/278088278 shutdown_connections 2024-04-25T14:03:02.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 -- 172.21.15.78:0/278088278 wait complete. 2024-04-25T14:03:02.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.966+0000 7f9c29664740 1 -- start start 2024-04-25T14:03:02.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 1 Processor -- start 2024-04-25T14:03:02.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff96b0 0x559c303c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c3001a390 0x559c303cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c303be530 con 0x559c2fff96b0 2024-04-25T14:03:02.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559c302a7dc0 con 0x559c3001a390 2024-04-25T14:03:02.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c303be6b0 con 0x559c2fff14d0 2024-04-25T14:03:02.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c29664740 10 client.0 _ll_register_callbacks cb 0x559c302c3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:02.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff96b0 0x559c303c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff96b0 0x559c303c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45358/0 (socket says 172.21.15.78:45358) 2024-04-25T14:03:02.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 -- 172.21.15.78:0/2879343389 learned_addr learned my addr 172.21.15.78:0/2879343389 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:02.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c3001a390 0x559c303cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 -- 172.21.15.78:0/2879343389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 msgr2=0x559c303c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 -- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c3001a390 msgr2=0x559c303cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:02.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c3001a390 0x559c303cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c1b7fe640 1 -- 172.21.15.78:0/2879343389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c300323f0 con 0x559c2fff96b0 2024-04-25T14:03:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.967+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c3001a390 0x559c303cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff96b0 0x559c303c6ad0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9c10010740 tx=0x7f9c10010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614300a566b44ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9bfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/2879343389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000cd20 con 0x559c2fff96b0 2024-04-25T14:03:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/2879343389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9c10004020 con 0x559c2fff96b0 2024-04-25T14:03:02.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0c0af310 con 0x559c2fff96b0 2024-04-25T14:03:02.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/2879343389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9c10010e50 con 0x559c2fff96b0 2024-04-25T14:03:02.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 10 client.16494 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:02.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559c300fc7d0 con 0x559c2fff96b0 2024-04-25T14:03:02.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/2879343389 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c0c0af310 con 0x559c2fff96b0 2024-04-25T14:03:02.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/2879343389 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9c10011440 con 0x559c2fff96b0 2024-04-25T14:03:02.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 10 client.16494 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:02.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:02.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 20 client.16494 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:02.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffd8f60 0x559c303d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:02.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 4 client.16494 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:02.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.968+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c3032e990 con 0x559c2ffd8f60 2024-04-25T14:03:02.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.969+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffd8f60 0x559c303d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:02.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.969+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffd8f60 0x559c303d4340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49789e5f75e9f811 server_cookie=e069e14a0ce7b5b8 in_seq=0 out_seq=0 2024-04-25T14:03:02.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.969+0000 7f9bfb7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:02.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/2879343389 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559c3032e990 con 0x559c2ffd8f60 2024-04-25T14:03:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9bfb7fe640 10 client.16494 handle_command_reply: tid=0 2024-04-25T14:03:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 client.16494 shutdown 2024-04-25T14:03:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff96b0 msgr2=0x559c303c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff96b0 0x559c303c6ad0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9c10010740 tx=0x7f9c10010c00 comp rx=0 tx=0).stop 2024-04-25T14:03:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 shutdown_connections 2024-04-25T14:03:02.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c3001a390 0x559c303cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c2fff96b0 0x559c303c6ad0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffd8f60 0x559c303d4340 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 --2- 172.21.15.78:0/2879343389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:02.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 >> 172.21.15.78:0/2879343389 conn(0x559c3000e4e0 msgr2=0x559c303baf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:02.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9bfb7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:02.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 shutdown_connections 2024-04-25T14:03:02.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 1 -- 172.21.15.78:0/2879343389 wait complete. 2024-04-25T14:03:02.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.970+0000 7f9c29664740 20 client.16494 trim_cache size 0 max 16384 2024-04-25T14:03:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 Processor -- start 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 -- start start 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff14d0 0x559c303c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff96b0 0x559c303cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1adb80 con 0x559c3001a390 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1ab450 con 0x559c2fff14d0 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c29664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1a92a0 con 0x559c2fff96b0 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45362/0 (socket says 172.21.15.78:45362) 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 -- 172.21.15.78:0/4055883494 learned_addr learned my addr 172.21.15.78:0/4055883494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff14d0 0x559c303c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff96b0 0x559c303cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 -- 172.21.15.78:0/4055883494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff96b0 msgr2=0x559c303cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff96b0 0x559c303cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 -- 172.21.15.78:0/4055883494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff14d0 msgr2=0x559c303c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff14d0 0x559c303c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 -- 172.21.15.78:0/4055883494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c303c5970 con 0x559c3001a390 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff14d0 0x559c303c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.984+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303b6610 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c075eb0 tx=0x7f9c14040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a7212a364a63a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c01ffb640 1 -- 172.21.15.78:0/4055883494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1402f660 con 0x559c3001a390 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c01ffb640 1 -- 172.21.15.78:0/4055883494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c14036040 con 0x559c3001a390 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c01ffb640 1 -- 172.21.15.78:0/4055883494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14030d10 con 0x559c3001a390 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c29664740 1 -- 172.21.15.78:0/4055883494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 msgr2=0x559c303b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c29664740 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303b6610 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c075eb0 tx=0x7f9c14040000 comp rx=0 tx=0).stop 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c29664740 1 -- 172.21.15.78:0/4055883494 shutdown_connections 2024-04-25T14:03:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c29664740 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303b6610 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c29664740 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff96b0 0x559c303cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c29664740 1 --2- 172.21.15.78:0/4055883494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff14d0 0x559c303c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.985+0000 7f9c29664740 1 -- 172.21.15.78:0/4055883494 >> 172.21.15.78:0/4055883494 conn(0x559c3000e4e0 msgr2=0x559c303c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 -- 172.21.15.78:0/4055883494 shutdown_connections 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 -- 172.21.15.78:0/4055883494 wait complete. 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 -- start start 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 Processor -- start 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1a9910 con 0x559c3001a390 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1aad80 con 0x559c2fff96b0 2024-04-25T14:03:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1a9fe0 con 0x559c2fff14d0 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c29664740 10 client.0 _ll_register_callbacks cb 0x7f9c1000e370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c20d4f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c20d4f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45364/0 (socket says 172.21.15.78:45364) 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.986+0000 7f9c20d4f640 1 -- 172.21.15.78:0/3228863658 learned_addr learned my addr 172.21.15.78:0/3228863658 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c1b7fe640 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c20d4f640 1 -- 172.21.15.78:0/3228863658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 msgr2=0x559c303caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c20d4f640 1 -- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 msgr2=0x559c303cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c20d4f640 1 -- 172.21.15.78:0/3228863658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c303cde90 con 0x559c3001a390 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c20d4f640 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303cc370 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9c14030fe0 tx=0x7f9c14030850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320f1d88347bd7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9bfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:03.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/3228863658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1402f9b0 con 0x559c3001a390 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/3228863658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9c1402fb50 con 0x559c3001a390 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c14036040 con 0x559c3001a390 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/3228863658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9c1402fd50 con 0x559c3001a390 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c29664740 10 client.16500 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559c303cd9e0 con 0x559c3001a390 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.987+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/3228863658 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c14036040 con 0x559c3001a390 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/3228863658 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9c14044040 con 0x559c3001a390 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c29664740 10 client.16500 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c29664740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c29664740 20 client.16500 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c29664740 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffae3a0 0x559c3043ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c29664740 4 client.16500 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:03.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c303bd190 con 0x559c2ffae3a0 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffae3a0 0x559c3043ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9c1bfff640 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffae3a0 0x559c3043ec60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10de27178e38a63f server_cookie=e68755c46405d541 in_seq=0 out_seq=0 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.988+0000 7f9bfb7fe640 10 client.16500 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9bfb7fe640 1 -- 172.21.15.78:0/3228863658 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c303bd190 con 0x559c2ffae3a0 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9bfb7fe640 10 client.16500 handle_command_reply: tid=0 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 client.16500 shutdown 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 msgr2=0x559c303cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303cc370 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9c14030fe0 tx=0x7f9c14030850 comp rx=0 tx=0).stop 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 shutdown_connections 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c3001a390 0x559c303cc370 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559c2fff96b0 0x559c303cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559c2ffae3a0 0x559c3043ec60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 --2- 172.21.15.78:0/3228863658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c2fff14d0 0x559c303caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 >> 172.21.15.78:0/3228863658 conn(0x559c3000e4e0 msgr2=0x559c303bae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:03.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9bfb7fe640 0 client.16500 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 shutdown_connections 2024-04-25T14:03:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 1 -- 172.21.15.78:0/3228863658 wait complete. 2024-04-25T14:03:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c29664740 20 client.16500 trim_cache size 0 max 16384 2024-04-25T14:03:03.008 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:03:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9c040afaf0 msgr2=0x7f9c040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c227b2640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9c040afaf0 0x7f9c040b2000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9c14031b50 tx=0x7f9c140094a0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1045f0 msgr2=0x7f9c1c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.989+0000 7f9c227b2640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1045f0 0x7f9c1c19a230 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00d740 tx=0x7f9c0c00dc00 comp rx=0 tx=0).stop 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 shutdown_connections 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c1c1098b0 0x7f9c1c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c1c106f50 0x7f9c1c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9c040afaf0 0x7f9c040b2000 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 --2- 172.21.15.78:0/2855255511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c1c1045f0 0x7f9c1c19a230 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 >> 172.21.15.78:0/2855255511 conn(0x7f9c1c0fdbf0 msgr2=0x7f9c1c105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 shutdown_connections 2024-04-25T14:03:03.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:02.990+0000 7f9c227b2640 1 -- 172.21.15.78:0/2855255511 wait complete. 2024-04-25T14:03:03.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:03 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1643334874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:03.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:03 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1643334874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:03.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:03 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1643334874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:04.012 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:03:04.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:04.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:03:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 -- 172.21.15.78:0/3472101211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0441029e0 msgr2=0x7fa0441036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/3472101211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0441029e0 0x7fa0441036e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa040009900 tx=0x7fa04002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 -- 172.21.15.78:0/3472101211 shutdown_connections 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/3472101211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa044108920 0x7fa04410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/3472101211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa044103c20 0x7fa044104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/3472101211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0441029e0 0x7fa0441036e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 -- 172.21.15.78:0/3472101211 >> 172.21.15.78:0/3472101211 conn(0x7fa0440fdbd0 msgr2=0x7fa044100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 -- 172.21.15.78:0/3472101211 shutdown_connections 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.141+0000 7fa04b6d1640 1 -- 172.21.15.78:0/3472101211 wait complete. 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 Processor -- start 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 -- start start 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441029e0 0x7fa04406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa044103c20 0x7fa0441a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa044108920 0x7fa0441a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa04410e4e0 con 0x7fa044108920 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa04410e360 con 0x7fa0441029e0 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04b6d1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa04410e660 con 0x7fa044103c20 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa044108920 0x7fa0441a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa044108920 0x7fa0441a3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45392/0 (socket says 172.21.15.78:45392) 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 -- 172.21.15.78:0/1010491128 learned_addr learned my addr 172.21.15.78:0/1010491128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa044103c20 0x7fa0441a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04946d640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441029e0 0x7fa04406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 -- 172.21.15.78:0/1010491128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa044103c20 msgr2=0x7fa0441a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa044103c20 0x7fa0441a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 -- 172.21.15.78:0/1010491128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441029e0 msgr2=0x7fa04406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441029e0 0x7fa04406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa049c6e640 1 -- 172.21.15.78:0/1010491128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0441a42d0 con 0x7fa044108920 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa04946d640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441029e0 0x7fa04406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.142+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa044103c20 0x7fa0441a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.143+0000 7fa049c6e640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa044108920 0x7fa0441a3c90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009fd0 tx=0x7fa03c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.143+0000 7fa03a7fc640 1 -- 172.21.15.78:0/1010491128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c017070 con 0x7fa044108920 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.143+0000 7fa03a7fc640 1 -- 172.21.15.78:0/1010491128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa03c003e20 con 0x7fa044108920 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.143+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441a45c0 con 0x7fa044108920 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.143+0000 7fa03a7fc640 1 -- 172.21.15.78:0/1010491128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c010460 con 0x7fa044108920 2024-04-25T14:03:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.143+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0441a4b10 con 0x7fa044108920 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 Processor -- start 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa03a7fc640 1 -- 172.21.15.78:0/1010491128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa03c003970 con 0x7fa044108920 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 -- start start 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80ddfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80de6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558b80d44b00 con 0x558b80a3e410 2024-04-25T14:03:04.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558b80ce0920 con 0x558b80a15550 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa05257f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558b80d40ac0 con 0x558b80a1d730 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45402/0 (socket says 172.21.15.78:45402) 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 -- 172.21.15.78:0/398482406 learned_addr learned my addr 172.21.15.78:0/398482406 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80de6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa03a7fc640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa0300afb70 0x7fa0300b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa04946d640 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80ddfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 -- 172.21.15.78:0/398482406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 msgr2=0x558b80de6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80de6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 -- 172.21.15.78:0/398482406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 msgr2=0x558b80ddfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80ddfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa048c6c640 1 -- 172.21.15.78:0/398482406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b80b1e7e0 con 0x558b80a3e410 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa04946d640 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80ddfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:04.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.146+0000 7fa049c6e640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa0300afb70 0x7fa0300b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.147+0000 7fa048c6c640 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddf4a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa03400be80 tx=0x7fa03400b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccc1020ff4161fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.147+0000 7fa03a7fc640 1 -- 172.21.15.78:0/1010491128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa03c0dea00 con 0x7fa044108920 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.147+0000 7fa02f7fe640 1 -- 172.21.15.78:0/398482406 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034019030 con 0x558b80a3e410 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.147+0000 7fa02f7fe640 1 -- 172.21.15.78:0/398482406 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0340119f0 con 0x558b80a3e410 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.147+0000 7fa02f7fe640 1 -- 172.21.15.78:0/398482406 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400f870 con 0x558b80a3e410 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 -- 172.21.15.78:0/398482406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 msgr2=0x558b80ddf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddf4a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa03400be80 tx=0x7fa03400b750 comp rx=0 tx=0).stop 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa049c6e640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa0300afb70 0x7fa0300b2080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa03c09fd70 tx=0x7fa03c0d5620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 -- 172.21.15.78:0/398482406 shutdown_connections 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80de6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80ddfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 --2- 172.21.15.78:0/398482406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddf4a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 -- 172.21.15.78:0/398482406 >> 172.21.15.78:0/398482406 conn(0x558b80a32560 msgr2=0x558b80ddae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 -- 172.21.15.78:0/398482406 shutdown_connections 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 -- 172.21.15.78:0/398482406 wait complete. 2024-04-25T14:03:04.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 -- start start 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.148+0000 7fa05257f740 1 Processor -- start 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa05257f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80de92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa05257f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a1d730 0x558b80de9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa05257f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a3e410 0x558b80df0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa05257f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558b80de7370 con 0x558b80a1d730 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa05257f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558b80cde5d0 con 0x558b80a15550 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa05257f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558b80de74f0 con 0x558b80a3e410 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa05257f740 10 client.0 _ll_register_callbacks cb 0x558b80ccc080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa04946d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a1d730 0x558b80de9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa04946d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a1d730 0x558b80de9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45404/0 (socket says 172.21.15.78:45404) 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa04946d640 1 -- 172.21.15.78:0/1042484742 learned_addr learned my addr 172.21.15.78:0/1042484742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a3e410 0x558b80df0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80de92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa04946d640 1 -- 172.21.15.78:0/1042484742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a3e410 msgr2=0x558b80df0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.149+0000 7fa04946d640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a3e410 0x558b80df0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa04946d640 1 -- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 msgr2=0x558b80de92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa04946d640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80de92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa04946d640 1 -- 172.21.15.78:0/1042484742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b80b7ef70 con 0x558b80a1d730 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a3e410 0x558b80df0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa04946d640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a1d730 0x558b80de9830 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa040000c00 tx=0x7fa0400317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fb7be2b91862b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa02cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa02cff9640 1 -- 172.21.15.78:0/1042484742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040035030 con 0x558b80a1d730 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa02cff9640 1 -- 172.21.15.78:0/1042484742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa040004450 con 0x558b80a1d730 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0340119f0 con 0x558b80a1d730 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa02cff9640 1 -- 172.21.15.78:0/1042484742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa04004b050 con 0x558b80a1d730 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa05257f740 10 client.16512 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558b80a16170 con 0x558b80a1d730 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.150+0000 7fa02cff9640 1 -- 172.21.15.78:0/1042484742 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0340119f0 con 0x558b80a1d730 2024-04-25T14:03:04.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa02cff9640 1 -- 172.21.15.78:0/1042484742 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa04003a040 con 0x558b80a1d730 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa05257f740 10 client.16512 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa05257f740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa05257f740 20 client.16512 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa05257f740 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809fd1f0 0x558b80df6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa05257f740 4 client.16512 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b80d52710 con 0x558b809fd1f0 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809fd1f0 0x558b80df6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809fd1f0 0x558b80df6c90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa42f2c0999786a server_cookie=a795e6ea268136bb in_seq=0 out_seq=0 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.151+0000 7fa02cff9640 10 client.16512 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa02cff9640 1 -- 172.21.15.78:0/1042484742 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b80d52710 con 0x558b809fd1f0 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa02cff9640 10 client.16512 handle_command_reply: tid=0 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 client.16512 shutdown 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a1d730 msgr2=0x558b80de9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a1d730 0x558b80de9830 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa040000c00 tx=0x7fa0400317d0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 shutdown_connections 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809fd1f0 0x558b80df6c90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a3e410 0x558b80df0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a1d730 0x558b80de9830 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 --2- 172.21.15.78:0/1042484742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a15550 0x558b80de92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa02cff9640 0 client.16512 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 >> 172.21.15.78:0/1042484742 conn(0x558b80a32560 msgr2=0x558b80dd9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 shutdown_connections 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.152+0000 7fa05257f740 1 -- 172.21.15.78:0/1042484742 wait complete. 2024-04-25T14:03:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.153+0000 7fa05257f740 20 client.16512 trim_cache size 0 max 16384 2024-04-25T14:03:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 Processor -- start 2024-04-25T14:03:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 -- start start 2024-04-25T14:03:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a15550 0x558b80de7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a1d730 0x558b80dda620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441afd80 con 0x558b80a3e410 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441ad650 con 0x558b80a1d730 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa05257f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441ab4a0 con 0x558b80a15550 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45420/0 (socket says 172.21.15.78:45420) 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 -- 172.21.15.78:0/1827735035 learned_addr learned my addr 172.21.15.78:0/1827735035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a15550 0x558b80de7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa04946d640 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a1d730 0x558b80dda620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 -- 172.21.15.78:0/1827735035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a15550 msgr2=0x558b80de7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a15550 0x558b80de7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 -- 172.21.15.78:0/1827735035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a1d730 msgr2=0x558b80dda620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a1d730 0x558b80dda620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa048c6c640 1 -- 172.21.15.78:0/1827735035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b80de41a0 con 0x558b80a3e410 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.173+0000 7fa04946d640 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a1d730 0x558b80dda620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.174+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddab60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa034013710 tx=0x7fa034015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3644c3f3e0273b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.174+0000 7fa02effd640 1 -- 172.21.15.78:0/1827735035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034019040 con 0x558b80a3e410 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.174+0000 7fa02effd640 1 -- 172.21.15.78:0/1827735035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa034011810 con 0x558b80a3e410 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.174+0000 7fa02effd640 1 -- 172.21.15.78:0/1827735035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03402e7c0 con 0x558b80a3e410 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 -- 172.21.15.78:0/1827735035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 msgr2=0x558b80ddab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddab60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa034013710 tx=0x7fa034015040 comp rx=0 tx=0).stop 2024-04-25T14:03:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 -- 172.21.15.78:0/1827735035 shutdown_connections 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a3e410 0x558b80ddab60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a1d730 0x558b80dda620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 --2- 172.21.15.78:0/1827735035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a15550 0x558b80de7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 -- 172.21.15.78:0/1827735035 >> 172.21.15.78:0/1827735035 conn(0x558b80a32560 msgr2=0x558b80de7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 -- 172.21.15.78:0/1827735035 shutdown_connections 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 -- 172.21.15.78:0/1827735035 wait complete. 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 -- start start 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.175+0000 7fa05257f740 1 Processor -- start 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a15550 0x558b80ded1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80ded6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a3e410 0x558b80dee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441abb10 con 0x558b80a15550 2024-04-25T14:03:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441acf80 con 0x558b80a3e410 2024-04-25T14:03:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441ac1e0 con 0x558b80a1d730 2024-04-25T14:03:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 10 client.0 _ll_register_callbacks cb 0x7fa040005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a15550 0x558b80ded1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a15550 0x558b80ded1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45426/0 (socket says 172.21.15.78:45426) 2024-04-25T14:03:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 -- 172.21.15.78:0/1846069506 learned_addr learned my addr 172.21.15.78:0/1846069506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a3e410 0x558b80dee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa04946d640 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80ded6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 -- 172.21.15.78:0/1846069506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 msgr2=0x558b80ded6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80ded6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 -- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a3e410 msgr2=0x558b80dee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a3e410 0x558b80dee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 -- 172.21.15.78:0/1846069506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b80deebd0 con 0x558b80a15550 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa048c6c640 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a15550 0x558b80ded1a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0340130f0 tx=0x7fa03400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e437ac39753cf15a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa02cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa02cff9640 1 -- 172.21.15.78:0/1846069506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03403e070 con 0x558b80a15550 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa02cff9640 1 -- 172.21.15.78:0/1846069506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa034005020 con 0x558b80a15550 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa034011810 con 0x558b80a15550 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.176+0000 7fa02cff9640 1 -- 172.21.15.78:0/1846069506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa034049050 con 0x558b80a15550 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 10 client.16518 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558b80defe00 con 0x558b80a15550 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa02cff9640 1 -- 172.21.15.78:0/1846069506 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa034011810 con 0x558b80a15550 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa02cff9640 1 -- 172.21.15.78:0/1846069506 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa034004680 con 0x558b80a15550 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 10 client.16518 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 20 client.16518 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809d2630 0x558b80e612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 4 client.16518 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b80d4c580 con 0x558b809d2630 2024-04-25T14:03:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.177+0000 7fa04946d640 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809d2630 0x558b80e612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.178+0000 7fa02cff9640 10 client.16518 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.178+0000 7fa04946d640 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809d2630 0x558b80e612f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c5e2675a10797b server_cookie=70a212fb7e92f1a in_seq=0 out_seq=0 2024-04-25T14:03:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa02cff9640 1 -- 172.21.15.78:0/1846069506 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b80d4c580 con 0x558b809d2630 2024-04-25T14:03:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa02cff9640 10 client.16518 handle_command_reply: tid=0 2024-04-25T14:03:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 client.16518 shutdown 2024-04-25T14:03:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a15550 msgr2=0x558b80ded1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a15550 0x558b80ded1a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0340130f0 tx=0x7fa03400b730 comp rx=0 tx=0).stop 2024-04-25T14:03:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 shutdown_connections 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558b80a3e410 0x558b80dee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558b809d2630 0x558b80e612f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b80a1d730 0x558b80ded6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 --2- 172.21.15.78:0/1846069506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b80a15550 0x558b80ded1a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa02cff9640 0 client.16518 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 >> 172.21.15.78:0/1846069506 conn(0x558b80a32560 msgr2=0x558b80dddc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 shutdown_connections 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 1 -- 172.21.15.78:0/1846069506 wait complete. 2024-04-25T14:03:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.179+0000 7fa05257f740 20 client.16518 trim_cache size 0 max 16384 2024-04-25T14:03:04.188 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:03:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:03:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "3e384998-5633-4752-85f6-b34b356d8419", 2024-04-25T14:03:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:03:04.188 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:03:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa0300afb70 msgr2=0x7fa0300b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa0300afb70 0x7fa0300b2080 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa03c09fd70 tx=0x7fa03c0d5620 comp rx=0 tx=0).stop 2024-04-25T14:03:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa044108920 msgr2=0x7fa0441a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa044108920 0x7fa0441a3c90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009fd0 tx=0x7fa03c00b260 comp rx=0 tx=0).stop 2024-04-25T14:03:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 shutdown_connections 2024-04-25T14:03:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa0300afb70 0x7fa0300b2080 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa044108920 0x7fa0441a3c90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa044103c20 0x7fa0441a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 --2- 172.21.15.78:0/1010491128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0441029e0 0x7fa04406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 >> 172.21.15.78:0/1010491128 conn(0x7fa0440fdbd0 msgr2=0x7fa044104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 shutdown_connections 2024-04-25T14:03:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.180+0000 7fa04b6d1640 1 -- 172.21.15.78:0/1010491128 wait complete. 2024-04-25T14:03:04.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:04.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:04.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 -- 172.21.15.78:0/3496832477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6941029f0 msgr2=0x7fe6941036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/3496832477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6941029f0 0x7fe6941036f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe690009900 tx=0x7fe69002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 -- 172.21.15.78:0/3496832477 shutdown_connections 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/3496832477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe694108930 0x7fe69410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/3496832477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe694103c30 0x7fe694104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/3496832477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6941029f0 0x7fe6941036f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 -- 172.21.15.78:0/3496832477 >> 172.21.15.78:0/3496832477 conn(0x7fe6940fdbc0 msgr2=0x7fe694100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 -- 172.21.15.78:0/3496832477 shutdown_connections 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.304+0000 7fe69b8e0640 1 -- 172.21.15.78:0/3496832477 wait complete. 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 Processor -- start 2024-04-25T14:03:04.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 -- start start 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6941029f0 0x7fe694075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe694103c30 0x7fe69419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe694108930 0x7fe6941a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410e540 con 0x7fe694103c30 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410e3c0 con 0x7fe6941029f0 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69b8e0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410e6c0 con 0x7fe694108930 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe694103c30 0x7fe69419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe694103c30 0x7fe69419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45442/0 (socket says 172.21.15.78:45442) 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe698e7b640 1 -- 172.21.15.78:0/1297086899 learned_addr learned my addr 172.21.15.78:0/1297086899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.305+0000 7fe69967c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6941029f0 0x7fe694075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe699e7d640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe694108930 0x7fe6941a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe698e7b640 1 -- 172.21.15.78:0/1297086899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe694108930 msgr2=0x7fe6941a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe698e7b640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe694108930 0x7fe6941a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe698e7b640 1 -- 172.21.15.78:0/1297086899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6941029f0 msgr2=0x7fe694075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe698e7b640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6941029f0 0x7fe694075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe698e7b640 1 -- 172.21.15.78:0/1297086899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6941a1d70 con 0x7fe694103c30 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe69967c640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6941029f0 0x7fe694075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.306+0000 7fe698e7b640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe694103c30 0x7fe69419f000 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe68400e9f0 tx=0x7fe68400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.308+0000 7fe68a7fc640 1 -- 172.21.15.78:0/1297086899 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400cd80 con 0x7fe694103c30 2024-04-25T14:03:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.308+0000 7fe68a7fc640 1 -- 172.21.15.78:0/1297086899 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6840108e0 con 0x7fe694103c30 2024-04-25T14:03:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.308+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941a2060 con 0x7fe694103c30 2024-04-25T14:03:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.308+0000 7fe68a7fc640 1 -- 172.21.15.78:0/1297086899 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684018980 con 0x7fe694103c30 2024-04-25T14:03:04.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.308+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a2600 con 0x7fe694103c30 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe68a7fc640 1 -- 172.21.15.78:0/1297086899 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe684010430 con 0x7fe694103c30 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 Processor -- start 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 -- start start 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac408a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac408fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac409c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5625ac36dd80 con 0x5625ac0466b0 2024-04-25T14:03:04.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625ac3278b0 con 0x5625ac03e4d0 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe6a2794740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5625ac369d40 con 0x5625ac067390 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac409c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac409c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45454/0 (socket says 172.21.15.78:45454) 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe698e7b640 1 -- 172.21.15.78:0/213361825 learned_addr learned my addr 172.21.15.78:0/213361825 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe69967c640 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac408fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.311+0000 7fe699e7d640 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac408a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe68a7fc640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe6800afb70 0x7fe6800b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe699e7d640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe6800afb70 0x7fe6800b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe68a7fc640 1 -- 172.21.15.78:0/1297086899 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe6840deeb0 con 0x7fe694103c30 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe698e7b640 1 -- 172.21.15.78:0/213361825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 msgr2=0x5625ac408a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe698e7b640 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac408a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe698e7b640 1 -- 172.21.15.78:0/213361825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 msgr2=0x5625ac408fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe698e7b640 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac408fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe698e7b640 1 -- 172.21.15.78:0/213361825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625ac2f8a40 con 0x5625ac0466b0 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe699e7d640 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac408a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe698e7b640 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac409c70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe6840dc8e0 tx=0x7fe68400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc544d1293d2d575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe699e7d640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe6800afb70 0x7fe6800b2080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5625ac409ab0 tx=0x7fe68c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe67f7fe640 1 -- 172.21.15.78:0/213361825 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840dce60 con 0x5625ac0466b0 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe67f7fe640 1 -- 172.21.15.78:0/213361825 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6840e25a0 con 0x5625ac0466b0 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.312+0000 7fe67f7fe640 1 -- 172.21.15.78:0/213361825 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840a1bc0 con 0x5625ac0466b0 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 -- 172.21.15.78:0/213361825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 msgr2=0x5625ac409c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac409c70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe6840dc8e0 tx=0x7fe68400cf20 comp rx=0 tx=0).stop 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 -- 172.21.15.78:0/213361825 shutdown_connections 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac409c70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac408fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 --2- 172.21.15.78:0/213361825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac408a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 -- 172.21.15.78:0/213361825 >> 172.21.15.78:0/213361825 conn(0x5625ac05b4e0 msgr2=0x5625ac404270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 -- 172.21.15.78:0/213361825 shutdown_connections 2024-04-25T14:03:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 -- 172.21.15.78:0/213361825 wait complete. 2024-04-25T14:03:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.313+0000 7fe6a2794740 1 -- start start 2024-04-25T14:03:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 1 Processor -- start 2024-04-25T14:03:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac416c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac4171a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac067390 0x5625ac41e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5625ac40a6c0 con 0x5625ac0466b0 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625ac2f4d10 con 0x5625ac067390 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5625ac40a840 con 0x5625ac03e4d0 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe6a2794740 10 client.0 _ll_register_callbacks cb 0x5625ac321210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac4171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe69967c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac416c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac4171a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45458/0 (socket says 172.21.15.78:45458) 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 -- 172.21.15.78:0/3651529887 learned_addr learned my addr 172.21.15.78:0/3651529887 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe699e7d640 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac067390 0x5625ac41e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 -- 172.21.15.78:0/3651529887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 msgr2=0x5625ac416c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac416c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 -- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac067390 msgr2=0x5625ac41e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac067390 0x5625ac41e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.314+0000 7fe698e7b640 1 -- 172.21.15.78:0/3651529887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ac1497d0 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe698e7b640 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac4171a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe6840dcd10 tx=0x7fe6840a3960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df560c381723e769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe67cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe67cff9640 1 -- 172.21.15.78:0/3651529887 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840dce60 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe67cff9640 1 -- 172.21.15.78:0/3651529887 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe684018b20 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5625ac37b990 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe67cff9640 1 -- 172.21.15.78:0/3651529887 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe6840a7430 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 10 client.16530 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5625ac07f3f0 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe67cff9640 1 -- 172.21.15.78:0/3651529887 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5625ac37b990 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe67cff9640 1 -- 172.21.15.78:0/3651529887 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe6840aeab0 con 0x5625ac0466b0 2024-04-25T14:03:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 10 client.16530 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 20 client.16530 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625ac025f60 0x5625ac4241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 4 client.16530 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.315+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625ac375800 con 0x5625ac025f60 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.316+0000 7fe69967c640 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625ac025f60 0x5625ac4241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.316+0000 7fe67cff9640 10 client.16530 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.316+0000 7fe69967c640 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625ac025f60 0x5625ac4241c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85fb02482cbbd5c9 server_cookie=c484148ae4b42999 in_seq=0 out_seq=0 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe67cff9640 1 -- 172.21.15.78:0/3651529887 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5625ac375800 con 0x5625ac025f60 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe67cff9640 10 client.16530 handle_command_reply: tid=0 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 client.16530 shutdown 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 msgr2=0x5625ac4171a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac4171a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe6840dcd10 tx=0x7fe6840a3960 comp rx=0 tx=0).stop 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 shutdown_connections 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac067390 0x5625ac41e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac4171a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625ac025f60 0x5625ac4241c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 --2- 172.21.15.78:0/3651529887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac416c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 >> 172.21.15.78:0/3651529887 conn(0x5625ac05b4e0 msgr2=0x5625ac407180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe67cff9640 0 client.16530 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 shutdown_connections 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 1 -- 172.21.15.78:0/3651529887 wait complete. 2024-04-25T14:03:04.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.317+0000 7fe6a2794740 20 client.16530 trim_cache size 0 max 16384 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 Processor -- start 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 -- start start 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac4043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac410e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac4113d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941aba00 con 0x5625ac0466b0 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941a92d0 con 0x5625ac03e4d0 2024-04-25T14:03:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe6a2794740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941a7120 con 0x5625ac067390 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac410e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac410e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45460/0 (socket says 172.21.15.78:45460) 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 -- 172.21.15.78:0/1608480255 learned_addr learned my addr 172.21.15.78:0/1608480255 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe69967c640 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac4043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 -- 172.21.15.78:0/1608480255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 msgr2=0x5625ac4113d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac4113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 -- 172.21.15.78:0/1608480255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 msgr2=0x5625ac4043b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac4043b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 -- 172.21.15.78:0/1608480255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625ac03f0f0 con 0x5625ac0466b0 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.335+0000 7fe698e7b640 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac410e90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c6a0 tx=0x7fe68400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fdb77a703c7a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe67effd640 1 -- 172.21.15.78:0/1608480255 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840dce60 con 0x5625ac0466b0 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe67effd640 1 -- 172.21.15.78:0/1608480255 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6840b01e0 con 0x5625ac0466b0 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe67effd640 1 -- 172.21.15.78:0/1608480255 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840a2da0 con 0x5625ac0466b0 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 -- 172.21.15.78:0/1608480255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 msgr2=0x5625ac410e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac410e90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c6a0 tx=0x7fe68400cf20 comp rx=0 tx=0).stop 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 -- 172.21.15.78:0/1608480255 shutdown_connections 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac067390 0x5625ac4113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac0466b0 0x5625ac410e90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 --2- 172.21.15.78:0/1608480255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac03e4d0 0x5625ac4043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 -- 172.21.15.78:0/1608480255 >> 172.21.15.78:0/1608480255 conn(0x5625ac05b4e0 msgr2=0x5625ac403ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 -- 172.21.15.78:0/1608480255 shutdown_connections 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.336+0000 7fe6a2794740 1 -- 172.21.15.78:0/1608480255 wait complete. 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 -- start start 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 Processor -- start 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac41aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac0466b0 0x5625ac41b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac067390 0x5625ac41bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941a7790 con 0x5625ac067390 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941a8c00 con 0x5625ac0466b0 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941a7e60 con 0x5625ac03e4d0 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe6a2794740 10 client.0 _ll_register_callbacks cb 0x7fe684090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe699e7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac067390 0x5625ac41bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe699e7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac067390 0x5625ac41bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45462/0 (socket says 172.21.15.78:45462) 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe699e7d640 1 -- 172.21.15.78:0/2221329074 learned_addr learned my addr 172.21.15.78:0/2221329074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.337+0000 7fe698e7b640 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac0466b0 0x5625ac41b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe69967c640 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac41aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe699e7d640 1 -- 172.21.15.78:0/2221329074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 msgr2=0x5625ac41aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe699e7d640 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac41aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe699e7d640 1 -- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac0466b0 msgr2=0x5625ac41b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe699e7d640 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac0466b0 0x5625ac41b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe699e7d640 1 -- 172.21.15.78:0/2221329074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ac32e4b0 con 0x5625ac067390 2024-04-25T14:03:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe699e7d640 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac067390 0x5625ac41bf00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00fcb0 tx=0x7fe68c009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f4757250c4974e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe67cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe67cff9640 1 -- 172.21.15.78:0/2221329074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0046d0 con 0x5625ac067390 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe67cff9640 1 -- 172.21.15.78:0/2221329074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe68c022450 con 0x5625ac067390 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6840b01e0 con 0x5625ac067390 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe67cff9640 1 -- 172.21.15.78:0/2221329074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe68c022650 con 0x5625ac067390 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe6a2794740 10 client.16536 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5625ac41d650 con 0x5625ac067390 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.338+0000 7fe67cff9640 1 -- 172.21.15.78:0/2221329074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6840b01e0 con 0x5625ac067390 2024-04-25T14:03:04.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe67cff9640 1 -- 172.21.15.78:0/2221329074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe68c01f040 con 0x5625ac067390 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe6a2794740 10 client.16536 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe6a2794740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe6a2794740 20 client.16536 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe6a2794740 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625abffb3a0 0x5625ac48e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe6a2794740 4 client.16536 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625ac2f04a0 con 0x5625abffb3a0 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe69967c640 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625abffb3a0 0x5625ac48e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe69967c640 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625abffb3a0 0x5625ac48e870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd6e61f17e20a76 server_cookie=e8d7653d07153b9b in_seq=0 out_seq=0 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.339+0000 7fe67cff9640 10 client.16536 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe67cff9640 1 -- 172.21.15.78:0/2221329074 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fe690048e50 con 0x5625abffb3a0 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe67cff9640 10 client.16536 handle_command_reply: tid=0 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 client.16536 shutdown 2024-04-25T14:03:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac067390 msgr2=0x5625ac41bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac067390 0x5625ac41bf00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00fcb0 tx=0x7fe68c009e50 comp rx=0 tx=0).stop 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 shutdown_connections 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625ac067390 0x5625ac41bf00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625ac0466b0 0x5625ac41b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5625abffb3a0 0x5625ac48e870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 --2- 172.21.15.78:0/2221329074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625ac03e4d0 0x5625ac41aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 >> 172.21.15.78:0/2221329074 conn(0x5625ac05b4e0 msgr2=0x5625ac410e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe67cff9640 0 client.16536 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 shutdown_connections 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 1 -- 172.21.15.78:0/2221329074 wait complete. 2024-04-25T14:03:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe6a2794740 20 client.16536 trim_cache size 0 max 16384 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: "3e384998-5633-4752-85f6-b34b356d8419": { 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "3e384998-5633-4752-85f6-b34b356d8419", 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:03:04.349 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:03:04.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe6800afb70 msgr2=0x7fe6800b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe6800afb70 0x7fe6800b2080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5625ac409ab0 tx=0x7fe68c004ab0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe694103c30 msgr2=0x7fe69419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:04.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe694103c30 0x7fe69419f000 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe68400e9f0 tx=0x7fe68400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 shutdown_connections 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe694108930 0x7fe6941a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe6800afb70 0x7fe6800b2080 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe694103c30 0x7fe69419f000 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 --2- 172.21.15.78:0/1297086899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6941029f0 0x7fe694075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.340+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 >> 172.21.15.78:0/1297086899 conn(0x7fe6940fdbc0 msgr2=0x7fe694104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.341+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 shutdown_connections 2024-04-25T14:03:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:04.341+0000 7fe69b8e0640 1 -- 172.21.15.78:0/1297086899 wait complete. 2024-04-25T14:03:04.352 INFO:tasks.cephfs.filesystem:scrub status for tag:3e384998-5633-4752-85f6-b34b356d8419 - {'path': '/', 'tag': '3e384998-5633-4752-85f6-b34b356d8419', 'options': 'recursive,force'} 2024-04-25T14:03:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:04 smithi129 ceph-mon[32569]: pgmap v455: 97 pgs: 97 active+clean; 2.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:04.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:04 smithi078 ceph-mon[29163]: pgmap v455: 97 pgs: 97 active+clean; 2.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:04.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:04 smithi112 ceph-mon[32617]: pgmap v455: 97 pgs: 97 active+clean; 2.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:05.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:05 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:03:05.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:05 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:05.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:05 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:03:05.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:05 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:05 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:03:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:05 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:05 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:03:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:05 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:05.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:05 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:03:05.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:05 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:05.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:05 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:03:05.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:05 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:06 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:03:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:06 smithi129 ceph-mon[32569]: pgmap v456: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:06 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:03:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:06 smithi078 ceph-mon[29163]: pgmap v456: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:06 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:03:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:06 smithi112 ceph-mon[32617]: pgmap v456: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:08 smithi129 ceph-mon[32569]: pgmap v457: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:03:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:08 smithi078 ceph-mon[29163]: pgmap v457: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:03:08.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:08 smithi112 ceph-mon[32617]: pgmap v457: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:03:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:10 smithi129 ceph-mon[32569]: pgmap v458: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:10 smithi078 ceph-mon[29163]: pgmap v458: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:10.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:10 smithi112 ceph-mon[32617]: pgmap v458: 97 pgs: 97 active+clean; 2.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:12.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:12 smithi129 ceph-mon[32569]: pgmap v459: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:12 smithi078 ceph-mon[29163]: pgmap v459: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:12.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:12 smithi112 ceph-mon[32617]: pgmap v459: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 -- 172.21.15.78:0/3258567027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4103a00 msgr2=0x7f85c4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 --2- 172.21.15.78:0/3258567027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4103a00 0x7f85c4104700 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009900 tx=0x7f85b402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 -- 172.21.15.78:0/3258567027 shutdown_connections 2024-04-25T14:03:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 --2- 172.21.15.78:0/3258567027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85c4109940 0x7f85c410bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 --2- 172.21.15.78:0/3258567027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c4104c40 0x7f85c4105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 --2- 172.21.15.78:0/3258567027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4103a00 0x7f85c4104700 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 -- 172.21.15.78:0/3258567027 >> 172.21.15.78:0/3258567027 conn(0x7f85c40febd0 msgr2=0x7f85c41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 -- 172.21.15.78:0/3258567027 shutdown_connections 2024-04-25T14:03:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.256+0000 7f85c919f640 1 -- 172.21.15.78:0/3258567027 wait complete. 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 Processor -- start 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 -- start start 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85c4103a00 0x7f85c419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4104c40 0x7f85c419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c4109940 0x7f85c41a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410f4e0 con 0x7f85c4104c40 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410f360 con 0x7f85c4109940 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.257+0000 7f85c919f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410f660 con 0x7f85c4103a00 2024-04-25T14:03:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4104c40 0x7f85c419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4104c40 0x7f85c419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34078/0 (socket says 172.21.15.78:34078) 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 -- 172.21.15.78:0/517031137 learned_addr learned my addr 172.21.15.78:0/517031137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c3fff640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85c4103a00 0x7f85c419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c899e640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c4109940 0x7f85c41a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 -- 172.21.15.78:0/517031137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85c4103a00 msgr2=0x7f85c419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85c4103a00 0x7f85c419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 -- 172.21.15.78:0/517031137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c4109940 msgr2=0x7f85c41a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c4109940 0x7f85c41a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 -- 172.21.15.78:0/517031137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c41a1c60 con 0x7f85c4104c40 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c37fe640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4104c40 0x7f85c419a660 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f85b800d950 tx=0x7f85b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c17fa640 1 -- 172.21.15.78:0/517031137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8017070 con 0x7f85c4104c40 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c41a1f50 con 0x7f85c4104c40 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c17fa640 1 -- 172.21.15.78:0/517031137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b8009c30 con 0x7f85c4104c40 2024-04-25T14:03:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c17fa640 1 -- 172.21.15.78:0/517031137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b800f620 con 0x7f85c4104c40 2024-04-25T14:03:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.258+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c41a23b0 con 0x7f85c4104c40 2024-04-25T14:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 Processor -- start 2024-04-25T14:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 -- start start 2024-04-25T14:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85c17fa640 1 -- 172.21.15.78:0/517031137 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f85b800f7c0 con 0x7f85c4104c40 2024-04-25T14:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6ef24d0 0x55d7c72baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7219460 con 0x55d7c6eea2f0 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7c71d3590 con 0x55d7c6ef24d0 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85d138a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7215420 con 0x55d7c6f131b0 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85c3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85c3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34080/0 (socket says 172.21.15.78:34080) 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.261+0000 7f85c3fff640 1 -- 172.21.15.78:0/1666458868 learned_addr learned my addr 172.21.15.78:0/1666458868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c37fe640 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6ef24d0 0x55d7c72baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c899e640 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c3fff640 1 -- 172.21.15.78:0/1666458868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 msgr2=0x55d7c72b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c3fff640 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c3fff640 1 -- 172.21.15.78:0/1666458868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6ef24d0 msgr2=0x55d7c72baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c3fff640 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6ef24d0 0x55d7c72baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c3fff640 1 -- 172.21.15.78:0/1666458868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c71e00d0 con 0x55d7c6eea2f0 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c17fa640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f85ac0afae0 0x7f85ac0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c3fff640 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b4090 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009230 tx=0x7f85b4031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a14f310c2ac541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c899e640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f85ac0afae0 0x7f85ac0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85c17fa640 1 -- 172.21.15.78:0/517031137 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f85b80de7b0 con 0x7f85c4104c40 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85aa7fc640 1 -- 172.21.15.78:0/1666458868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4035030 con 0x55d7c6eea2f0 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85aa7fc640 1 -- 172.21.15.78:0/1666458868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b4040070 con 0x55d7c6eea2f0 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.262+0000 7f85aa7fc640 1 -- 172.21.15.78:0/1666458868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4003a80 con 0x55d7c6eea2f0 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 -- 172.21.15.78:0/1666458868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6eea2f0 msgr2=0x55d7c72b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85c899e640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f85ac0afae0 0x7f85ac0b1ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d7c72b4ce0 tx=0x7f85bc009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b4090 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009230 tx=0x7f85b4031bc0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 -- 172.21.15.78:0/1666458868 shutdown_connections 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6ef24d0 0x55d7c72baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b4090 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 --2- 172.21.15.78:0/1666458868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 -- 172.21.15.78:0/1666458868 >> 172.21.15.78:0/1666458868 conn(0x55d7c6f07300 msgr2=0x55d7c72af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 -- 172.21.15.78:0/1666458868 shutdown_connections 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.263+0000 7f85d138a740 1 -- 172.21.15.78:0/1666458868 wait complete. 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 -- start start 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 Processor -- start 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6ef24d0 0x55d7c72bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7c72bb980 con 0x55d7c6ef24d0 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7c71a0b60 con 0x55d7c6eea2f0 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7c72bbb00 con 0x55d7c6f131b0 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85d138a740 10 client.0 _ll_register_callbacks cb 0x55d7c71c52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6ef24d0 0x55d7c72bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6ef24d0 0x55d7c72bdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34088/0 (socket says 172.21.15.78:34088) 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85c37fe640 1 -- 172.21.15.78:0/2794812182 learned_addr learned my addr 172.21.15.78:0/2794812182 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85c3fff640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.264+0000 7f85c899e640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85c37fe640 1 -- 172.21.15.78:0/2794812182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 msgr2=0x55d7c72c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85c37fe640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85c37fe640 1 -- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 msgr2=0x55d7c72bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85c37fe640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85c37fe640 1 -- 172.21.15.78:0/2794812182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c6f2b210 con 0x55d7c6ef24d0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85c3fff640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85c37fe640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6ef24d0 0x55d7c72bdcc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f85b8000c00 tx=0x7f85b800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb7f3f4af03a567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85a37fe640 1 -- 172.21.15.78:0/2794812182 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8020070 con 0x55d7c6ef24d0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85a37fe640 1 -- 172.21.15.78:0/2794812182 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f85b80a3320 con 0x55d7c6ef24d0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b4040070 con 0x55d7c6ef24d0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85a37fe640 1 -- 172.21.15.78:0/2794812182 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f85b80a6840 con 0x55d7c6ef24d0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 10 client.16548 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d7c6ff7620 con 0x55d7c6ef24d0 2024-04-25T14:03:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85a37fe640 1 -- 172.21.15.78:0/2794812182 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b4040070 con 0x55d7c6ef24d0 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85a37fe640 1 -- 172.21.15.78:0/2794812182 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f85b80a26a0 con 0x55d7c6ef24d0 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 10 client.16548 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 20 client.16548 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ed1d80 0x55d7c72cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 4 client.16548 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.265+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c7227070 con 0x55d7c6ed1d80 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.266+0000 7f85c3fff640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ed1d80 0x55d7c72cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.266+0000 7f85a37fe640 10 client.16548 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.266+0000 7f85c3fff640 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ed1d80 0x55d7c72cb4c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2083b6ec4cae48de server_cookie=d916ada9db6fa04f in_seq=0 out_seq=0 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85a37fe640 1 -- 172.21.15.78:0/2794812182 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7c7227070 con 0x55d7c6ed1d80 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85a37fe640 10 client.16548 handle_command_reply: tid=0 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 client.16548 shutdown 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6ef24d0 msgr2=0x55d7c72bdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6ef24d0 0x55d7c72bdcc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f85b8000c00 tx=0x7f85b800bf60 comp rx=0 tx=0).stop 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 shutdown_connections 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ed1d80 0x55d7c72cb4c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6f131b0 0x55d7c72c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6ef24d0 0x55d7c72bdcc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85a37fe640 0 client.16548 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 --2- 172.21.15.78:0/2794812182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 >> 172.21.15.78:0/2794812182 conn(0x55d7c6f07300 msgr2=0x55d7c72b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 shutdown_connections 2024-04-25T14:03:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 1 -- 172.21.15.78:0/2794812182 wait complete. 2024-04-25T14:03:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.267+0000 7f85d138a740 20 client.16548 trim_cache size 0 max 16384 2024-04-25T14:03:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.282+0000 7f85d138a740 1 Processor -- start 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.282+0000 7f85d138a740 1 -- start start 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85d138a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85d138a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c72c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85d138a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c72d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85d138a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ad970 con 0x55d7c6f131b0 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85d138a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41ab240 con 0x55d7c6eea2f0 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85d138a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41a9090 con 0x55d7c6ef24d0 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c72d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c72d0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34092/0 (socket says 172.21.15.78:34092) 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 -- 172.21.15.78:0/3695995269 learned_addr learned my addr 172.21.15.78:0/3695995269 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c37fe640 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c72c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c3fff640 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 -- 172.21.15.78:0/3695995269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 msgr2=0x55d7c72c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c72c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 -- 172.21.15.78:0/3695995269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 msgr2=0x55d7c72b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 -- 172.21.15.78:0/3695995269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c72b8690 con 0x55d7c6f131b0 2024-04-25T14:03:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.283+0000 7f85c899e640 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c72d0cc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f85bc003c00 tx=0x7f85bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89303f34d84c7af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.285+0000 7f85a9ffb640 1 -- 172.21.15.78:0/3695995269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0043a0 con 0x55d7c6f131b0 2024-04-25T14:03:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.285+0000 7f85a9ffb640 1 -- 172.21.15.78:0/3695995269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85bc026030 con 0x55d7c6f131b0 2024-04-25T14:03:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.285+0000 7f85a9ffb640 1 -- 172.21.15.78:0/3695995269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc022470 con 0x55d7c6f131b0 2024-04-25T14:03:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 -- 172.21.15.78:0/3695995269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 msgr2=0x55d7c72d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c72d0cc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f85bc003c00 tx=0x7f85bc016000 comp rx=0 tx=0).stop 2024-04-25T14:03:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 -- 172.21.15.78:0/3695995269 shutdown_connections 2024-04-25T14:03:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c72d0cc0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c72c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 --2- 172.21.15.78:0/3695995269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c72b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 -- 172.21.15.78:0/3695995269 >> 172.21.15.78:0/3695995269 conn(0x55d7c6f07300 msgr2=0x55d7c72b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 -- 172.21.15.78:0/3695995269 shutdown_connections 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 -- 172.21.15.78:0/3695995269 wait complete. 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 -- start start 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.286+0000 7f85d138a740 1 Processor -- start 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85d138a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c732d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85d138a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c732fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85d138a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c73322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85d138a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41a9700 con 0x55d7c6f131b0 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85d138a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41aab70 con 0x55d7c6eea2f0 2024-04-25T14:03:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85d138a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41a9dd0 con 0x55d7c6ef24d0 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85d138a740 10 client.0 _ll_register_callbacks cb 0x7f85b800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c73322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c732fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c73322e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34096/0 (socket says 172.21.15.78:34096) 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 -- 172.21.15.78:0/4288094577 learned_addr learned my addr 172.21.15.78:0/4288094577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c3fff640 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c732d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 -- 172.21.15.78:0/4288094577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 msgr2=0x55d7c732fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c732fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 -- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 msgr2=0x55d7c732d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c732d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 -- 172.21.15.78:0/4288094577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c7332820 con 0x55d7c6f131b0 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85c899e640 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c73322e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0098b0 tx=0x7f85bc011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a51970ebbe36885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.287+0000 7f85a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85a37fe640 1 -- 172.21.15.78:0/4288094577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc03a060 con 0x55d7c6f131b0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85a37fe640 1 -- 172.21.15.78:0/4288094577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f85bc0043a0 con 0x55d7c6f131b0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85bc026030 con 0x55d7c6f131b0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85a37fe640 1 -- 172.21.15.78:0/4288094577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f85bc0045a0 con 0x55d7c6f131b0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 10 client.16554 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d7c7333ab0 con 0x55d7c6f131b0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85a37fe640 1 -- 172.21.15.78:0/4288094577 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85bc026030 con 0x55d7c6f131b0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85a37fe640 1 -- 172.21.15.78:0/4288094577 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f85bc034070 con 0x55d7c6f131b0 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 10 client.16554 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 20 client.16554 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ea71c0 0x55d7c73364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 4 client.16554 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7c7220ee0 con 0x55d7c6ea71c0 2024-04-25T14:03:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.288+0000 7f85c3fff640 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ea71c0 0x55d7c73364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.289+0000 7f85c3fff640 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ea71c0 0x55d7c73364a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a77e224df78b51 server_cookie=21d3aa0c1733f177 in_seq=0 out_seq=0 2024-04-25T14:03:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.289+0000 7f85a37fe640 10 client.16554 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.303+0000 7f85a37fe640 1 -- 172.21.15.78:0/4288094577 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7c7220ee0 con 0x55d7c6ea71c0 2024-04-25T14:03:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.303+0000 7f85a37fe640 10 client.16554 handle_command_reply: tid=0 2024-04-25T14:03:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 client.16554 shutdown 2024-04-25T14:03:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 msgr2=0x55d7c73322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c73322e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0098b0 tx=0x7f85bc011ae0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 shutdown_connections 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7c6f131b0 0x55d7c73322e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d7c6ea71c0 0x55d7c73364a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7c6ef24d0 0x55d7c732fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 --2- 172.21.15.78:0/4288094577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7c6eea2f0 0x55d7c732d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85a37fe640 0 client.16554 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 >> 172.21.15.78:0/4288094577 conn(0x55d7c6f07300 msgr2=0x55d7c72cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 shutdown_connections 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 1 -- 172.21.15.78:0/4288094577 wait complete. 2024-04-25T14:03:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85d138a740 20 client.16554 trim_cache size 0 max 16384 2024-04-25T14:03:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:03:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:03:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:03:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:03:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f85ac0afae0 msgr2=0x7f85ac0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85c919f640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f85ac0afae0 0x7f85ac0b1ff0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d7c72b4ce0 tx=0x7f85bc009600 comp rx=0 tx=0).stop 2024-04-25T14:03:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.304+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4104c40 msgr2=0x7f85c419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4104c40 0x7f85c419a660 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f85b800d950 tx=0x7f85b800de10 comp rx=0 tx=0).stop 2024-04-25T14:03:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 shutdown_connections 2024-04-25T14:03:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c4109940 0x7f85c41a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f85ac0afae0 0x7f85ac0b1ff0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85c4104c40 0x7f85c419a660 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 --2- 172.21.15.78:0/517031137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85c4103a00 0x7f85c419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 >> 172.21.15.78:0/517031137 conn(0x7f85c40febd0 msgr2=0x7f85c4105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 shutdown_connections 2024-04-25T14:03:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:14.305+0000 7f85c919f640 1 -- 172.21.15.78:0/517031137 wait complete. 2024-04-25T14:03:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:14 smithi129 ceph-mon[32569]: pgmap v460: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:14 smithi078 ceph-mon[29163]: pgmap v460: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:14 smithi112 ceph-mon[32617]: pgmap v460: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:16.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:16 smithi129 ceph-mon[32569]: pgmap v461: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:03:16.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:16 smithi078 ceph-mon[29163]: pgmap v461: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:03:16.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:16 smithi112 ceph-mon[32617]: pgmap v461: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:03:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:18 smithi129 ceph-mon[32569]: pgmap v462: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:18.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:18 smithi078 ceph-mon[29163]: pgmap v462: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:18.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:18 smithi112 ceph-mon[32617]: pgmap v462: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:20 smithi129 ceph-mon[32569]: pgmap v463: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:03:20.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:20 smithi078 ceph-mon[29163]: pgmap v463: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:03:20.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:20 smithi112 ceph-mon[32617]: pgmap v463: 97 pgs: 97 active+clean; 2.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:03:22.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:22 smithi112 ceph-mon[32617]: pgmap v464: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:22 smithi129 ceph-mon[32569]: pgmap v464: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:22 smithi078 ceph-mon[29163]: pgmap v464: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:24.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:24 smithi129 ceph-mon[32569]: pgmap v465: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:24 smithi078 ceph-mon[29163]: pgmap v465: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:24.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:24 smithi112 ceph-mon[32617]: pgmap v465: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:26.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:26 smithi129 ceph-mon[32569]: pgmap v466: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:26 smithi078 ceph-mon[29163]: pgmap v466: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:26.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:26 smithi112 ceph-mon[32617]: pgmap v466: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 -- 172.21.15.78:0/3060992386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac103c60 msgr2=0x7f50ac104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/3060992386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac103c60 0x7f50ac104070 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 -- 172.21.15.78:0/3060992386 shutdown_connections 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/3060992386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50ac108960 0x7f50ac10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/3060992386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac103c60 0x7f50ac104070 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/3060992386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50ac102a20 0x7f50ac103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 -- 172.21.15.78:0/3060992386 >> 172.21.15.78:0/3060992386 conn(0x7f50ac0fdc10 msgr2=0x7f50ac1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 -- 172.21.15.78:0/3060992386 shutdown_connections 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 -- 172.21.15.78:0/3060992386 wait complete. 2024-04-25T14:03:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 Processor -- start 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.409+0000 7f50b2bc7640 1 -- start start 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b2bc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac102a20 0x7f50ac19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b2bc7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50ac103c60 0x7f50ac19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b2bc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50ac108960 0x7f50ac1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b2bc7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac10f640 con 0x7f50ac102a20 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b2bc7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac10f4c0 con 0x7f50ac108960 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b2bc7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac10f7c0 con 0x7f50ac103c60 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac102a20 0x7f50ac19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac102a20 0x7f50ac19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43950/0 (socket says 172.21.15.78:43950) 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 -- 172.21.15.78:0/2228341191 learned_addr learned my addr 172.21.15.78:0/2228341191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50abfff640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50ac103c60 0x7f50ac19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b1164640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50ac108960 0x7f50ac1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 -- 172.21.15.78:0/2228341191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50ac103c60 msgr2=0x7f50ac19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50ac103c60 0x7f50ac19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 -- 172.21.15.78:0/2228341191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50ac108960 msgr2=0x7f50ac1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50ac108960 0x7f50ac1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 -- 172.21.15.78:0/2228341191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac1a1ce0 con 0x7f50ac102a20 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b0963640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac102a20 0x7f50ac19a250 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f509c005d00 tx=0x7f509c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50a9ffb640 1 -- 172.21.15.78:0/2228341191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c017070 con 0x7f50ac102a20 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50a9ffb640 1 -- 172.21.15.78:0/2228341191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c009c30 con 0x7f50ac102a20 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac1a1fd0 con 0x7f50ac102a20 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.410+0000 7f50a9ffb640 1 -- 172.21.15.78:0/2228341191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c00f620 con 0x7f50ac102a20 2024-04-25T14:03:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.411+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1a23e0 con 0x7f50ac102a20 2024-04-25T14:03:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 Processor -- start 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50a9ffb640 1 -- 172.21.15.78:0/2228341191 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f509c01a070 con 0x7f50ac102a20 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 -- start start 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155067970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154c9e2f0 0x557155067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154ca64d0 0x55715506edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557154fcd460 con 0x557154cc71b0 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557154f87590 con 0x557154c9e2f0 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.413+0000 7f50b9a74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557154fc9420 con 0x557154ca64d0 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155067970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155067970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43954/0 (socket says 172.21.15.78:43954) 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 -- 172.21.15.78:0/3612956603 learned_addr learned my addr 172.21.15.78:0/3612956603 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b0963640 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154c9e2f0 0x557155067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50abfff640 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154ca64d0 0x55715506edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50a9ffb640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f50940afae0 0x7f50940b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 -- 172.21.15.78:0/3612956603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154ca64d0 msgr2=0x55715506edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154ca64d0 0x55715506edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 -- 172.21.15.78:0/3612956603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154c9e2f0 msgr2=0x557155067ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154c9e2f0 0x557155067ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 -- 172.21.15.78:0/3612956603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557154f940d0 con 0x557154cc71b0 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b0963640 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154c9e2f0 0x557155067ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f50940afae0 0x7f50940b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50a9ffb640 1 -- 172.21.15.78:0/2228341191 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f509c0de690 con 0x7f50ac102a20 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f50b1164640 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155067970 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f50a4007fd0 tx=0x7f50a400e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c6f92f650c1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.414+0000 7f5092ffd640 1 -- 172.21.15.78:0/3612956603 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4022070 con 0x557154cc71b0 2024-04-25T14:03:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f5092ffd640 1 -- 172.21.15.78:0/3612956603 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a4011c30 con 0x557154cc71b0 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f5092ffd640 1 -- 172.21.15.78:0/3612956603 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a401a400 con 0x557154cc71b0 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 -- 172.21.15.78:0/3612956603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 msgr2=0x557155067970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b1164640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f50940afae0 0x7f50940b1ff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f50a401f000 tx=0x7f50a4020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155067970 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f50a4007fd0 tx=0x7f50a400e7e0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 -- 172.21.15.78:0/3612956603 shutdown_connections 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154ca64d0 0x55715506edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154c9e2f0 0x557155067ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3612956603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155067970 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 -- 172.21.15.78:0/3612956603 >> 172.21.15.78:0/3612956603 conn(0x557154cbb300 msgr2=0x557155063300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 -- 172.21.15.78:0/3612956603 shutdown_connections 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.415+0000 7f50b9a74740 1 -- 172.21.15.78:0/3612956603 wait complete. 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 -- start start 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 Processor -- start 2024-04-25T14:03:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155071620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155078ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55715506f840 con 0x557154cc71b0 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557154f54b60 con 0x557154ca64d0 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55715506f9c0 con 0x557154c9e2f0 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b9a74740 10 client.0 _ll_register_callbacks cb 0x557154f792b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b1164640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155078ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b1164640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155078ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43968/0 (socket says 172.21.15.78:43968) 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.416+0000 7f50b1164640 1 -- 172.21.15.78:0/3811416703 learned_addr learned my addr 172.21.15.78:0/3811416703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50abfff640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b0963640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b1164640 1 -- 172.21.15.78:0/3811416703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 msgr2=0x557155071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b1164640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b1164640 1 -- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 msgr2=0x557155071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b1164640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b1164640 1 -- 172.21.15.78:0/3811416703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557154cdf210 con 0x557154cc71b0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b0963640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50abfff640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b1164640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155078ac0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f50a4012490 tx=0x7f50a401ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff7c5b8c7d0ab2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f508bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f508bfff640 1 -- 172.21.15.78:0/3811416703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4034060 con 0x557154cc71b0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f508bfff640 1 -- 172.21.15.78:0/3811416703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f50a401cc90 con 0x557154cc71b0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a4011c30 con 0x557154cc71b0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f508bfff640 1 -- 172.21.15.78:0/3811416703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f50a4022040 con 0x557154cc71b0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b9a74740 10 client.16566 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557154dab620 con 0x557154cc71b0 2024-04-25T14:03:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.417+0000 7f508bfff640 1 -- 172.21.15.78:0/3811416703 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a4011c30 con 0x557154cc71b0 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f508bfff640 1 -- 172.21.15.78:0/3811416703 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f50a4019040 con 0x557154cc71b0 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f50b9a74740 10 client.16566 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f50b9a74740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f50b9a74740 20 client.16566 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c85d80 0x55715507f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f50b9a74740 4 client.16566 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557154fdb070 con 0x557154c85d80 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.418+0000 7f50b0963640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c85d80 0x55715507f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b0963640 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c85d80 0x55715507f0e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b15b07aa593f39e6 server_cookie=2b862ad8bcb5dc06 in_seq=0 out_seq=0 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f508bfff640 10 client.16566 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f508bfff640 1 -- 172.21.15.78:0/3811416703 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557154fdb070 con 0x557154c85d80 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f508bfff640 10 client.16566 handle_command_reply: tid=0 2024-04-25T14:03:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 client.16566 shutdown 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 msgr2=0x557155078ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155078ac0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f50a4012490 tx=0x7f50a401ef90 comp rx=0 tx=0).stop 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 shutdown_connections 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154cc71b0 0x557155078ac0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c85d80 0x55715507f0e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 --2- 172.21.15.78:0/3811416703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f508bfff640 0 client.16566 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.419+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 >> 172.21.15.78:0/3811416703 conn(0x557154cbb300 msgr2=0x557155062250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.420+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 shutdown_connections 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.420+0000 7f50b9a74740 1 -- 172.21.15.78:0/3811416703 wait complete. 2024-04-25T14:03:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.420+0000 7f50b9a74740 20 client.16566 trim_cache size 0 max 16384 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 Processor -- start 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 -- start start 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155062be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154ca64d0 0x5571550789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154cc71b0 0x557155063120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1adb30 con 0x557154ca64d0 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1ab400 con 0x557154cc71b0 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b9a74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1a9250 con 0x557154c9e2f0 2024-04-25T14:03:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154ca64d0 0x5571550789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154ca64d0 0x5571550789e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43982/0 (socket says 172.21.15.78:43982) 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 -- 172.21.15.78:0/1337947394 learned_addr learned my addr 172.21.15.78:0/1337947394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b0963640 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155062be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b1164640 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154cc71b0 0x557155063120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 -- 172.21.15.78:0/1337947394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 msgr2=0x557155062be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155062be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 -- 172.21.15.78:0/1337947394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154cc71b0 msgr2=0x557155063120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154cc71b0 0x557155063120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 -- 172.21.15.78:0/1337947394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557155078f20 con 0x557154ca64d0 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50b1164640 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154cc71b0 0x557155063120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50abfff640 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154ca64d0 0x5571550789e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a0031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8682c20a4b198a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.435+0000 7f50927fc640 1 -- 172.21.15.78:0/1337947394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0035070 con 0x557154ca64d0 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50927fc640 1 -- 172.21.15.78:0/1337947394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0004030 con 0x557154ca64d0 2024-04-25T14:03:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50927fc640 1 -- 172.21.15.78:0/1337947394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0003b20 con 0x557154ca64d0 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 -- 172.21.15.78:0/1337947394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154ca64d0 msgr2=0x5571550789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154ca64d0 0x5571550789e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009900 tx=0x7f50a0031de0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 -- 172.21.15.78:0/1337947394 shutdown_connections 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154cc71b0 0x557155063120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154ca64d0 0x5571550789e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 --2- 172.21.15.78:0/1337947394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154c9e2f0 0x557155062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 -- 172.21.15.78:0/1337947394 >> 172.21.15.78:0/1337947394 conn(0x557154cbb300 msgr2=0x55715506baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 -- 172.21.15.78:0/1337947394 shutdown_connections 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.436+0000 7f50b9a74740 1 -- 172.21.15.78:0/1337947394 wait complete. 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 -- start start 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 Processor -- start 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154c9e2f0 0x5571550759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155075ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154cc71b0 0x5571550e5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1a98c0 con 0x557154c9e2f0 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1aad30 con 0x557154ca64d0 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1a9f90 con 0x557154cc71b0 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b9a74740 10 client.0 _ll_register_callbacks cb 0x7f50a4020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b0963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154c9e2f0 0x5571550759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b0963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154c9e2f0 0x5571550759a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43986/0 (socket says 172.21.15.78:43986) 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b0963640 1 -- 172.21.15.78:0/2983488800 learned_addr learned my addr 172.21.15.78:0/2983488800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50abfff640 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155075ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b1164640 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154cc71b0 0x5571550e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b0963640 1 -- 172.21.15.78:0/2983488800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154cc71b0 msgr2=0x5571550e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b0963640 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154cc71b0 0x5571550e5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b0963640 1 -- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 msgr2=0x557155075ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.437+0000 7f50b0963640 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155075ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b0963640 1 -- 172.21.15.78:0/2983488800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571550e59c0 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b0963640 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154c9e2f0 0x5571550759a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f509c0a6860 tx=0x7f509c010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e950267574f7281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f508bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f508bfff640 1 -- 172.21.15.78:0/2983488800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c021040 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f508bfff640 1 -- 172.21.15.78:0/2983488800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f509c0deb40 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a0004030 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f508bfff640 1 -- 172.21.15.78:0/2983488800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f509c0a6670 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 10 client.16572 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5571550e6bb0 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f508bfff640 1 -- 172.21.15.78:0/2983488800 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0004030 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f508bfff640 1 -- 172.21.15.78:0/2983488800 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f509c0a2040 con 0x557154c9e2f0 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 10 client.16572 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 20 client.16572 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c5b1c0 0x5571550e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 4 client.16572 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.438+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557154fd4ee0 con 0x557154c5b1c0 2024-04-25T14:03:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.439+0000 7f50abfff640 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c5b1c0 0x5571550e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.439+0000 7f508bfff640 10 client.16572 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.439+0000 7f50abfff640 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c5b1c0 0x5571550e9530 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c81258d338e5d6c server_cookie=5be2091539fe0590 in_seq=0 out_seq=0 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.453+0000 7f508bfff640 1 -- 172.21.15.78:0/2983488800 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557154fd4ee0 con 0x557154c5b1c0 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.453+0000 7f508bfff640 10 client.16572 handle_command_reply: tid=0 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.453+0000 7f50b9a74740 1 client.16572 shutdown 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154c9e2f0 msgr2=0x5571550759a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154c9e2f0 0x5571550759a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f509c0a6860 tx=0x7f509c010c00 comp rx=0 tx=0).stop 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 shutdown_connections 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557154c5b1c0 0x5571550e9530 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557154cc71b0 0x5571550e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557154ca64d0 0x557155075ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 --2- 172.21.15.78:0/2983488800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557154c9e2f0 0x5571550759a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f508bfff640 0 client.16572 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 >> 172.21.15.78:0/2983488800 conn(0x557154cbb300 msgr2=0x557155062be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 shutdown_connections 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 1 -- 172.21.15.78:0/2983488800 wait complete. 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b9a74740 20 client.16572 trim_cache size 0 max 16384 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:03:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:03:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f50940afae0 msgr2=0x7f50940b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f50940afae0 0x7f50940b1ff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f50a401f000 tx=0x7f50a4020000 comp rx=0 tx=0).stop 2024-04-25T14:03:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac102a20 msgr2=0x7f50ac19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.454+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac102a20 0x7f50ac19a250 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f509c005d00 tx=0x7f509c00dde0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 shutdown_connections 2024-04-25T14:03:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50ac108960 0x7f50ac1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50ac103c60 0x7f50ac19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f50940afae0 0x7f50940b1ff0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 --2- 172.21.15.78:0/2228341191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50ac102a20 0x7f50ac19a250 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 >> 172.21.15.78:0/2228341191 conn(0x7f50ac0fdc10 msgr2=0x7f50ac104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 shutdown_connections 2024-04-25T14:03:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:27.455+0000 7f50b2bc7640 1 -- 172.21.15.78:0/2228341191 wait complete. 2024-04-25T14:03:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:28 smithi129 ceph-mon[32569]: pgmap v467: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:28.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:28 smithi078 ceph-mon[29163]: pgmap v467: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:28.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:28 smithi112 ceph-mon[32617]: pgmap v467: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:30 smithi129 ceph-mon[32569]: pgmap v468: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:03:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:30 smithi078 ceph-mon[29163]: pgmap v468: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:03:30.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:30 smithi112 ceph-mon[32617]: pgmap v468: 97 pgs: 97 active+clean; 2.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:03:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:32 smithi129 ceph-mon[32569]: pgmap v469: 97 pgs: 97 active+clean; 2.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:32 smithi078 ceph-mon[29163]: pgmap v469: 97 pgs: 97 active+clean; 2.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:32.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:32 smithi112 ceph-mon[32617]: pgmap v469: 97 pgs: 97 active+clean; 2.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:03:34.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:34.354 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.481+0000 7f29e891f640 1 -- 172.21.15.78:0/4113398671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4102260 msgr2=0x7f29e410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.481+0000 7f29e891f640 1 --2- 172.21.15.78:0/4113398671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4102260 0x7f29e410bcc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009ca0 tx=0x7f29dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 -- 172.21.15.78:0/4113398671 shutdown_connections 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 --2- 172.21.15.78:0/4113398671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4102260 0x7f29e410bcc0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 --2- 172.21.15.78:0/4113398671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4101760 0x7f29e4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 --2- 172.21.15.78:0/4113398671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e41050b0 0x7f29e41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 -- 172.21.15.78:0/4113398671 >> 172.21.15.78:0/4113398671 conn(0x7f29e40fd380 msgr2=0x7f29e40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 -- 172.21.15.78:0/4113398671 shutdown_connections 2024-04-25T14:03:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 -- 172.21.15.78:0/4113398671 wait complete. 2024-04-25T14:03:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 Processor -- start 2024-04-25T14:03:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.482+0000 7f29e891f640 1 -- start start 2024-04-25T14:03:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e891f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4101760 0x7f29e419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e891f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4102260 0x7f29e419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e891f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e41050b0 0x7f29e41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e891f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410f620 con 0x7f29e4101760 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e891f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410f4a0 con 0x7f29e41050b0 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e891f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410f7a0 con 0x7f29e4102260 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4101760 0x7f29e419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4101760 0x7f29e419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43992/0 (socket says 172.21.15.78:43992) 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 -- 172.21.15.78:0/3704960421 learned_addr learned my addr 172.21.15.78:0/3704960421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e41050b0 0x7f29e41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4102260 0x7f29e419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 -- 172.21.15.78:0/3704960421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4102260 msgr2=0x7f29e419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4102260 0x7f29e419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 -- 172.21.15.78:0/3704960421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e41050b0 msgr2=0x7f29e41a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e41050b0 0x7f29e41a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 -- 172.21.15.78:0/3704960421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e41a1c60 con 0x7f29e4101760 2024-04-25T14:03:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.483+0000 7f29e259c640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4101760 0x7f29e419a1d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c970 tx=0x7f29d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.484+0000 7f29d37fe640 1 -- 172.21.15.78:0/3704960421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4007c70 con 0x7f29e4101760 2024-04-25T14:03:34.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.484+0000 7f29d37fe640 1 -- 172.21.15.78:0/3704960421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4007e10 con 0x7f29e4101760 2024-04-25T14:03:34.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.484+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e41a1f50 con 0x7f29e4101760 2024-04-25T14:03:34.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.484+0000 7f29d37fe640 1 -- 172.21.15.78:0/3704960421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4018760 con 0x7f29e4101760 2024-04-25T14:03:34.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.484+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e41a23b0 con 0x7f29e4101760 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.486+0000 7f29ef7cf740 1 Processor -- start 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.486+0000 7f29d37fe640 1 -- 172.21.15.78:0/3704960421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f29d400fab0 con 0x7f29e4101760 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29ef7cf740 1 -- start start 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86d3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559bb8637d80 con 0x559bb83106b0 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bb85f18b0 con 0x559bb8331390 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559bb8633d40 con 0x559bb83084d0 2024-04-25T14:03:34.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86d3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86d3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44006/0 (socket says 172.21.15.78:44006) 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 -- 172.21.15.78:0/724682041 learned_addr learned my addr 172.21.15.78:0/724682041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e259c640 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29d37fe640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29cc0afaf0 0x7f29cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 -- 172.21.15.78:0/724682041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 msgr2=0x559bb86d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 -- 172.21.15.78:0/724682041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 msgr2=0x559bb86d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e1d9b640 1 -- 172.21.15.78:0/724682041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb85c2a40 con 0x559bb83106b0 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29cc0afaf0 0x7f29cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.487+0000 7f29d37fe640 1 -- 172.21.15.78:0/3704960421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f29d40dec80 con 0x7f29e4101760 2024-04-25T14:03:34.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.488+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86d3c70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f29d800bce0 tx=0x7f29d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b54e1e57623732e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.488+0000 7f29d0ff9640 1 -- 172.21.15.78:0/724682041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8015070 con 0x559bb83106b0 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.488+0000 7f29d0ff9640 1 -- 172.21.15.78:0/724682041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d801a910 con 0x559bb83106b0 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.488+0000 7f29d0ff9640 1 -- 172.21.15.78:0/724682041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8011a30 con 0x559bb83106b0 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 -- 172.21.15.78:0/724682041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 msgr2=0x559bb86d3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29cc0afaf0 0x7f29cc0b2000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f29dc02f500 tx=0x7f29dc03f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86d3c70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f29d800bce0 tx=0x7f29d800f730 comp rx=0 tx=0).stop 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 -- 172.21.15.78:0/724682041 shutdown_connections 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86d3c70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/724682041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 -- 172.21.15.78:0/724682041 >> 172.21.15.78:0/724682041 conn(0x559bb83254e0 msgr2=0x559bb86ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 -- 172.21.15.78:0/724682041 shutdown_connections 2024-04-25T14:03:34.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 -- 172.21.15.78:0/724682041 wait complete. 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 -- start start 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.489+0000 7f29ef7cf740 1 Processor -- start 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86dc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559bb86d4780 con 0x559bb83106b0 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bb85bed10 con 0x559bb8331390 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559bb86d4900 con 0x559bb83084d0 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 10 client.0 _ll_register_callbacks cb 0x559bb85eb210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86dcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44016/0 (socket says 172.21.15.78:44016) 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 -- 172.21.15.78:0/832108337 learned_addr learned my addr 172.21.15.78:0/832108337 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e259c640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 -- 172.21.15.78:0/832108337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 msgr2=0x559bb86dc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86dc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 -- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 msgr2=0x559bb86e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 -- 172.21.15.78:0/832108337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb84137d0 con 0x559bb83106b0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e259c640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86dcea0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004630 tx=0x7f29d800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a05e187082667a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29c9ffb640 1 -- 172.21.15.78:0/832108337 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8028060 con 0x559bb83106b0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29c9ffb640 1 -- 172.21.15.78:0/832108337 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f29d801acc0 con 0x559bb83106b0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d801a910 con 0x559bb83106b0 2024-04-25T14:03:34.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29c9ffb640 1 -- 172.21.15.78:0/832108337 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f29d8015040 con 0x559bb83106b0 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.490+0000 7f29ef7cf740 10 client.16584 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559bb83493f0 con 0x559bb83106b0 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29c9ffb640 1 -- 172.21.15.78:0/832108337 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d801a910 con 0x559bb83106b0 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29c9ffb640 1 -- 172.21.15.78:0/832108337 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f29d8022030 con 0x559bb83106b0 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29ef7cf740 10 client.16584 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29ef7cf740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29ef7cf740 20 client.16584 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82eff60 0x559bb86e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29ef7cf740 4 client.16584 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bb8645990 con 0x559bb82eff60 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.491+0000 7f29e259c640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82eff60 0x559bb86e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.492+0000 7f29c9ffb640 10 client.16584 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.492+0000 7f29e259c640 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82eff60 0x559bb86e9f60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ece8ce7fa04d93 server_cookie=55ef5ef754cb78ca in_seq=0 out_seq=0 2024-04-25T14:03:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.492+0000 7f29c9ffb640 1 -- 172.21.15.78:0/832108337 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559bb8645990 con 0x559bb82eff60 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.492+0000 7f29c9ffb640 10 client.16584 handle_command_reply: tid=0 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 client.16584 shutdown 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 msgr2=0x559bb86dcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86dcea0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004630 tx=0x7f29d800ba50 comp rx=0 tx=0).stop 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 shutdown_connections 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb8331390 0x559bb86e3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86dcea0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82eff60 0x559bb86e9f60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/832108337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb83084d0 0x559bb86dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29c9ffb640 0 client.16584 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 >> 172.21.15.78:0/832108337 conn(0x559bb83254e0 msgr2=0x559bb86d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 shutdown_connections 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 1 -- 172.21.15.78:0/832108337 wait complete. 2024-04-25T14:03:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.493+0000 7f29ef7cf740 20 client.16584 trim_cache size 0 max 16384 2024-04-25T14:03:34.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 Processor -- start 2024-04-25T14:03:34.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 -- start start 2024-04-25T14:03:34.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 0x559bb86ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41adb10 con 0x559bb83106b0 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41ab3e0 con 0x559bb83084d0 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41a9230 con 0x559bb8331390 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.510+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44028/0 (socket says 172.21.15.78:44028) 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 -- 172.21.15.78:0/1173441667 learned_addr learned my addr 172.21.15.78:0/1173441667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 -- 172.21.15.78:0/1173441667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 msgr2=0x559bb86ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/1173441667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 0x559bb86ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 -- 172.21.15.78:0/1173441667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 msgr2=0x559bb86d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/1173441667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 -- 172.21.15.78:0/1173441667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb83090f0 con 0x559bb83106b0 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/1173441667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e3cb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004600 tx=0x7f29d8004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc41535058871510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29cbfff640 1 -- 172.21.15.78:0/1173441667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8028070 con 0x559bb83106b0 2024-04-25T14:03:34.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29cbfff640 1 -- 172.21.15.78:0/1173441667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d801a7f0 con 0x559bb83106b0 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.511+0000 7f29cbfff640 1 -- 172.21.15.78:0/1173441667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8037680 con 0x559bb83106b0 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 -- 172.21.15.78:0/1173441667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 msgr2=0x559bb86e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/1173441667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e3cb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004600 tx=0x7f29d8004810 comp rx=0 tx=0).stop 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 -- 172.21.15.78:0/1173441667 shutdown_connections 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/1173441667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 0x559bb86ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/1173441667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e3cb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/1173441667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 -- 172.21.15.78:0/1173441667 >> 172.21.15.78:0/1173441667 conn(0x559bb83254e0 msgr2=0x559bb86cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 -- 172.21.15.78:0/1173441667 shutdown_connections 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 -- 172.21.15.78:0/1173441667 wait complete. 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 -- start start 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 Processor -- start 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.512+0000 7f29ef7cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 0x559bb8750220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41a98a0 con 0x559bb83106b0 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41aad10 con 0x559bb83084d0 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29ef7cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41a9f70 con 0x559bb8331390 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29ef7cf740 10 client.0 _ll_register_callbacks cb 0x7f29d40a37c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44036/0 (socket says 172.21.15.78:44036) 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 -- 172.21.15.78:0/198825376 learned_addr learned my addr 172.21.15.78:0/198825376 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e259c640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e2d9d640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 0x559bb8750220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 -- 172.21.15.78:0/198825376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 msgr2=0x559bb8750220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 0x559bb8750220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 -- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 msgr2=0x559bb86e0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86e0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 -- 172.21.15.78:0/198825376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb85f84b0 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e259c640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86e0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29e1d9b640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e11e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004630 tx=0x7f29d8011730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f326e11b2e81d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29c9ffb640 1 -- 172.21.15.78:0/198825376 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8028070 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29c9ffb640 1 -- 172.21.15.78:0/198825376 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f29d8040670 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d801a7f0 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29c9ffb640 1 -- 172.21.15.78:0/198825376 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f29d800b3a0 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29ef7cf740 10 client.16590 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.513+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559bb8751690 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29c9ffb640 1 -- 172.21.15.78:0/198825376 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d801a7f0 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29c9ffb640 1 -- 172.21.15.78:0/198825376 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f29d8035020 con 0x559bb83106b0 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29ef7cf740 10 client.16590 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29ef7cf740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29ef7cf740 20 client.16590 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:34.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82c53a0 0x559bb8754100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29ef7cf740 4 client.16590 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559bb863f800 con 0x559bb82c53a0 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.514+0000 7f29e259c640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82c53a0 0x559bb8754100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29c9ffb640 10 client.16590 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29e259c640 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82c53a0 0x559bb8754100 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b801b6da91137498 server_cookie=e1007af74913a954 in_seq=0 out_seq=0 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29c9ffb640 1 -- 172.21.15.78:0/198825376 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559bb863f800 con 0x559bb82c53a0 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29c9ffb640 10 client.16590 handle_command_reply: tid=0 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 client.16590 shutdown 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 msgr2=0x559bb86e11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e11e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f29d8004630 tx=0x7f29d8011730 comp rx=0 tx=0).stop 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 shutdown_connections 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559bb82c53a0 0x559bb8754100 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559bb8331390 0x559bb8750220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29c9ffb640 0 client.16590 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:34.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb83106b0 0x559bb86e11e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 --2- 172.21.15.78:0/198825376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bb83084d0 0x559bb86e0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 >> 172.21.15.78:0/198825376 conn(0x559bb83254e0 msgr2=0x559bb86cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 shutdown_connections 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 1 -- 172.21.15.78:0/198825376 wait complete. 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.515+0000 7f29ef7cf740 20 client.16590 trim_cache size 0 max 16384 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:03:34.521 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:03:34.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29cc0afaf0 msgr2=0x7f29cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29cc0afaf0 0x7f29cc0b2000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f29dc02f500 tx=0x7f29dc03f040 comp rx=0 tx=0).stop 2024-04-25T14:03:34.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4101760 msgr2=0x7f29e419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4101760 0x7f29e419a1d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c970 tx=0x7f29d400ce30 comp rx=0 tx=0).stop 2024-04-25T14:03:34.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 shutdown_connections 2024-04-25T14:03:34.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29e41050b0 0x7f29e41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4102260 0x7f29e419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f29cc0afaf0 0x7f29cc0b2000 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 --2- 172.21.15.78:0/3704960421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29e4101760 0x7f29e419a1d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 >> 172.21.15.78:0/3704960421 conn(0x7f29e40fd380 msgr2=0x7f29e4107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 shutdown_connections 2024-04-25T14:03:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.516+0000 7f29e891f640 1 -- 172.21.15.78:0/3704960421 wait complete. 2024-04-25T14:03:34.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:03:34.527 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 -- 172.21.15.78:0/3600951108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 msgr2=0x7f97241094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 --2- 172.21.15.78:0/3600951108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f97241094e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 -- 172.21.15.78:0/3600951108 shutdown_connections 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 --2- 172.21.15.78:0/3600951108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9724109a20 0x7f972410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 --2- 172.21.15.78:0/3600951108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f97241094e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 --2- 172.21.15.78:0/3600951108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9724101f10 0x7f9724106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 -- 172.21.15.78:0/3600951108 >> 172.21.15.78:0/3600951108 conn(0x7f97240fd370 msgr2=0x7f97240ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 -- 172.21.15.78:0/3600951108 shutdown_connections 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.634+0000 7f972ac92640 1 -- 172.21.15.78:0/3600951108 wait complete. 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 Processor -- start 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 -- start start 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9724101f10 0x7f972419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f972419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9724109a20 0x7f97241a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f972410f6c0 con 0x7f97241070c0 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f972410f540 con 0x7f9724101f10 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972ac92640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f972410f840 con 0x7f9724109a20 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f972419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f972419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44046/0 (socket says 172.21.15.78:44046) 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 -- 172.21.15.78:0/4084698426 learned_addr learned my addr 172.21.15.78:0/4084698426 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f972922f640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9724109a20 0x7f97241a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9728a2e640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9724101f10 0x7f972419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 -- 172.21.15.78:0/4084698426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9724109a20 msgr2=0x7f97241a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9724109a20 0x7f97241a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 -- 172.21.15.78:0/4084698426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9724101f10 msgr2=0x7f972419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9724101f10 0x7f972419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 -- 172.21.15.78:0/4084698426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97241a1cf0 con 0x7f97241070c0 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9728a2e640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9724101f10 0x7f972419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.635+0000 7f9723fff640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f972419a7a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9714006010 tx=0x7f9714031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.636+0000 7f9721ffb640 1 -- 172.21.15.78:0/4084698426 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971403f030 con 0x7f97241070c0 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.636+0000 7f9721ffb640 1 -- 172.21.15.78:0/4084698426 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9714003aa0 con 0x7f97241070c0 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.636+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97241a1f80 con 0x7f97241070c0 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.636+0000 7f9721ffb640 1 -- 172.21.15.78:0/4084698426 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714035070 con 0x7f97241070c0 2024-04-25T14:03:34.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.636+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97241a2390 con 0x7f97241070c0 2024-04-25T14:03:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.637+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97240fee90 con 0x7f97241070c0 2024-04-25T14:03:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.638+0000 7f9721ffb640 1 -- 172.21.15.78:0/4084698426 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9714042030 con 0x7f97241070c0 2024-04-25T14:03:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.639+0000 7f9721ffb640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97100afaf0 0x7f97100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.639+0000 7f9728a2e640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97100afaf0 0x7f97100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.639+0000 7f9721ffb640 1 -- 172.21.15.78:0/4084698426 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f97140ff8a0 con 0x7f97241070c0 2024-04-25T14:03:34.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.640+0000 7f9728a2e640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97100afaf0 0x7f97100b2000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9724104b00 tx=0x7f971c008350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.643+0000 7f9721ffb640 1 -- 172.21.15.78:0/4084698426 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97140c8470 con 0x7f97241070c0 2024-04-25T14:03:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:34 smithi129 ceph-mon[32569]: pgmap v470: 97 pgs: 97 active+clean; 2.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:34 smithi078 ceph-mon[29163]: pgmap v470: 97 pgs: 97 active+clean; 2.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:34 smithi112 ceph-mon[32617]: pgmap v470: 97 pgs: 97 active+clean; 2.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:34.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.857+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f972402db50 con 0x7f97241070c0 2024-04-25T14:03:34.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.857+0000 7f9721ffb640 1 -- 172.21.15.78:0/4084698426 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f97140c7d70 con 0x7f97241070c0 2024-04-25T14:03:34.857 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:03:34.857 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:03:34.858 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97100afaf0 msgr2=0x7f97100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97100afaf0 0x7f97100b2000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9724104b00 tx=0x7f971c008350 comp rx=0 tx=0).stop 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 msgr2=0x7f972419a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f972419a7a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9714006010 tx=0x7f9714031a50 comp rx=0 tx=0).stop 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 shutdown_connections 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9724109a20 0x7f97241a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97100afaf0 0x7f97100b2000 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97241070c0 0x7f972419a7a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 --2- 172.21.15.78:0/4084698426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9724101f10 0x7f972419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 >> 172.21.15.78:0/4084698426 conn(0x7f97240fd370 msgr2=0x7f9724105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 shutdown_connections 2024-04-25T14:03:34.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.860+0000 7f972ac92640 1 -- 172.21.15.78:0/4084698426 wait complete. 2024-04-25T14:03:34.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 -- 172.21.15.78:0/2346428788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 msgr2=0x7f9934106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 --2- 172.21.15.78:0/2346428788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f9934106b90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9924009900 tx=0x7f992402a410 comp rx=0 tx=0).stop 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 -- 172.21.15.78:0/2346428788 shutdown_connections 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 --2- 172.21.15.78:0/2346428788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9934109a30 0x7f993410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 --2- 172.21.15.78:0/2346428788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99341070d0 0x7f99341094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 --2- 172.21.15.78:0/2346428788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f9934106b90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 -- 172.21.15.78:0/2346428788 >> 172.21.15.78:0/2346428788 conn(0x7f99340fd380 msgr2=0x7f99340ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 -- 172.21.15.78:0/2346428788 shutdown_connections 2024-04-25T14:03:34.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 -- 172.21.15.78:0/2346428788 wait complete. 2024-04-25T14:03:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 Processor -- start 2024-04-25T14:03:34.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.981+0000 7f9938e29640 1 -- start start 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f993419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99341070d0 0x7f993419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9934109a30 0x7f99341a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f993410f6a0 con 0x7f9934101f20 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f993410f520 con 0x7f99341070d0 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f993410f820 con 0x7f9934109a30 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f993419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f993419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44062/0 (socket says 172.21.15.78:44062) 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 -- 172.21.15.78:0/940112123 learned_addr learned my addr 172.21.15.78:0/940112123 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9932d9d640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9934109a30 0x7f99341a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9931d9b640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99341070d0 0x7f993419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 -- 172.21.15.78:0/940112123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9934109a30 msgr2=0x7f99341a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9934109a30 0x7f99341a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 -- 172.21.15.78:0/940112123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99341070d0 msgr2=0x7f993419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99341070d0 0x7f993419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 -- 172.21.15.78:0/940112123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99341a1e20 con 0x7f9934101f20 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f993259c640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f993419a2e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9924006010 tx=0x7f9924004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f99237fe640 1 -- 172.21.15.78:0/940112123 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992403f030 con 0x7f9934101f20 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99341a20b0 con 0x7f9934101f20 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f99237fe640 1 -- 172.21.15.78:0/940112123 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9924003aa0 con 0x7f9934101f20 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f99237fe640 1 -- 172.21.15.78:0/940112123 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924035070 con 0x7f9934101f20 2024-04-25T14:03:34.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.982+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99341a25a0 con 0x7f9934101f20 2024-04-25T14:03:34.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 Processor -- start 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f99237fe640 1 -- 172.21.15.78:0/940112123 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9924042030 con 0x7f9934101f20 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 -- start start 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155f3390 0x559515995810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155ca4d0 0x559515995d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x559515996a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595158f9d80 con 0x5595155ca4d0 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5595158b3a20 con 0x5595155f3390 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993fdb0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5595158f5d40 con 0x5595155d26b0 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.985+0000 7f993259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155ca4d0 0x559515995d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155ca4d0 0x559515995d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44076/0 (socket says 172.21.15.78:44076) 2024-04-25T14:03:34.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 -- 172.21.15.78:0/2556116379 learned_addr learned my addr 172.21.15.78:0/2556116379 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9932d9d640 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155f3390 0x559515995810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9931d9b640 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x559515996a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f99237fe640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f991c0afaf0 0x7f991c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 -- 172.21.15.78:0/2556116379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 msgr2=0x559515996a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x559515996a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 -- 172.21.15.78:0/2556116379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155f3390 msgr2=0x559515995810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155f3390 0x559515995810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 -- 172.21.15.78:0/2556116379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595158bec30 con 0x5595155ca4d0 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9931d9b640 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x559515996a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9932d9d640 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155f3390 0x559515995810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9932d9d640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f991c0afaf0 0x7f991c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f99237fe640 1 -- 172.21.15.78:0/940112123 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f99240ff3b0 con 0x7f9934101f20 2024-04-25T14:03:34.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f993259c640 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155ca4d0 0x559515995d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f992402db70 tx=0x7f992404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ad7675fc0033a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9920ff9640 1 -- 172.21.15.78:0/2556116379 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240c71d0 con 0x5595155ca4d0 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9932d9d640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f991c0afaf0 0x7f991c0b2000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x559515996860 tx=0x7f992c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9920ff9640 1 -- 172.21.15.78:0/2556116379 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992402dbd0 con 0x5595155ca4d0 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.986+0000 7f9920ff9640 1 -- 172.21.15.78:0/2556116379 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992402d9d0 con 0x5595155ca4d0 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 -- 172.21.15.78:0/2556116379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155ca4d0 msgr2=0x559515995d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155ca4d0 0x559515995d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f992402db70 tx=0x7f992404b000 comp rx=0 tx=0).stop 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 -- 172.21.15.78:0/2556116379 shutdown_connections 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x559515996a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155ca4d0 0x559515995d90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 --2- 172.21.15.78:0/2556116379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155f3390 0x559515995810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 -- 172.21.15.78:0/2556116379 >> 172.21.15.78:0/2556116379 conn(0x5595155e74e0 msgr2=0x559515991020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 -- 172.21.15.78:0/2556116379 shutdown_connections 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.987+0000 7f993fdb0740 1 -- 172.21.15.78:0/2556116379 wait complete. 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 -- start start 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 Processor -- start 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x55951599f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155f3390 0x5595159a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595159972f0 con 0x5595155d26b0 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559515880dc0 con 0x5595155ca4d0 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559515997470 con 0x5595155f3390 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993fdb0740 10 client.0 _ll_register_callbacks cb 0x55951589c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x55951599f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x55951599f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44090/0 (socket says 172.21.15.78:44090) 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 -- 172.21.15.78:0/3984614464 learned_addr learned my addr 172.21.15.78:0/3984614464 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9932d9d640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155f3390 0x5595159a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993259c640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 -- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155f3390 msgr2=0x5595159a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155f3390 0x5595159a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 -- 172.21.15.78:0/3984614464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 msgr2=0x55951599f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f9931d9b640 1 -- 172.21.15.78:0/3984614464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951560b3f0 con 0x5595155d26b0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.988+0000 7f993259c640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f9931d9b640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x55951599f770 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f99280136e0 tx=0x7f9928013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e82e8c64f221e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f9919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f9919ffb640 1 -- 172.21.15.78:0/3984614464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928011dd0 con 0x5595155d26b0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f9919ffb640 1 -- 172.21.15.78:0/3984614464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9928004020 con 0x5595155d26b0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559515907990 con 0x5595155d26b0 2024-04-25T14:03:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f9919ffb640 1 -- 172.21.15.78:0/3984614464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9928004220 con 0x5595155d26b0 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 10 client.16608 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5595156d57d0 con 0x5595155d26b0 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f9919ffb640 1 -- 172.21.15.78:0/3984614464 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559515907990 con 0x5595155d26b0 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f9919ffb640 1 -- 172.21.15.78:0/3984614464 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f992801a070 con 0x5595155d26b0 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 10 client.16608 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 10 client.16608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 20 client.16608 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155b1f60 0x5595159ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 4 client.16608 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.989+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559515901800 con 0x5595155b1f60 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.990+0000 7f993259c640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155b1f60 0x5595159ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.990+0000 7f9919ffb640 10 client.16608 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.990+0000 7f993259c640 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155b1f60 0x5595159ad1f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de5bfa9057d1d3d8 server_cookie=5c2ca48bfd1c999d in_seq=0 out_seq=0 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f9919ffb640 1 -- 172.21.15.78:0/3984614464 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559515901800 con 0x5595155b1f60 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f9919ffb640 10 client.16608 handle_command_reply: tid=0 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 client.16608 shutdown 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 msgr2=0x55951599f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x55951599f770 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f99280136e0 tx=0x7f9928013ba0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 shutdown_connections 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155b1f60 0x5595159ad1f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155f3390 0x5595159a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f9919ffb640 0 client.16608 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x55951599f770 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3984614464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 >> 172.21.15.78:0/3984614464 conn(0x5595155e74e0 msgr2=0x559515993f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 shutdown_connections 2024-04-25T14:03:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 1 -- 172.21.15.78:0/3984614464 wait complete. 2024-04-25T14:03:35.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:34.991+0000 7f993fdb0740 20 client.16608 trim_cache size 0 max 16384 2024-04-25T14:03:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.004+0000 7f993fdb0740 1 Processor -- start 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.004+0000 7f993fdb0740 1 -- start start 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155b1f60 0x55951599d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x5595159a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993fdb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99341adbb0 con 0x5595155b1f60 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993fdb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99341ab480 con 0x5595155ca4d0 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993fdb0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99341a92d0 con 0x5595155d26b0 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155b1f60 0x55951599d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155b1f60 0x55951599d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44094/0 (socket says 172.21.15.78:44094) 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 -- 172.21.15.78:0/3970411763 learned_addr learned my addr 172.21.15.78:0/3970411763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 -- 172.21.15.78:0/3970411763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 msgr2=0x5595159a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 --2- 172.21.15.78:0/3970411763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x5595159a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 -- 172.21.15.78:0/3970411763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 msgr2=0x55951599dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 --2- 172.21.15.78:0/3970411763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 -- 172.21.15.78:0/3970411763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595159a6b30 con 0x5595155b1f60 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f993259c640 1 --2- 172.21.15.78:0/3970411763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155b1f60 0x55951599d850 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f99240c7810 tx=0x7f9924003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75701af950373954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f991bfff640 1 -- 172.21.15.78:0/3970411763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924049070 con 0x5595155b1f60 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f991bfff640 1 -- 172.21.15.78:0/3970411763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99240c7390 con 0x5595155b1f60 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.005+0000 7f991bfff640 1 -- 172.21.15.78:0/3970411763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240c6ff0 con 0x5595155b1f60 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 -- 172.21.15.78:0/3970411763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155b1f60 msgr2=0x55951599d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3970411763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155b1f60 0x55951599d850 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f99240c7810 tx=0x7f9924003f90 comp rx=0 tx=0).stop 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 -- 172.21.15.78:0/3970411763 shutdown_connections 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3970411763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155d26b0 0x5595159a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3970411763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155ca4d0 0x55951599dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3970411763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155b1f60 0x55951599d850 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 -- 172.21.15.78:0/3970411763 >> 172.21.15.78:0/3970411763 conn(0x5595155e74e0 msgr2=0x55951599a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 -- 172.21.15.78:0/3970411763 shutdown_connections 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 -- 172.21.15.78:0/3970411763 wait complete. 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 -- start start 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.006+0000 7f993fdb0740 1 Processor -- start 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155b1f60 0x5595159a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155ca4d0 0x5595159a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x5595159a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99341a9940 con 0x5595155d26b0 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99341aadb0 con 0x5595155b1f60 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99341aa010 con 0x5595155ca4d0 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 10 client.0 _ll_register_callbacks cb 0x7f9928005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x5595159a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x5595159a52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44100/0 (socket says 172.21.15.78:44100) 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 -- 172.21.15.78:0/3225199919 learned_addr learned my addr 172.21.15.78:0/3225199919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9931d9b640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155ca4d0 0x5595159a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993259c640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155b1f60 0x5595159a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 -- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155ca4d0 msgr2=0x5595159a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155ca4d0 0x5595159a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 -- 172.21.15.78:0/3225199919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155b1f60 msgr2=0x5595159a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155b1f60 0x5595159a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 -- 172.21.15.78:0/3225199919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595159a5a30 con 0x5595155d26b0 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993259c640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155b1f60 0x5595159a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9932d9d640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x5595159a52a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f992c012e70 tx=0x7f992c0085f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fb4fb4aa71d2a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9919ffb640 1 -- 172.21.15.78:0/3225199919 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c008d50 con 0x5595155d26b0 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9919ffb640 1 -- 172.21.15.78:0/3225199919 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f992c01b650 con 0x5595155d26b0 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99240c7390 con 0x5595155d26b0 2024-04-25T14:03:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f9919ffb640 1 -- 172.21.15.78:0/3225199919 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f992c01b850 con 0x5595155d26b0 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 10 client.16614 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.007+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559515a146a0 con 0x5595155d26b0 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f9919ffb640 1 -- 172.21.15.78:0/3225199919 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99240c7390 con 0x5595155d26b0 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f9919ffb640 1 -- 172.21.15.78:0/3225199919 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f992c026020 con 0x5595155d26b0 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993fdb0740 10 client.16614 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993fdb0740 10 client.16614 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993fdb0740 20 client.16614 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155f3390 0x559515a170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993fdb0740 4 client.16614 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55951587c610 con 0x5595155f3390 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993259c640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155f3390 0x559515a170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f9919ffb640 10 client.16614 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.008+0000 7f993259c640 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155f3390 0x559515a170f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4108c8b3cb3f27b1 server_cookie=878ea4102d31f337 in_seq=0 out_seq=0 2024-04-25T14:03:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f9919ffb640 1 -- 172.21.15.78:0/3225199919 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55951587c610 con 0x5595155f3390 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f9919ffb640 10 client.16614 handle_command_reply: tid=0 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 client.16614 shutdown 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 msgr2=0x5595159a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x5595159a52a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f992c012e70 tx=0x7f992c0085f0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 shutdown_connections 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5595155f3390 0x559515a170f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595155d26b0 0x5595159a52a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595155ca4d0 0x5595159a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f9919ffb640 0 client.16614 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 --2- 172.21.15.78:0/3225199919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595155b1f60 0x5595159a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 >> 172.21.15.78:0/3225199919 conn(0x5595155e74e0 msgr2=0x55951599d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 shutdown_connections 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 1 -- 172.21.15.78:0/3225199919 wait complete. 2024-04-25T14:03:35.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f993fdb0740 20 client.16614 trim_cache size 0 max 16384 2024-04-25T14:03:35.016 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:03:35.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f991c0afaf0 msgr2=0x7f991c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f9938e29640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f991c0afaf0 0x7f991c0b2000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x559515996860 tx=0x7f992c004a40 comp rx=0 tx=0).stop 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.009+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 msgr2=0x7f993419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f993419a2e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9924006010 tx=0x7f9924004270 comp rx=0 tx=0).stop 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 shutdown_connections 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9934109a30 0x7f99341a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99341070d0 0x7f993419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f991c0afaf0 0x7f991c0b2000 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 --2- 172.21.15.78:0/940112123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9934101f20 0x7f993419a2e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 >> 172.21.15.78:0/940112123 conn(0x7f99340fd380 msgr2=0x7f99341053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 shutdown_connections 2024-04-25T14:03:35.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.010+0000 7f9938e29640 1 -- 172.21.15.78:0/940112123 wait complete. 2024-04-25T14:03:35.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:03:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 -- 172.21.15.78:0/2366110375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c103c20 msgr2=0x7f546c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 --2- 172.21.15.78:0/2366110375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c103c20 0x7f546c104030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5460009900 tx=0x7f546002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 -- 172.21.15.78:0/2366110375 shutdown_connections 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 --2- 172.21.15.78:0/2366110375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c108920 0x7f546c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 --2- 172.21.15.78:0/2366110375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c103c20 0x7f546c104030 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 --2- 172.21.15.78:0/2366110375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f546c1029e0 0x7f546c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 -- 172.21.15.78:0/2366110375 >> 172.21.15.78:0/2366110375 conn(0x7f546c0fdbd0 msgr2=0x7f546c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 -- 172.21.15.78:0/2366110375 shutdown_connections 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.130+0000 7f54721c5640 1 -- 172.21.15.78:0/2366110375 wait complete. 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 Processor -- start 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 -- start start 2024-04-25T14:03:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c1029e0 0x7f546c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c103c20 0x7f546c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f546c108920 0x7f546c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f546c10e510 con 0x7f546c1029e0 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f546c10e390 con 0x7f546c108920 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f54721c5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f546c10e690 con 0x7f546c103c20 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c1029e0 0x7f546c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c1029e0 0x7f546c199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44114/0 (socket says 172.21.15.78:44114) 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.131+0000 7f546b7fe640 1 -- 172.21.15.78:0/650086320 learned_addr learned my addr 172.21.15.78:0/650086320 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546bfff640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f546c108920 0x7f546c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546affd640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c103c20 0x7f546c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546b7fe640 1 -- 172.21.15.78:0/650086320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c103c20 msgr2=0x7f546c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546b7fe640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c103c20 0x7f546c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546b7fe640 1 -- 172.21.15.78:0/650086320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f546c108920 msgr2=0x7f546c1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546b7fe640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f546c108920 0x7f546c1a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546b7fe640 1 -- 172.21.15.78:0/650086320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c1a0f80 con 0x7f546c1029e0 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546bfff640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f546c108920 0x7f546c1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546affd640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c103c20 0x7f546c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f546b7fe640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c1029e0 0x7f546c199440 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f545c005fd0 tx=0x7f545c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f5468ff9640 1 -- 172.21.15.78:0/650086320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c017070 con 0x7f546c1029e0 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c1a1270 con 0x7f546c1029e0 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f5468ff9640 1 -- 172.21.15.78:0/650086320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f545c009c30 con 0x7f546c1029e0 2024-04-25T14:03:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f5468ff9640 1 -- 172.21.15.78:0/650086320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c003d10 con 0x7f546c1029e0 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.132+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1a17c0 con 0x7f546c1029e0 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 Processor -- start 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 -- start start 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f5468ff9640 1 -- 172.21.15.78:0/650086320 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f545c010460 con 0x7f546c1029e0 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d682a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d682fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d683c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55709d5e4d80 con 0x55709d2de390 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55709d59ea20 con 0x55709d2bd6b0 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f547907b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55709d5e0d40 con 0x55709d2b54d0 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d682a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d682a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44122/0 (socket says 172.21.15.78:44122) 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 -- 172.21.15.78:0/1611607062 learned_addr learned my addr 172.21.15.78:0/1611607062 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546b7fe640 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d682fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546affd640 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d683c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 -- 172.21.15.78:0/1611607062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 msgr2=0x55709d682fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d682fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 -- 172.21.15.78:0/1611607062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 msgr2=0x55709d683c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d683c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 -- 172.21.15.78:0/1611607062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709d5a9c30 con 0x55709d2de390 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f5468ff9640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54540afaf0 0x7f54540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.135+0000 7f546bfff640 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d682a20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5464009e40 tx=0x7f546400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a72370897490f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f546b7fe640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54540afaf0 0x7f54540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f5451ffb640 1 -- 172.21.15.78:0/1611607062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464004d20 con 0x55709d2de390 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f5451ffb640 1 -- 172.21.15.78:0/1611607062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54640054d0 con 0x55709d2de390 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f5451ffb640 1 -- 172.21.15.78:0/1611607062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464011800 con 0x55709d2de390 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f5468ff9640 1 -- 172.21.15.78:0/650086320 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f545c0de9e0 con 0x7f546c1029e0 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f547907b740 1 -- 172.21.15.78:0/1611607062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 msgr2=0x55709d682a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f546b7fe640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54540afaf0 0x7f54540b2000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f545c0dc840 tx=0x7f545c0e4630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.136+0000 7f547907b740 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d682a20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5464009e40 tx=0x7f546400f730 comp rx=0 tx=0).stop 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- 172.21.15.78:0/1611607062 shutdown_connections 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d683c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d682fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 --2- 172.21.15.78:0/1611607062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d682a20 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- 172.21.15.78:0/1611607062 >> 172.21.15.78:0/1611607062 conn(0x55709d2d24e0 msgr2=0x55709d67c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- 172.21.15.78:0/1611607062 shutdown_connections 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- 172.21.15.78:0/1611607062 wait complete. 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- start start 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 Processor -- start 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2b54d0 0x55709d68c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d68cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2de390 0x55709d693c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55709d684740 con 0x55709d2b54d0 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55709d56bdc0 con 0x55709d2bd6b0 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55709d6848c0 con 0x55709d2de390 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f547907b740 10 client.0 _ll_register_callbacks cb 0x55709d587600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.137+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2b54d0 0x55709d68c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2b54d0 0x55709d68c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44138/0 (socket says 172.21.15.78:44138) 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 -- 172.21.15.78:0/3770608441 learned_addr learned my addr 172.21.15.78:0/3770608441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546affd640 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d68cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2de390 0x55709d693c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 -- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2de390 msgr2=0x55709d693c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2de390 0x55709d693c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 -- 172.21.15.78:0/3770608441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 msgr2=0x55709d68cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d68cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 -- 172.21.15.78:0/3770608441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55709d2f63f0 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f546bfff640 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2b54d0 0x55709d68c800 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5464006010 tx=0x7f546400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41387c85420d56f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f544affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f544affd640 1 -- 172.21.15.78:0/3770608441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464011540 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f544affd640 1 -- 172.21.15.78:0/3770608441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f54640116e0 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54640054d0 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f544affd640 1 -- 172.21.15.78:0/3770608441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f54640118e0 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f547907b740 10 client.16626 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55709d3c07d0 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.138+0000 7f544affd640 1 -- 172.21.15.78:0/3770608441 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54640054d0 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f544affd640 1 -- 172.21.15.78:0/3770608441 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f54640199a0 con 0x55709d2b54d0 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f547907b740 10 client.16626 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f547907b740 10 client.16626 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:03:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f547907b740 20 client.16626 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f547907b740 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d29cf60 0x55709d69a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f547907b740 4 client.16626 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55709d5f2990 con 0x55709d29cf60 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.139+0000 7f546affd640 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d29cf60 0x55709d69a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.140+0000 7f544affd640 10 client.16626 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.140+0000 7f546affd640 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d29cf60 0x55709d69a150 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9c9f9ccaf38753 server_cookie=d6c4128c6a2eaee6 in_seq=0 out_seq=0 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.140+0000 7f544affd640 1 -- 172.21.15.78:0/3770608441 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55709d5f2990 con 0x55709d29cf60 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.140+0000 7f544affd640 10 client.16626 handle_command_reply: tid=0 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.140+0000 7f547907b740 1 client.16626 shutdown 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.140+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2b54d0 msgr2=0x55709d68c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.140+0000 7f547907b740 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2b54d0 0x55709d68c800 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5464006010 tx=0x7f546400ff00 comp rx=0 tx=0).stop 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 shutdown_connections 2024-04-25T14:03:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d29cf60 0x55709d69a150 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2de390 0x55709d693c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d68cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 --2- 172.21.15.78:0/3770608441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2b54d0 0x55709d68c800 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f544affd640 0 client.16626 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 >> 172.21.15.78:0/3770608441 conn(0x55709d2d24e0 msgr2=0x55709d67ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 shutdown_connections 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 1 -- 172.21.15.78:0/3770608441 wait complete. 2024-04-25T14:03:35.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.141+0000 7f547907b740 20 client.16626 trim_cache size 0 max 16384 2024-04-25T14:03:35.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 Processor -- start 2024-04-25T14:03:35.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 -- start start 2024-04-25T14:03:35.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d68cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d693bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d69e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1aba80 con 0x55709d2de390 2024-04-25T14:03:35.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1a9350 con 0x55709d2bd6b0 2024-04-25T14:03:35.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.155+0000 7f547907b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1a71a0 con 0x55709d2b54d0 2024-04-25T14:03:35.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d69e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d69e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44154/0 (socket says 172.21.15.78:44154) 2024-04-25T14:03:35.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 -- 172.21.15.78:0/1955704021 learned_addr learned my addr 172.21.15.78:0/1955704021 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d68cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546affd640 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d693bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 -- 172.21.15.78:0/1955704021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 msgr2=0x55709d68cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d68cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 -- 172.21.15.78:0/1955704021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 msgr2=0x55709d693bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d693bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 -- 172.21.15.78:0/1955704021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709d687f90 con 0x55709d2de390 2024-04-25T14:03:35.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546affd640 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d693bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:35.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.156+0000 7f546bfff640 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d69e830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x55709d696e90 tx=0x7f5464009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab183d4bdc6258a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.157+0000 7f54517fa640 1 -- 172.21.15.78:0/1955704021 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464005890 con 0x55709d2de390 2024-04-25T14:03:35.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.157+0000 7f54517fa640 1 -- 172.21.15.78:0/1955704021 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546401e030 con 0x55709d2de390 2024-04-25T14:03:35.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.157+0000 7f54517fa640 1 -- 172.21.15.78:0/1955704021 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464011a80 con 0x55709d2de390 2024-04-25T14:03:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.157+0000 7f547907b740 1 -- 172.21.15.78:0/1955704021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 msgr2=0x55709d69e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.157+0000 7f547907b740 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d69e830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x55709d696e90 tx=0x7f5464009f90 comp rx=0 tx=0).stop 2024-04-25T14:03:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- 172.21.15.78:0/1955704021 shutdown_connections 2024-04-25T14:03:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d69e830 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d693bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 --2- 172.21.15.78:0/1955704021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d68cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- 172.21.15.78:0/1955704021 >> 172.21.15.78:0/1955704021 conn(0x55709d2d24e0 msgr2=0x55709d68c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- 172.21.15.78:0/1955704021 shutdown_connections 2024-04-25T14:03:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- 172.21.15.78:0/1955704021 wait complete. 2024-04-25T14:03:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- start start 2024-04-25T14:03:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 Processor -- start 2024-04-25T14:03:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d6939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d692070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d6925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1a7810 con 0x55709d2de390 2024-04-25T14:03:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1a8c80 con 0x55709d2bd6b0 2024-04-25T14:03:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1a7ee0 con 0x55709d2b54d0 2024-04-25T14:03:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.158+0000 7f547907b740 10 client.0 _ll_register_callbacks cb 0x7f545c0e4500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d6925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d692070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d6925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44164/0 (socket says 172.21.15.78:44164) 2024-04-25T14:03:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 -- 172.21.15.78:0/2158864584 learned_addr learned my addr 172.21.15.78:0/2158864584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d6939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 -- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 msgr2=0x55709d6939b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d6939b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 -- 172.21.15.78:0/2158864584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 msgr2=0x55709d692070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d692070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 -- 172.21.15.78:0/2158864584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55709d692c20 con 0x55709d2de390 2024-04-25T14:03:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f546bfff640 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d6925b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5464004a20 tx=0x7f5464000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3145355ca9c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f544affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f544affd640 1 -- 172.21.15.78:0/2158864584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464011540 con 0x55709d2de390 2024-04-25T14:03:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f544affd640 1 -- 172.21.15.78:0/2158864584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f54640116e0 con 0x55709d2de390 2024-04-25T14:03:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546401e030 con 0x55709d2de390 2024-04-25T14:03:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f544affd640 1 -- 172.21.15.78:0/2158864584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f546402b0d0 con 0x55709d2de390 2024-04-25T14:03:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f547907b740 10 client.16632 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55709d701210 con 0x55709d2de390 2024-04-25T14:03:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.159+0000 7f544affd640 1 -- 172.21.15.78:0/2158864584 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546401e030 con 0x55709d2de390 2024-04-25T14:03:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f544affd640 1 -- 172.21.15.78:0/2158864584 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f546402ba80 con 0x55709d2de390 2024-04-25T14:03:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f547907b740 10 client.16632 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f547907b740 10 client.16632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:03:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f547907b740 20 client.16632 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f547907b740 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d2723a0 0x55709d7039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f547907b740 4 client.16632 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55709d5ec800 con 0x55709d2723a0 2024-04-25T14:03:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f546affd640 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d2723a0 0x55709d7039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f546affd640 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d2723a0 0x55709d7039b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56e3ca50ae614e24 server_cookie=40f73b97caf70987 in_seq=0 out_seq=0 2024-04-25T14:03:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.160+0000 7f544affd640 10 client.16632 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f544affd640 1 -- 172.21.15.78:0/2158864584 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55709d5ec800 con 0x55709d2723a0 2024-04-25T14:03:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f544affd640 10 client.16632 handle_command_reply: tid=0 2024-04-25T14:03:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 client.16632 shutdown 2024-04-25T14:03:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 msgr2=0x55709d6925b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d6925b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5464004a20 tx=0x7f5464000f40 comp rx=0 tx=0).stop 2024-04-25T14:03:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 shutdown_connections 2024-04-25T14:03:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55709d2de390 0x55709d6925b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55709d2bd6b0 0x55709d692070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55709d2723a0 0x55709d7039b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 --2- 172.21.15.78:0/2158864584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55709d2b54d0 0x55709d6939b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 >> 172.21.15.78:0/2158864584 conn(0x55709d2d24e0 msgr2=0x55709d67eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f544affd640 0 client.16632 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:03:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 shutdown_connections 2024-04-25T14:03:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 1 -- 172.21.15.78:0/2158864584 wait complete. 2024-04-25T14:03:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f547907b740 20 client.16632 trim_cache size 0 max 16384 2024-04-25T14:03:35.179 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:03:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54540afaf0 msgr2=0x7f54540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54540afaf0 0x7f54540b2000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f545c0dc840 tx=0x7f545c0e4630 comp rx=0 tx=0).stop 2024-04-25T14:03:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c1029e0 msgr2=0x7f546c199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c1029e0 0x7f546c199440 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f545c005fd0 tx=0x7f545c00dde0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 shutdown_connections 2024-04-25T14:03:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f546c108920 0x7f546c1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c103c20 0x7f546c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f54540afaf0 0x7f54540b2000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 --2- 172.21.15.78:0/650086320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f546c1029e0 0x7f546c199440 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.161+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 >> 172.21.15.78:0/650086320 conn(0x7f546c0fdbd0 msgr2=0x7f546c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.162+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 shutdown_connections 2024-04-25T14:03:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.162+0000 7f54721c5640 1 -- 172.21.15.78:0/650086320 wait complete. 2024-04-25T14:03:35.183 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 -- 172.21.15.78:0/765401703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101045f0 msgr2=0x7fa910106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 --2- 172.21.15.78:0/765401703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101045f0 0x7fa910106a10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009900 tx=0x7fa90c02a410 comp rx=0 tx=0).stop 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 -- 172.21.15.78:0/765401703 shutdown_connections 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 --2- 172.21.15.78:0/765401703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101098b0 0x7fa91010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 --2- 172.21.15.78:0/765401703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa910106f50 0x7fa910109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 --2- 172.21.15.78:0/765401703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101045f0 0x7fa910106a10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 -- 172.21.15.78:0/765401703 >> 172.21.15.78:0/765401703 conn(0x7fa9100fdbf0 msgr2=0x7fa9101000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 -- 172.21.15.78:0/765401703 shutdown_connections 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 -- 172.21.15.78:0/765401703 wait complete. 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 Processor -- start 2024-04-25T14:03:35.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.294+0000 7fa9182b1640 1 -- start start 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101045f0 0x7fa91019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa910106f50 0x7fa91019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101098b0 0x7fa9101a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010f650 con 0x7fa9101098b0 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010f4d0 con 0x7fa9101045f0 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010f7d0 con 0x7fa910106f50 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101098b0 0x7fa9101a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101098b0 0x7fa9101a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44174/0 (socket says 172.21.15.78:44174) 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 -- 172.21.15.78:0/1868976930 learned_addr learned my addr 172.21.15.78:0/1868976930 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91604d640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101045f0 0x7fa91019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91584c640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa910106f50 0x7fa91019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 -- 172.21.15.78:0/1868976930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa910106f50 msgr2=0x7fa91019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa910106f50 0x7fa91019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 -- 172.21.15.78:0/1868976930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101045f0 msgr2=0x7fa91019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101045f0 0x7fa91019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 -- 172.21.15.78:0/1868976930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9101a1cf0 con 0x7fa9101098b0 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa91684e640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101098b0 0x7fa9101a1730 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa90800ccb0 tx=0x7fa90800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9077fe640 1 -- 172.21.15.78:0/1868976930 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908003970 con 0x7fa9101098b0 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9077fe640 1 -- 172.21.15.78:0/1868976930 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa908004030 con 0x7fa9101098b0 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9101a1fe0 con 0x7fa9101098b0 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9077fe640 1 -- 172.21.15.78:0/1868976930 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90800f6b0 con 0x7fa9101098b0 2024-04-25T14:03:35.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.295+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101a23f0 con 0x7fa9101098b0 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.298+0000 7fa91f23a740 1 Processor -- start 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.298+0000 7fa9077fe640 1 -- 172.21.15.78:0/1868976930 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa908003b10 con 0x7fa9101098b0 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.298+0000 7fa91f23a740 1 -- start start 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91f23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619a560ed80 con 0x5619a52df4d0 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91f23a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5619a55c8a20 con 0x5619a52e76b0 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91f23a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5619a560ad40 con 0x5619a5308390 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44188/0 (socket says 172.21.15.78:44188) 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 -- 172.21.15.78:0/1045147558 learned_addr learned my addr 172.21.15.78:0/1045147558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91584c640 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91584c640 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa9077fe640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa8fc0afaf0 0x7fa8fc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91684e640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa8fc0afaf0 0x7fa8fc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 -- 172.21.15.78:0/1045147558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 msgr2=0x5619a56aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 -- 172.21.15.78:0/1045147558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 msgr2=0x5619a56aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 -- 172.21.15.78:0/1045147558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a55d3c30 con 0x5619a52df4d0 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91584c640 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa9077fe640 1 -- 172.21.15.78:0/1868976930 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa9080dedb0 con 0x7fa9101098b0 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.299+0000 7fa91604d640 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56aad90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009900 tx=0x7fa90c004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f3d2fd87736108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.300+0000 7fa91684e640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa8fc0afaf0 0x7fa8fc0b2000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9080a44f0 tx=0x7fa90809bbc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.300+0000 7fa904bfe640 1 -- 172.21.15.78:0/1045147558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c035030 con 0x5619a52df4d0 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.300+0000 7fa904bfe640 1 -- 172.21.15.78:0/1045147558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90c0054f0 con 0x5619a52df4d0 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.300+0000 7fa904bfe640 1 -- 172.21.15.78:0/1045147558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c003850 con 0x5619a52df4d0 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.300+0000 7fa91f23a740 1 -- 172.21.15.78:0/1045147558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 msgr2=0x5619a56aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.300+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56aad90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009900 tx=0x7fa90c004200 comp rx=0 tx=0).stop 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- 172.21.15.78:0/1045147558 shutdown_connections 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56aad90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1045147558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- 172.21.15.78:0/1045147558 >> 172.21.15.78:0/1045147558 conn(0x5619a52fc4e0 msgr2=0x5619a56a6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- 172.21.15.78:0/1045147558 shutdown_connections 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- 172.21.15.78:0/1045147558 wait complete. 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- start start 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 Processor -- start 2024-04-25T14:03:35.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5308390 0x5619a56bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619a56ac4d0 con 0x5619a5308390 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5619a5595dc0 con 0x5619a52e76b0 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5619a56ac650 con 0x5619a52df4d0 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.301+0000 7fa91f23a740 10 client.0 _ll_register_callbacks cb 0x5619a55b1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5308390 0x5619a56bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5308390 0x5619a56bba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44200/0 (socket says 172.21.15.78:44200) 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 -- 172.21.15.78:0/1315298367 learned_addr learned my addr 172.21.15.78:0/1315298367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91604d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 -- 172.21.15.78:0/1315298367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 msgr2=0x5619a56b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 -- 172.21.15.78:0/1315298367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 msgr2=0x5619a56b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 -- 172.21.15.78:0/1315298367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a53203f0 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91584c640 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5308390 0x5619a56bba00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa900005870 tx=0x7fa9000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f009bb645db860e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa8f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/1315298367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900012450 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/1315298367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa9000125f0 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c0054f0 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/1315298367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa900027050 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91f23a740 10 client.16644 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.302+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5619a53ea7d0 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/1315298367 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90c0054f0 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/1315298367 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa90001e070 con 0x5619a5308390 2024-04-25T14:03:35.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa91f23a740 10 client.16644 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa91f23a740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa91f23a740 20 client.16644 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a52c6f60 0x5619a56c2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa91f23a740 4 client.16644 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619a561c990 con 0x5619a52c6f60 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.303+0000 7fa91604d640 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a52c6f60 0x5619a56c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa8f9ffb640 10 client.16644 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91604d640 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a52c6f60 0x5619a56c2250 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639508678c6312cf server_cookie=908ca0c83eef7346 in_seq=0 out_seq=0 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/1315298367 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619a561c990 con 0x5619a52c6f60 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa8f9ffb640 10 client.16644 handle_command_reply: tid=0 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 client.16644 shutdown 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5308390 msgr2=0x5619a56bba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5308390 0x5619a56bba00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa900005870 tx=0x7fa9000046c0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 shutdown_connections 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5308390 0x5619a56bba00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a52c6f60 0x5619a56c2250 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 --2- 172.21.15.78:0/1315298367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa8f9ffb640 0 client.16644 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.304+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 >> 172.21.15.78:0/1315298367 conn(0x5619a52fc4e0 msgr2=0x5619a56a8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.305+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 shutdown_connections 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.305+0000 7fa91f23a740 1 -- 172.21.15.78:0/1315298367 wait complete. 2024-04-25T14:03:35.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.305+0000 7fa91f23a740 20 client.16644 trim_cache size 0 max 16384 2024-04-25T14:03:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 Processor -- start 2024-04-25T14:03:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 -- start start 2024-04-25T14:03:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ada70 con 0x5619a52df4d0 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ab340 con 0x5619a52e76b0 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91f23a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a9190 con 0x5619a5308390 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44216/0 (socket says 172.21.15.78:44216) 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 -- 172.21.15.78:0/4012413028 learned_addr learned my addr 172.21.15.78:0/4012413028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91604d640 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 -- 172.21.15.78:0/4012413028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 msgr2=0x5619a56a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 -- 172.21.15.78:0/4012413028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 msgr2=0x5619a56bb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56bb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 -- 172.21.15.78:0/4012413028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a56aa360 con 0x5619a52df4d0 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa91584c640 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56a5720 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa900005d00 tx=0x7fa90000c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823daa87b418f2ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa8fbfff640 1 -- 172.21.15.78:0/4012413028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000151b0 con 0x5619a52df4d0 2024-04-25T14:03:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa8fbfff640 1 -- 172.21.15.78:0/4012413028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa900015350 con 0x5619a52df4d0 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.319+0000 7fa8fbfff640 1 -- 172.21.15.78:0/4012413028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900004730 con 0x5619a52df4d0 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 -- 172.21.15.78:0/4012413028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 msgr2=0x5619a56a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56a5720 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa900005d00 tx=0x7fa90000c5f0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 -- 172.21.15.78:0/4012413028 shutdown_connections 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a5308390 0x5619a56a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a52e76b0 0x5619a56bb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 --2- 172.21.15.78:0/4012413028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52df4d0 0x5619a56a5720 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 -- 172.21.15.78:0/4012413028 >> 172.21.15.78:0/4012413028 conn(0x5619a52fc4e0 msgr2=0x5619a56aa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 -- 172.21.15.78:0/4012413028 shutdown_connections 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 -- 172.21.15.78:0/4012413028 wait complete. 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 -- start start 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.320+0000 7fa91f23a740 1 Processor -- start 2024-04-25T14:03:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52e76b0 0x5619a56b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a5308390 0x5619a56ba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a9800 con 0x5619a52e76b0 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101aac70 con 0x5619a5308390 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a9ed0 con 0x5619a52df4d0 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 10 client.0 _ll_register_callbacks cb 0x7fa90000c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52e76b0 0x5619a56b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52e76b0 0x5619a56b9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44224/0 (socket says 172.21.15.78:44224) 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 -- 172.21.15.78:0/3820518569 learned_addr learned my addr 172.21.15.78:0/3820518569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91584c640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a5308390 0x5619a56ba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91584c640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 -- 172.21.15.78:0/3820518569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 msgr2=0x5619a56b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 -- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a5308390 msgr2=0x5619a56ba060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a5308390 0x5619a56ba060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 -- 172.21.15.78:0/3820518569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a56ba620 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91584c640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91604d640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52e76b0 0x5619a56b9130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005710 tx=0x7fa90c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef157ffd1cbc0fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa8f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/3820518569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c035040 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/3820518569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa90c003a00 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619a5616800 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/3820518569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa90c0043b0 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 10 client.16650 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.321+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5619a56bb900 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/3820518569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5619a5616800 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/3820518569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa90c03b430 con 0x5619a52e76b0 2024-04-25T14:03:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa91f23a740 10 client.16650 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa91f23a740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa91f23a740 20 client.16650 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa91f23a740 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a529c3a0 0x5619a572c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa91f23a740 4 client.16650 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619a5591610 con 0x5619a529c3a0 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.322+0000 7fa91584c640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a529c3a0 0x5619a572c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa8f9ffb640 10 client.16650 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91584c640 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a529c3a0 0x5619a572c5d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a117c11dbaf046 server_cookie=e24a41bee91f36d1 in_seq=0 out_seq=0 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa8f9ffb640 1 -- 172.21.15.78:0/3820518569 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619a5591610 con 0x5619a529c3a0 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa8f9ffb640 10 client.16650 handle_command_reply: tid=0 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 client.16650 shutdown 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52e76b0 msgr2=0x5619a56b9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52e76b0 0x5619a56b9130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005710 tx=0x7fa90c009580 comp rx=0 tx=0).stop 2024-04-25T14:03:35.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 shutdown_connections 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5619a5308390 0x5619a56ba060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a52e76b0 0x5619a56b9130 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5619a529c3a0 0x5619a572c5d0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 --2- 172.21.15.78:0/3820518569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619a52df4d0 0x5619a56b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 >> 172.21.15.78:0/3820518569 conn(0x5619a52fc4e0 msgr2=0x5619a56aa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa8f9ffb640 0 client.16650 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 shutdown_connections 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 1 -- 172.21.15.78:0/3820518569 wait complete. 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa91f23a740 20 client.16650 trim_cache size 0 max 16384 2024-04-25T14:03:35.330 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:03:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa8fc0afaf0 msgr2=0x7fa8fc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.323+0000 7fa9182b1640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa8fc0afaf0 0x7fa8fc0b2000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9080a44f0 tx=0x7fa90809bbc0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101098b0 msgr2=0x7fa9101a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:35.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101098b0 0x7fa9101a1730 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa90800ccb0 tx=0x7fa90800a620 comp rx=0 tx=0).stop 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 shutdown_connections 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa8fc0afaf0 0x7fa8fc0b2000 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9101098b0 0x7fa9101a1730 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa910106f50 0x7fa91019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 --2- 172.21.15.78:0/1868976930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9101045f0 0x7fa91019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 >> 172.21.15.78:0/1868976930 conn(0x7fa9100fdbf0 msgr2=0x7fa9101052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 shutdown_connections 2024-04-25T14:03:35.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:35.324+0000 7fa9182b1640 1 -- 172.21.15.78:0/1868976930 wait complete. 2024-04-25T14:03:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:35 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/4084698426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:35.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:35 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/4084698426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:35 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/4084698426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:03:36.335 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:03:36.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:03:36.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:03:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.457+0000 7f3736fee640 1 -- 172.21.15.78:0/437374863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730101750 msgr2=0x7f3730101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.457+0000 7f3736fee640 1 --2- 172.21.15.78:0/437374863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730101750 0x7f3730101bc0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f371c009900 tx=0x7f371c02a410 comp rx=0 tx=0).stop 2024-04-25T14:03:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.457+0000 7f3736fee640 1 -- 172.21.15.78:0/437374863 shutdown_connections 2024-04-25T14:03:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.457+0000 7f3736fee640 1 --2- 172.21.15.78:0/437374863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730102250 0x7f373010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.457+0000 7f3736fee640 1 --2- 172.21.15.78:0/437374863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730101750 0x7f3730101bc0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.457+0000 7f3736fee640 1 --2- 172.21.15.78:0/437374863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37301050a0 0x7f37301054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.457+0000 7f3736fee640 1 -- 172.21.15.78:0/437374863 >> 172.21.15.78:0/437374863 conn(0x7f37300fd350 msgr2=0x7f37300ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 -- 172.21.15.78:0/437374863 shutdown_connections 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 -- 172.21.15.78:0/437374863 wait complete. 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 Processor -- start 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 -- start start 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730101750 0x7f373019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730102250 0x7f373019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37301050a0 0x7f37301a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f373010f630 con 0x7f3730102250 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f373010f4b0 con 0x7f3730101750 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f3736fee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f373010f7b0 con 0x7f37301050a0 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f372ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730102250 0x7f373019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f372ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730102250 0x7f373019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44234/0 (socket says 172.21.15.78:44234) 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.458+0000 7f372ffff640 1 -- 172.21.15.78:0/1023653107 learned_addr learned my addr 172.21.15.78:0/1023653107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f3734d8a640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730101750 0x7f373019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f373558b640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37301050a0 0x7f37301a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372ffff640 1 -- 172.21.15.78:0/1023653107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37301050a0 msgr2=0x7f37301a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372ffff640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37301050a0 0x7f37301a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372ffff640 1 -- 172.21.15.78:0/1023653107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730101750 msgr2=0x7f373019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372ffff640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730101750 0x7f373019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372ffff640 1 -- 172.21.15.78:0/1023653107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301a1d70 con 0x7f3730102250 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372ffff640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730102250 0x7f373019a770 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f371c006010 tx=0x7f371c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372dffb640 1 -- 172.21.15.78:0/1023653107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c03f030 con 0x7f3730102250 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372dffb640 1 -- 172.21.15.78:0/1023653107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c003aa0 con 0x7f3730102250 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37301a2000 con 0x7f3730102250 2024-04-25T14:03:36.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f372dffb640 1 -- 172.21.15.78:0/1023653107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c035070 con 0x7f3730102250 2024-04-25T14:03:36.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.459+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301a2540 con 0x7f3730102250 2024-04-25T14:03:36.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f372dffb640 1 -- 172.21.15.78:0/1023653107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f371c042030 con 0x7f3730102250 2024-04-25T14:03:36.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 Processor -- start 2024-04-25T14:03:36.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 -- start start 2024-04-25T14:03:36.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c87410 0x5609ca0284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca028a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca02f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609c9f8db00 con 0x5609c9c5e550 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609c9f29920 con 0x5609c9c87410 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373df77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5609c9f89ac0 con 0x5609c9c66730 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f3734d8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca028a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f3734d8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca028a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44250/0 (socket says 172.21.15.78:44250) 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f3734d8a640 1 -- 172.21.15.78:0/901709503 learned_addr learned my addr 172.21.15.78:0/901709503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.462+0000 7f373558b640 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c87410 0x5609ca0284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f372ffff640 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca02f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f372dffb640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f37200afb70 0x7f37200b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f3734d8a640 1 -- 172.21.15.78:0/901709503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 msgr2=0x5609ca02f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f3734d8a640 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca02f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f3734d8a640 1 -- 172.21.15.78:0/901709503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c87410 msgr2=0x5609ca0284a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f3734d8a640 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c87410 0x5609ca0284a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f3734d8a640 1 -- 172.21.15.78:0/901709503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c9d677e0 con 0x5609c9c5e550 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f372ffff640 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca02f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f373558b640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f37200afb70 0x7f37200b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f372dffb640 1 -- 172.21.15.78:0/1023653107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f371c0ff1a0 con 0x7f3730102250 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f3734d8a640 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca028a20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3724009c60 tx=0x7f3724011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6f8a0b8be59ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f371affd640 1 -- 172.21.15.78:0/901709503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724004220 con 0x5609c9c5e550 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f371affd640 1 -- 172.21.15.78:0/901709503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37240043c0 con 0x5609c9c5e550 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.463+0000 7f371affd640 1 -- 172.21.15.78:0/901709503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724019830 con 0x5609c9c5e550 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 -- 172.21.15.78:0/901709503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 msgr2=0x5609ca028a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca028a20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3724009c60 tx=0x7f3724011810 comp rx=0 tx=0).stop 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373558b640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f37200afb70 0x7f37200b2080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5609ca029670 tx=0x7f3728004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 -- 172.21.15.78:0/901709503 shutdown_connections 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca02f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca028a20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 --2- 172.21.15.78:0/901709503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c87410 0x5609ca0284a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 -- 172.21.15.78:0/901709503 >> 172.21.15.78:0/901709503 conn(0x5609c9c7b560 msgr2=0x5609ca023e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 -- 172.21.15.78:0/901709503 shutdown_connections 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 -- 172.21.15.78:0/901709503 wait complete. 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 -- start start 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.464+0000 7f373df77740 1 Processor -- start 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373df77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c5e550 0x5609ca032240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373df77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca032780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373df77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c87410 0x5609ca0396e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373df77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609ca030370 con 0x5609c9c87410 2024-04-25T14:03:36.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373df77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609c9f275d0 con 0x5609c9c5e550 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373df77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5609ca0304f0 con 0x5609c9c66730 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373df77740 10 client.0 _ll_register_callbacks cb 0x5609c9f15080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c87410 0x5609ca0396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c87410 0x5609ca0396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44252/0 (socket says 172.21.15.78:44252) 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 -- 172.21.15.78:0/1541397390 learned_addr learned my addr 172.21.15.78:0/1541397390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f372ffff640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca032780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f3734d8a640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c5e550 0x5609ca032240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 -- 172.21.15.78:0/1541397390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 msgr2=0x5609ca032780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca032780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 -- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c5e550 msgr2=0x5609ca032240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c5e550 0x5609ca032240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 -- 172.21.15.78:0/1541397390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c9dc7f70 con 0x5609c9c87410 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f372ffff640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca032780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f3734d8a640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c5e550 0x5609ca032240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f373558b640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c87410 0x5609ca0396e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3728013970 tx=0x7f3728004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e76fc08b167d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.465+0000 7f370ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f370ffff640 1 -- 172.21.15.78:0/1541397390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728009cd0 con 0x5609c9c87410 2024-04-25T14:03:36.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f370ffff640 1 -- 172.21.15.78:0/1541397390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f372801ed70 con 0x5609c9c87410 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37240043c0 con 0x5609c9c87410 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f370ffff640 1 -- 172.21.15.78:0/1541397390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3728004b20 con 0x5609c9c87410 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 10 client.16662 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5609c9c5f170 con 0x5609c9c87410 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f370ffff640 1 -- 172.21.15.78:0/1541397390 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37240043c0 con 0x5609c9c87410 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f370ffff640 1 -- 172.21.15.78:0/1541397390 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f372800f040 con 0x5609c9c87410 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 10 client.16662 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 20 client.16662 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c461f0 0x5609ca03fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 4 client.16662 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:36.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c9f9b710 con 0x5609c9c461f0 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.466+0000 7f3734d8a640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c461f0 0x5609ca03fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.467+0000 7f370ffff640 10 client.16662 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.467+0000 7f3734d8a640 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c461f0 0x5609ca03fc80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c34584eb052b4e4 server_cookie=e339d82ec85c0427 in_seq=0 out_seq=0 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f370ffff640 1 -- 172.21.15.78:0/1541397390 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609c9f9b710 con 0x5609c9c461f0 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f370ffff640 10 client.16662 handle_command_reply: tid=0 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 client.16662 shutdown 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c87410 msgr2=0x5609ca0396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c87410 0x5609ca0396e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3728013970 tx=0x7f3728004ab0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 shutdown_connections 2024-04-25T14:03:36.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c87410 0x5609ca0396e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c461f0 0x5609ca03fc80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c66730 0x5609ca032780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 --2- 172.21.15.78:0/1541397390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c5e550 0x5609ca032240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f370ffff640 0 client.16662 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 >> 172.21.15.78:0/1541397390 conn(0x5609c9c7b560 msgr2=0x5609ca022ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 shutdown_connections 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 1 -- 172.21.15.78:0/1541397390 wait complete. 2024-04-25T14:03:36.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.468+0000 7f373df77740 20 client.16662 trim_cache size 0 max 16384 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 Processor -- start 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- start start 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca030f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca039600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 0x5609ca039b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37301adae0 con 0x5609c9c5e550 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37301ab3b0 con 0x5609c9c66730 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37301a9200 con 0x5609c9c7b560 2024-04-25T14:03:36.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca039600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca039600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44256/0 (socket says 172.21.15.78:44256) 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 -- 172.21.15.78:0/1345025452 learned_addr learned my addr 172.21.15.78:0/1345025452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f3734d8a640 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca030f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 -- 172.21.15.78:0/1345025452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 msgr2=0x5609ca030f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca030f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 -- 172.21.15.78:0/1345025452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 msgr2=0x5609ca039b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 0x5609ca039b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 -- 172.21.15.78:0/1345025452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ca02d0c0 con 0x5609c9c5e550 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f3734d8a640 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca030f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f372ffff640 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca039600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f371c031440 tx=0x7f371c03b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb204907fcb7f7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f371a7fc640 1 -- 172.21.15.78:0/1345025452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c049040 con 0x5609c9c5e550 2024-04-25T14:03:36.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f371a7fc640 1 -- 172.21.15.78:0/1345025452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371c03b460 con 0x5609c9c5e550 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f371a7fc640 1 -- 172.21.15.78:0/1345025452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0cba20 con 0x5609c9c5e550 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- 172.21.15.78:0/1345025452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 msgr2=0x5609ca039600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca039600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f371c031440 tx=0x7f371c03b640 comp rx=0 tx=0).stop 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- 172.21.15.78:0/1345025452 shutdown_connections 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 0x5609ca039b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca039600 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 --2- 172.21.15.78:0/1345025452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca030f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- 172.21.15.78:0/1345025452 >> 172.21.15.78:0/1345025452 conn(0x5609ca032a10 msgr2=0x5609ca0237f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- 172.21.15.78:0/1345025452 shutdown_connections 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.487+0000 7f373df77740 1 -- 172.21.15.78:0/1345025452 wait complete. 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 -- start start 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 Processor -- start 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca037910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 0x5609ca037e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca038d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37301a9870 con 0x5609c9c5e550 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37301aace0 con 0x5609c9c66730 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37301a9f40 con 0x5609c9c7b560 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373df77740 10 client.0 _ll_register_callbacks cb 0x7f3728013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f3734d8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f3734d8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca037910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44264/0 (socket says 172.21.15.78:44264) 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f3734d8a640 1 -- 172.21.15.78:0/3666445344 learned_addr learned my addr 172.21.15.78:0/3666445344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f373558b640 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca038d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f372ffff640 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 0x5609ca037e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f3734d8a640 1 -- 172.21.15.78:0/3666445344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 msgr2=0x5609ca038d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.488+0000 7f3734d8a640 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca038d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f3734d8a640 1 -- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 msgr2=0x5609ca037e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f3734d8a640 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 0x5609ca037e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f3734d8a640 1 -- 172.21.15.78:0/3666445344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ca0392d0 con 0x5609c9c5e550 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f3734d8a640 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca037910 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3724011d10 tx=0x7f3724009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b27cb2b7678bd0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f370ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f370ffff640 1 -- 172.21.15.78:0/3666445344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724005250 con 0x5609c9c5e550 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f370ffff640 1 -- 172.21.15.78:0/3666445344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f37240053f0 con 0x5609c9c5e550 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609c9f95580 con 0x5609c9c5e550 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f370ffff640 1 -- 172.21.15.78:0/3666445344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f37240055f0 con 0x5609c9c5e550 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 10 client.16668 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5609ca0a8180 con 0x5609c9c5e550 2024-04-25T14:03:36.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f370ffff640 1 -- 172.21.15.78:0/3666445344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5609c9f95580 con 0x5609c9c5e550 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f370ffff640 1 -- 172.21.15.78:0/3666445344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3724019660 con 0x5609c9c5e550 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 10 client.16668 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 20 client.16668 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c87410 0x5609ca0aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 4 client.16668 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.489+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5609c9f10ac0 con 0x5609c9c87410 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.490+0000 7f372ffff640 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c87410 0x5609ca0aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.490+0000 7f372ffff640 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c87410 0x5609ca0aa9e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c31ae0092a3714 server_cookie=b228a5c0cc9470ba in_seq=0 out_seq=0 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.490+0000 7f370ffff640 10 client.16668 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f370ffff640 1 -- 172.21.15.78:0/3666445344 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5609c9f10ac0 con 0x5609c9c87410 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f370ffff640 10 client.16668 handle_command_reply: tid=0 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 client.16668 shutdown 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 msgr2=0x5609ca037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca037910 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3724011d10 tx=0x7f3724009bb0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 shutdown_connections 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609c9c87410 0x5609ca0aa9e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609c9c7b560 0x5609ca038d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609c9c66730 0x5609ca037e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 --2- 172.21.15.78:0/3666445344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609c9c5e550 0x5609ca037910 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 >> 172.21.15.78:0/3666445344 conn(0x5609ca032a10 msgr2=0x5609ca044430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f370ffff640 0 client.16668 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 shutdown_connections 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.491+0000 7f373df77740 1 -- 172.21.15.78:0/3666445344 wait complete. 2024-04-25T14:03:36.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f373df77740 20 client.16668 trim_cache size 0 max 16384 2024-04-25T14:03:36.499 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:03:36.499 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:03:36.499 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "8a7123b0-0011-480c-b8c1-9a7f163f1a99", 2024-04-25T14:03:36.499 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:03:36.499 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f37200afb70 msgr2=0x7f37200b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f37200afb70 0x7f37200b2080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5609ca029670 tx=0x7f3728004a40 comp rx=0 tx=0).stop 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730102250 msgr2=0x7f373019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730102250 0x7f373019a770 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f371c006010 tx=0x7f371c031a50 comp rx=0 tx=0).stop 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 shutdown_connections 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37301050a0 0x7f37301a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f37200afb70 0x7f37200b2080 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3730102250 0x7f373019a770 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 --2- 172.21.15.78:0/1023653107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730101750 0x7f373019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 >> 172.21.15.78:0/1023653107 conn(0x7f37300fd350 msgr2=0x7f3730105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 shutdown_connections 2024-04-25T14:03:36.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.492+0000 7f3736fee640 1 -- 172.21.15.78:0/1023653107 wait complete. 2024-04-25T14:03:36.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:03:36.503 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:03:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1944565174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb054106f50 msgr2=0x7fb054109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1944565174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb054106f50 0x7fb054109370 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb048009900 tx=0x7fb04802a410 comp rx=0 tx=0).stop 2024-04-25T14:03:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1944565174 shutdown_connections 2024-04-25T14:03:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1944565174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0541098b0 0x7fb05410bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1944565174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb054106f50 0x7fb054109370 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1944565174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0541045f0 0x7fb054106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1944565174 >> 172.21.15.78:0/1944565174 conn(0x7fb0540fdbf0 msgr2=0x7fb0541000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1944565174 shutdown_connections 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1944565174 wait complete. 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 Processor -- start 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.613+0000 7fb05b1ee640 1 -- start start 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05b1ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0541045f0 0x7fb05419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05b1ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb054106f50 0x7fb05419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05b1ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0541098b0 0x7fb0541a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05b1ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb05410f630 con 0x7fb0541098b0 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05b1ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb05410f4b0 con 0x7fb054106f50 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05b1ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb05410f7b0 con 0x7fb0541045f0 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0541098b0 0x7fb0541a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0541098b0 0x7fb0541a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48252/0 (socket says 172.21.15.78:48252) 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 -- 172.21.15.78:0/1845664582 learned_addr learned my addr 172.21.15.78:0/1845664582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb053fff640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb054106f50 0x7fb05419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb058f8a640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0541045f0 0x7fb05419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 -- 172.21.15.78:0/1845664582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0541045f0 msgr2=0x7fb05419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0541045f0 0x7fb05419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 -- 172.21.15.78:0/1845664582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb054106f50 msgr2=0x7fb05419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb054106f50 0x7fb05419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 -- 172.21.15.78:0/1845664582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0541a1da0 con 0x7fb0541098b0 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb053fff640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb054106f50 0x7fb05419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05978b640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0541098b0 0x7fb0541a1730 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00cad0 tx=0x7fb04c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb051ffb640 1 -- 172.21.15.78:0/1845664582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00ae20 con 0x7fb0541098b0 2024-04-25T14:03:36.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb051ffb640 1 -- 172.21.15.78:0/1845664582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb04c003e40 con 0x7fb0541098b0 2024-04-25T14:03:36.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0541a2090 con 0x7fb0541098b0 2024-04-25T14:03:36.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.614+0000 7fb051ffb640 1 -- 172.21.15.78:0/1845664582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00f6b0 con 0x7fb0541098b0 2024-04-25T14:03:36.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.615+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541a25e0 con 0x7fb0541098b0 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 Processor -- start 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb051ffb640 1 -- 172.21.15.78:0/1845664582 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb04c0042c0 con 0x7fb0541098b0 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 -- start start 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a21fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a22c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558744986d80 con 0x558744680390 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587449408b0 con 0x5587446574d0 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb062175740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558744982d40 con 0x55874465f6b0 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a21a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48266/0 (socket says 172.21.15.78:48266) 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 -- 172.21.15.78:0/3700862672 learned_addr learned my addr 172.21.15.78:0/3700862672 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a22c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb058f8a640 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a21fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 -- 172.21.15.78:0/3700862672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 msgr2=0x558744a22c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a22c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 -- 172.21.15.78:0/3700862672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 msgr2=0x558744a21fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a21fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb053fff640 1 -- 172.21.15.78:0/3700862672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558744911a40 con 0x558744680390 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.618+0000 7fb051ffb640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb03c0afaf0 0x7fb03c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.619+0000 7fb053fff640 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a21a60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb048031440 tx=0x7fb0480058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5009d3ce2db65f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.619+0000 7fb058f8a640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb03c0afaf0 0x7fb03c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.619+0000 7fb051ffb640 1 -- 172.21.15.78:0/1845664582 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb04c0df030 con 0x7fb0541098b0 2024-04-25T14:03:36.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.619+0000 7fb03affd640 1 -- 172.21.15.78:0/3700862672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048043020 con 0x558744680390 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.619+0000 7fb03affd640 1 -- 172.21.15.78:0/3700862672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb048005040 con 0x558744680390 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.619+0000 7fb03affd640 1 -- 172.21.15.78:0/3700862672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048035050 con 0x558744680390 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 -- 172.21.15.78:0/3700862672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 msgr2=0x558744a21a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb058f8a640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb03c0afaf0 0x7fb03c0b2000 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb044005fd0 tx=0x7fb044005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a21a60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb048031440 tx=0x7fb0480058b0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 -- 172.21.15.78:0/3700862672 shutdown_connections 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a22c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a21fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 --2- 172.21.15.78:0/3700862672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a21a60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 -- 172.21.15.78:0/3700862672 >> 172.21.15.78:0/3700862672 conn(0x5587446744e0 msgr2=0x558744a1d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 -- 172.21.15.78:0/3700862672 shutdown_connections 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 -- 172.21.15.78:0/3700862672 wait complete. 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 -- start start 2024-04-25T14:03:36.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.620+0000 7fb062175740 1 Processor -- start 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a2b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a2be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558744680390 0x558744a32da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558744a23720 con 0x5587446574d0 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55874490dd10 con 0x558744680390 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558744a238a0 con 0x55874465f6b0 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 10 client.0 _ll_register_callbacks cb 0x55874493a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a2b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a2b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48268/0 (socket says 172.21.15.78:48268) 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 -- 172.21.15.78:0/2930183525 learned_addr learned my addr 172.21.15.78:0/2930183525 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb05978b640 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558744680390 0x558744a32da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb053fff640 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a2be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 -- 172.21.15.78:0/2930183525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 msgr2=0x558744a2be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a2be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 -- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558744680390 msgr2=0x558744a32da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558744680390 0x558744a32da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 -- 172.21.15.78:0/2930183525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587447627d0 con 0x5587446574d0 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb058f8a640 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a2b900 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb044010f80 tx=0x7fb044004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2688e62031559013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb033fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb033fff640 1 -- 172.21.15.78:0/2930183525 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044023070 con 0x5587446574d0 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb033fff640 1 -- 172.21.15.78:0/2930183525 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb04401f860 con 0x5587446574d0 2024-04-25T14:03:36.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.621+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb048005040 con 0x5587446574d0 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb033fff640 1 -- 172.21.15.78:0/2930183525 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb04402f050 con 0x5587446574d0 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 10 client.16680 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5587446983f0 con 0x5587446574d0 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb033fff640 1 -- 172.21.15.78:0/2930183525 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb048005040 con 0x5587446574d0 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb033fff640 1 -- 172.21.15.78:0/2930183525 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb04400f040 con 0x5587446574d0 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 10 client.16680 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 20 client.16680 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55874463ef60 0x558744a39340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 4 client.16680 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558744994990 con 0x55874463ef60 2024-04-25T14:03:36.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.622+0000 7fb053fff640 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55874463ef60 0x558744a39340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb033fff640 10 client.16680 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb053fff640 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55874463ef60 0x558744a39340 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b7ca00097fc17f server_cookie=fa4602fbabc7a3b3 in_seq=0 out_seq=0 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb033fff640 1 -- 172.21.15.78:0/2930183525 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558744994990 con 0x55874463ef60 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb033fff640 10 client.16680 handle_command_reply: tid=0 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb062175740 1 client.16680 shutdown 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 msgr2=0x558744a2b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb062175740 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a2b900 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb044010f80 tx=0x7fb044004730 comp rx=0 tx=0).stop 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 shutdown_connections 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.623+0000 7fb062175740 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558744680390 0x558744a32da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb062175740 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55874463ef60 0x558744a39340 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb062175740 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a2be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb062175740 1 --2- 172.21.15.78:0/2930183525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a2b900 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb033fff640 0 client.16680 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 >> 172.21.15.78:0/2930183525 conn(0x5587446744e0 msgr2=0x558744a20180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 shutdown_connections 2024-04-25T14:03:36.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb062175740 1 -- 172.21.15.78:0/2930183525 wait complete. 2024-04-25T14:03:36.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.624+0000 7fb062175740 20 client.16680 trim_cache size 0 max 16384 2024-04-25T14:03:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 Processor -- start 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 -- start start 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a2a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a2bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541adb10 con 0x558744680390 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541ab3e0 con 0x5587446574d0 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb062175740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541a9230 con 0x55874465f6b0 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb05978b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a2bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb05978b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a2bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48280/0 (socket says 172.21.15.78:48280) 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.641+0000 7fb05978b640 1 -- 172.21.15.78:0/4218884682 learned_addr learned my addr 172.21.15.78:0/4218884682 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb053fff640 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb058f8a640 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a2a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb05978b640 1 -- 172.21.15.78:0/4218884682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 msgr2=0x558744a268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb05978b640 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb05978b640 1 -- 172.21.15.78:0/4218884682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 msgr2=0x558744a2a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb05978b640 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a2a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb05978b640 1 -- 172.21.15.78:0/4218884682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587446580f0 con 0x558744680390 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb05978b640 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a2bbe0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0dce90 tx=0x7fb04c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77034bfd96d87219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb03a7fc640 1 -- 172.21.15.78:0/4218884682 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c01d040 con 0x558744680390 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb03a7fc640 1 -- 172.21.15.78:0/4218884682 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb04c0a7500 con 0x558744680390 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.642+0000 7fb03a7fc640 1 -- 172.21.15.78:0/4218884682 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0a79c0 con 0x558744680390 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 -- 172.21.15.78:0/4218884682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 msgr2=0x558744a2bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a2bbe0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0dce90 tx=0x7fb04c005f90 comp rx=0 tx=0).stop 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 -- 172.21.15.78:0/4218884682 shutdown_connections 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558744680390 0x558744a2bbe0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55874465f6b0 0x558744a268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 --2- 172.21.15.78:0/4218884682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587446574d0 0x558744a2a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 -- 172.21.15.78:0/4218884682 >> 172.21.15.78:0/4218884682 conn(0x5587446744e0 msgr2=0x558744a2b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 -- 172.21.15.78:0/4218884682 shutdown_connections 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 -- 172.21.15.78:0/4218884682 wait complete. 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 -- start start 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 Processor -- start 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.643+0000 7fb062175740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55874465f6b0 0x558744a309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558744680390 0x558744a9f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541a98a0 con 0x5587446574d0 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541aad10 con 0x55874465f6b0 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541a9f70 con 0x558744680390 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 10 client.0 _ll_register_callbacks cb 0x7fb044016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a304b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48284/0 (socket says 172.21.15.78:48284) 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 -- 172.21.15.78:0/2388475276 learned_addr learned my addr 172.21.15.78:0/2388475276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb05978b640 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558744680390 0x558744a9f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb053fff640 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55874465f6b0 0x558744a309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 -- 172.21.15.78:0/2388475276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558744680390 msgr2=0x558744a9f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558744680390 0x558744a9f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 -- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55874465f6b0 msgr2=0x558744a309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55874465f6b0 0x558744a309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 -- 172.21.15.78:0/2388475276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587449474b0 con 0x5587446574d0 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb058f8a640 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a304b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb044000c00 tx=0x7fb0440274a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f642263708c95c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb033fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb033fff640 1 -- 172.21.15.78:0/2388475276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044023040 con 0x5587446574d0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb033fff640 1 -- 172.21.15.78:0/2388475276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb04401fa70 con 0x5587446574d0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04c0a7500 con 0x5587446574d0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb033fff640 1 -- 172.21.15.78:0/2388475276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb04401fc70 con 0x5587446574d0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 10 client.16686 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558744aa0cb0 con 0x5587446574d0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.644+0000 7fb033fff640 1 -- 172.21.15.78:0/2388475276 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb04c0a7500 con 0x5587446574d0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb033fff640 1 -- 172.21.15.78:0/2388475276 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb0440110d0 con 0x5587446574d0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb062175740 10 client.16686 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb062175740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb062175740 20 client.16686 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb062175740 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587446143a0 0x558744aa36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb062175740 4 client.16686 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55874498e800 con 0x5587446143a0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb053fff640 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587446143a0 0x558744aa36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb033fff640 10 client.16686 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.645+0000 7fb053fff640 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587446143a0 0x558744aa36d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f4fc5617f4be35 server_cookie=2253807726e0ade8 in_seq=0 out_seq=0 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb033fff640 1 -- 172.21.15.78:0/2388475276 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55874498e800 con 0x5587446143a0 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb033fff640 10 client.16686 handle_command_reply: tid=0 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 client.16686 shutdown 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 msgr2=0x558744a304b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a304b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb044000c00 tx=0x7fb0440274a0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 shutdown_connections 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587446143a0 0x558744aa36d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558744680390 0x558744a9f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55874465f6b0 0x558744a309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 --2- 172.21.15.78:0/2388475276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587446574d0 0x558744a304b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb033fff640 0 client.16686 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 >> 172.21.15.78:0/2388475276 conn(0x5587446744e0 msgr2=0x558744a1b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 shutdown_connections 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 1 -- 172.21.15.78:0/2388475276 wait complete. 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.646+0000 7fb062175740 20 client.16686 trim_cache size 0 max 16384 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stdout: "8a7123b0-0011-480c-b8c1-9a7f163f1a99": { 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:03:36.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "8a7123b0-0011-480c-b8c1-9a7f163f1a99", 2024-04-25T14:03:36.654 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:03:36.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:03:36.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:03:36.654 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:03:36.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb03c0afaf0 msgr2=0x7fb03c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb03c0afaf0 0x7fb03c0b2000 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb044005fd0 tx=0x7fb044005850 comp rx=0 tx=0).stop 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0541098b0 msgr2=0x7fb0541a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0541098b0 0x7fb0541a1730 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00cad0 tx=0x7fb04c00cf90 comp rx=0 tx=0).stop 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 shutdown_connections 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb03c0afaf0 0x7fb03c0b2000 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0541098b0 0x7fb0541a1730 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb054106f50 0x7fb05419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 --2- 172.21.15.78:0/1845664582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0541045f0 0x7fb05419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 >> 172.21.15.78:0/1845664582 conn(0x7fb0540fdbf0 msgr2=0x7fb054105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 shutdown_connections 2024-04-25T14:03:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:03:36.647+0000 7fb05b1ee640 1 -- 172.21.15.78:0/1845664582 wait complete. 2024-04-25T14:03:36.656 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:36 smithi078 ceph-mon[29163]: pgmap v471: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:03:36.657 INFO:tasks.cephfs.filesystem:scrub status for tag:8a7123b0-0011-480c-b8c1-9a7f163f1a99 - {'path': '/', 'tag': '8a7123b0-0011-480c-b8c1-9a7f163f1a99', 'options': 'recursive,force'} 2024-04-25T14:03:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:36 smithi129 ceph-mon[32569]: pgmap v471: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:03:36.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:36 smithi112 ceph-mon[32617]: pgmap v471: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:03:37.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:37 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:03:37.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:37 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:37.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:37 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:03:37.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:37 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:37 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:03:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:37 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:37 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:03:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:37 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:37.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:37 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:03:37.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:37 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:37.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:37 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:03:37.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:37 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:03:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:03:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:03:38.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:03:38.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:38 smithi129 ceph-mon[32569]: pgmap v472: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:03:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:03:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:03:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:38 smithi078 ceph-mon[29163]: pgmap v472: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:03:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:03:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:03:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:03:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:38 smithi112 ceph-mon[32617]: pgmap v472: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 -- 172.21.15.78:0/1203916610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681017a0 msgr2=0x7f3b68101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1203916610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681017a0 0x7f3b68101c10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009900 tx=0x7f3b5002a410 comp rx=0 tx=0).stop 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 -- 172.21.15.78:0/1203916610 shutdown_connections 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1203916610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b681022a0 0x7f3b6810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1203916610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681017a0 0x7f3b68101c10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1203916610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b681050f0 0x7f3b68105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 -- 172.21.15.78:0/1203916610 >> 172.21.15.78:0/1203916610 conn(0x7f3b680fd3c0 msgr2=0x7f3b680ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 -- 172.21.15.78:0/1203916610 shutdown_connections 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 -- 172.21.15.78:0/1203916610 wait complete. 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.563+0000 7f3b6d500640 1 Processor -- start 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b6d500640 1 -- start start 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b6d500640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b681017a0 0x7f3b6819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b6d500640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681022a0 0x7f3b6819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b6d500640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b681050f0 0x7f3b681a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b6d500640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810f680 con 0x7f3b681022a0 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b6d500640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810f500 con 0x7f3b681050f0 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b6d500640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810f800 con 0x7f3b681017a0 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681022a0 0x7f3b6819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681022a0 0x7f3b6819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48300/0 (socket says 172.21.15.78:48300) 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/1038934375 learned_addr learned my addr 172.21.15.78:0/1038934375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b681017a0 0x7f3b6819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/1038934375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b681017a0 msgr2=0x7f3b6819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b677fe640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b681050f0 0x7f3b681a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b681017a0 0x7f3b6819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/1038934375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b681050f0 msgr2=0x7f3b681a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b681050f0 0x7f3b681a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/1038934375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b681a1d70 con 0x7f3b681022a0 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b681017a0 0x7f3b6819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b677fe640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b681050f0 0x7f3b681a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.564+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681022a0 0x7f3b6819a770 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006010 tx=0x7f3b50031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.565+0000 7f3b64ff9640 1 -- 172.21.15.78:0/1038934375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5003f030 con 0x7f3b681022a0 2024-04-25T14:03:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.565+0000 7f3b64ff9640 1 -- 172.21.15.78:0/1038934375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b50003aa0 con 0x7f3b681022a0 2024-04-25T14:03:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.565+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b681a2000 con 0x7f3b681022a0 2024-04-25T14:03:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.565+0000 7f3b64ff9640 1 -- 172.21.15.78:0/1038934375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50035070 con 0x7f3b681022a0 2024-04-25T14:03:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.565+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a24f0 con 0x7f3b681022a0 2024-04-25T14:03:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.567+0000 7f3b64ff9640 1 -- 172.21.15.78:0/1038934375 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3b50042030 con 0x7f3b681022a0 2024-04-25T14:03:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.567+0000 7f3b743b1740 1 Processor -- start 2024-04-25T14:03:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.567+0000 7f3b743b1740 1 -- start start 2024-04-25T14:03:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f02f0 0x55c5feab9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f84d0 0x55c5feac0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b743b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5fea1f460 con 0x55c5fe6f84d0 2024-04-25T14:03:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b743b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5fe9d9590 con 0x55c5fe7191b0 2024-04-25T14:03:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b743b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c5fea1b420 con 0x55c5fe6f02f0 2024-04-25T14:03:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b5e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f84d0 0x55c5feac0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b5e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f84d0 0x55c5feac0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48308/0 (socket says 172.21.15.78:48308) 2024-04-25T14:03:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/2307979247 learned_addr learned my addr 172.21.15.78:0/2307979247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f02f0 0x55c5feab9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b64ff9640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3b540afae0 0x7f3b540b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.568+0000 7f3b677fe640 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/2307979247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f02f0 msgr2=0x55c5feab9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f02f0 0x55c5feab9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/2307979247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 msgr2=0x55c5feab9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5e7fc640 1 -- 172.21.15.78:0/2307979247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5fe9e60d0 con 0x55c5fe6f84d0 2024-04-25T14:03:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b677fe640 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b677fe640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3b540afae0 0x7f3b540b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b64ff9640 1 -- 172.21.15.78:0/1038934375 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3b500ff360 con 0x7f3b681022a0 2024-04-25T14:03:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f84d0 0x55c5feac0df0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3b50040c70 tx=0x7f3b5004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31beb9db7a3dc246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5d7fa640 1 -- 172.21.15.78:0/2307979247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b500c7120 con 0x55c5fe6f84d0 2024-04-25T14:03:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b677fe640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3b540afae0 0x7f3b540b1ff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009810 tx=0x7f3b60004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5d7fa640 1 -- 172.21.15.78:0/2307979247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b500c72e0 con 0x55c5fe6f84d0 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.569+0000 7f3b5d7fa640 1 -- 172.21.15.78:0/2307979247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b500c6f40 con 0x55c5fe6f84d0 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 -- 172.21.15.78:0/2307979247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f84d0 msgr2=0x55c5feac0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f84d0 0x55c5feac0df0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3b50040c70 tx=0x7f3b5004b000 comp rx=0 tx=0).stop 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 -- 172.21.15.78:0/2307979247 shutdown_connections 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f84d0 0x55c5feac0df0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f02f0 0x55c5feab9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2307979247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 -- 172.21.15.78:0/2307979247 >> 172.21.15.78:0/2307979247 conn(0x55c5fe70d300 msgr2=0x55c5feab5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 -- 172.21.15.78:0/2307979247 shutdown_connections 2024-04-25T14:03:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 -- 172.21.15.78:0/2307979247 wait complete. 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.570+0000 7f3b743b1740 1 -- start start 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 1 Processor -- start 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe6f02f0 0x55c5feac3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe7191b0 0x55c5feaca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5feac1600 con 0x55c5fe7191b0 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5fe9a6b60 con 0x55c5fe6f02f0 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c5feac1780 con 0x55c5fe6f84d0 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b743b1740 10 client.0 _ll_register_callbacks cb 0x55c5fe9cb2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe7191b0 0x55c5feaca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe7191b0 0x55c5feaca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48322/0 (socket says 172.21.15.78:48322) 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 -- 172.21.15.78:0/2228389087 learned_addr learned my addr 172.21.15.78:0/2228389087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe6f02f0 0x55c5feac3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 -- 172.21.15.78:0/2228389087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 msgr2=0x55c5feac3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 -- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe6f02f0 msgr2=0x55c5feac3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe6f02f0 0x55c5feac3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b677fe640 1 -- 172.21.15.78:0/2228389087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5fe731210 con 0x55c5fe7191b0 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.571+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe6f02f0 0x55c5feac3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b677fe640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe7191b0 0x55c5feaca7e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009fd0 tx=0x7f3b60009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a12c36e8370629d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/2228389087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6001bb30 con 0x55c5fe7191b0 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/2228389087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3b6001bcd0 con 0x55c5fe7191b0 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b500c72e0 con 0x55c5fe7191b0 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/2228389087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3b6001c400 con 0x55c5fe7191b0 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 10 client.16698 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c5fe7fd620 con 0x55c5fe7191b0 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/2228389087 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b500c72e0 con 0x55c5fe7191b0 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/2228389087 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3b60018030 con 0x55c5fe7191b0 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 10 client.16698 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 20 client.16698 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6d7d80 0x55c5fead10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 4 client.16698 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.572+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5fea2d070 con 0x55c5fe6d7d80 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.573+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6d7d80 0x55c5fead10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.573+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6d7d80 0x55c5fead10a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbcbcc64d2ae448 server_cookie=a2d067c9355c8f4c in_seq=0 out_seq=0 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.573+0000 7f3b4e7fc640 10 client.16698 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/2228389087 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c5fea2d070 con 0x55c5fe6d7d80 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b4e7fc640 10 client.16698 handle_command_reply: tid=0 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 client.16698 shutdown 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe7191b0 msgr2=0x55c5feaca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe7191b0 0x55c5feaca7e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009fd0 tx=0x7f3b60009500 comp rx=0 tx=0).stop 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 shutdown_connections 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe7191b0 0x55c5feaca7e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6d7d80 0x55c5fead10a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 --2- 172.21.15.78:0/2228389087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe6f02f0 0x55c5feac3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b4e7fc640 0 client.16698 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 >> 172.21.15.78:0/2228389087 conn(0x55c5fe70d300 msgr2=0x55c5feab8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 shutdown_connections 2024-04-25T14:03:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 1 -- 172.21.15.78:0/2228389087 wait complete. 2024-04-25T14:03:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.574+0000 7f3b743b1740 20 client.16698 trim_cache size 0 max 16384 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.588+0000 7f3b743b1740 1 Processor -- start 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.588+0000 7f3b743b1740 1 -- start start 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feab4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feaca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b743b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681adbd0 con 0x55c5fe6f02f0 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b743b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681ab4a0 con 0x55c5fe7191b0 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b743b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681a92f0 con 0x55c5fe6f84d0 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feab4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feab4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48334/0 (socket says 172.21.15.78:48334) 2024-04-25T14:03:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 -- 172.21.15.78:0/1175361366 learned_addr learned my addr 172.21.15.78:0/1175361366 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feaca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b677fe640 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 -- 172.21.15.78:0/1175361366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 msgr2=0x55c5feaca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feaca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 -- 172.21.15.78:0/1175361366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 msgr2=0x55c5feab5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 -- 172.21.15.78:0/1175361366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5feacac40 con 0x55c5fe6f02f0 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b677fe640 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feab4c50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000c00 tx=0x7f3b580094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0186474a2aeac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b5cff9640 1 -- 172.21.15.78:0/1175361366 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58005220 con 0x55c5fe6f02f0 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b5cff9640 1 -- 172.21.15.78:0/1175361366 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b5801e030 con 0x55c5fe6f02f0 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.589+0000 7f3b5cff9640 1 -- 172.21.15.78:0/1175361366 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5801a6b0 con 0x55c5fe6f02f0 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 -- 172.21.15.78:0/1175361366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 msgr2=0x55c5feab4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feab4c50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000c00 tx=0x7f3b580094c0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 -- 172.21.15.78:0/1175361366 shutdown_connections 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feab5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feaca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 --2- 172.21.15.78:0/1175361366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feab4c50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 -- 172.21.15.78:0/1175361366 >> 172.21.15.78:0/1175361366 conn(0x55c5fe70d300 msgr2=0x55c5feabdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 -- 172.21.15.78:0/1175361366 shutdown_connections 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.590+0000 7f3b743b1740 1 -- 172.21.15.78:0/1175361366 wait complete. 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 -- start start 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 Processor -- start 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feac7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feb37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681a9960 con 0x55c5fe6f02f0 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681aadd0 con 0x55c5fe7191b0 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681aa030 con 0x55c5fe6f84d0 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b743b1740 10 client.0 _ll_register_callbacks cb 0x7f3b60009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feac7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feac7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48350/0 (socket says 172.21.15.78:48350) 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 -- 172.21.15.78:0/4024379554 learned_addr learned my addr 172.21.15.78:0/4024379554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 -- 172.21.15.78:0/4024379554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 msgr2=0x55c5feac7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 -- 172.21.15.78:0/4024379554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 msgr2=0x55c5feb37440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feb37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 -- 172.21.15.78:0/4024379554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5feac8120 con 0x55c5fe6f02f0 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b66ffd640 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feac7570 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3b580048b0 tx=0x7f3b580048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8f8ddd0d17c8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.591+0000 7f3b4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/4024379554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58031060 con 0x55c5fe6f02f0 2024-04-25T14:03:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/4024379554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3b58004b00 con 0x55c5fe6f02f0 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b5801e030 con 0x55c5fe6f02f0 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/4024379554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3b58005020 con 0x55c5fe6f02f0 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 10 client.16704 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c5feb38a80 con 0x55c5fe6f02f0 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/4024379554 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b5801e030 con 0x55c5fe6f02f0 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/4024379554 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3b5800f040 con 0x55c5fe6f02f0 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 10 client.16704 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 20 client.16704 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6ad1c0 0x55c5feb3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 4 client.16704 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.592+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5fea26ee0 con 0x55c5fe6ad1c0 2024-04-25T14:03:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.593+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6ad1c0 0x55c5feb3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.593+0000 7f3b4e7fc640 10 client.16704 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.593+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6ad1c0 0x55c5feb3b4d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c8d13fb9214a0f server_cookie=d6f6a8695b7e17fc in_seq=0 out_seq=0 2024-04-25T14:03:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b4e7fc640 1 -- 172.21.15.78:0/4024379554 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5fea26ee0 con 0x55c5fe6ad1c0 2024-04-25T14:03:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b4e7fc640 10 client.16704 handle_command_reply: tid=0 2024-04-25T14:03:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 client.16704 shutdown 2024-04-25T14:03:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 msgr2=0x55c5feac7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feac7570 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3b580048b0 tx=0x7f3b580048e0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 shutdown_connections 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5fe7191b0 0x55c5feb37440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c5fe6ad1c0 0x55c5feb3b4d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c5fe6f84d0 0x55c5feac7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 --2- 172.21.15.78:0/4024379554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5fe6f02f0 0x55c5feac7570 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b4e7fc640 0 client.16704 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 >> 172.21.15.78:0/4024379554 conn(0x55c5fe70d300 msgr2=0x55c5feab8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 shutdown_connections 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 1 -- 172.21.15.78:0/4024379554 wait complete. 2024-04-25T14:03:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.607+0000 7f3b743b1740 20 client.16704 trim_cache size 0 max 16384 2024-04-25T14:03:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:03:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:03:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:03:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3b540afae0 msgr2=0x7f3b540b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3b540afae0 0x7f3b540b1ff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009810 tx=0x7f3b60004a40 comp rx=0 tx=0).stop 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681022a0 msgr2=0x7f3b6819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681022a0 0x7f3b6819a770 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006010 tx=0x7f3b50031a50 comp rx=0 tx=0).stop 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 shutdown_connections 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b681050f0 0x7f3b681a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3b540afae0 0x7f3b540b1ff0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681022a0 0x7f3b6819a770 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 --2- 172.21.15.78:0/1038934375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b681017a0 0x7f3b6819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 >> 172.21.15.78:0/1038934375 conn(0x7f3b680fd3c0 msgr2=0x7f3b68105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 shutdown_connections 2024-04-25T14:03:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:40.608+0000 7f3b6d500640 1 -- 172.21.15.78:0/1038934375 wait complete. 2024-04-25T14:03:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:40 smithi129 ceph-mon[32569]: pgmap v473: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:40 smithi078 ceph-mon[29163]: pgmap v473: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:40 smithi112 ceph-mon[32617]: pgmap v473: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:41 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:03:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:41 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:03:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:41 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:03:42.564 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:42 smithi129 ceph-mon[32569]: pgmap v474: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:03:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:42 smithi078 ceph-mon[29163]: pgmap v474: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:03:42.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:42 smithi112 ceph-mon[32617]: pgmap v474: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:03:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:44 smithi129 ceph-mon[32569]: pgmap v475: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:03:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:44 smithi078 ceph-mon[29163]: pgmap v475: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:03:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:44 smithi112 ceph-mon[32617]: pgmap v475: 97 pgs: 97 active+clean; 2.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:03:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:46 smithi129 ceph-mon[32569]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:03:46.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:46 smithi078 ceph-mon[29163]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:03:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:46 smithi112 ceph-mon[32617]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:03:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:48 smithi129 ceph-mon[32569]: pgmap v477: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:03:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:48 smithi078 ceph-mon[29163]: pgmap v477: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:03:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:48 smithi112 ceph-mon[32617]: pgmap v477: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:03:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:50 smithi129 ceph-mon[32569]: pgmap v478: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:03:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:50 smithi078 ceph-mon[29163]: pgmap v478: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:03:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:50 smithi112 ceph-mon[32617]: pgmap v478: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:127/600: open fd = 3 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:128/600: open fd = 3 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.896 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:129/600: open fd = 3 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:130/600: open fd = 3 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:131/600: open fd = 3 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.897 INFO:tasks.workunit.client.0.smithi078.stdout:132/600: open fd = 3 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:133/600: open fd = 3 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:134/600: open fd = 3 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.898 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.899 INFO:tasks.workunit.client.0.smithi078.stdout:135/600: open fd = 3 2024-04-25T14:03:50.899 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.899 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.899 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.899 INFO:tasks.workunit.client.0.smithi078.stdout:136/600: open fd = 3 2024-04-25T14:03:50.899 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.899 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:137/600: open fd = 3 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:138/600: open fd = 3 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:139/600: open fd = 3 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.900 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:140/600: open fd = 3 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:141/600: open fd = 3 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:142/600: open fd = 3 2024-04-25T14:03:50.901 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.902 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.902 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.902 INFO:tasks.workunit.client.0.smithi078.stdout:143/600: open fd = 3 2024-04-25T14:03:50.902 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.902 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.902 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.903 INFO:tasks.workunit.client.0.smithi078.stdout:144/600: open fd = 3 2024-04-25T14:03:50.903 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.903 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.903 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.903 INFO:tasks.workunit.client.0.smithi078.stdout:145/600: open fd = 3 2024-04-25T14:03:50.903 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.903 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:146/600: open fd = 3 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:147/600: open fd = 3 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:148/600: open fd = 3 2024-04-25T14:03:50.904 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:149/600: open fd = 3 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:150/600: open fd = 3 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:151/600: open fd = 3 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.905 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:152/600: open fd = 3 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:153/600: open fd = 3 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:154/600: open fd = 3 2024-04-25T14:03:50.906 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:155/600: open fd = 3 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:156/600: open fd = 3 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:157/600: open fd = 3 2024-04-25T14:03:50.907 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:158/600: open fd = 3 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:159/600: open fd = 3 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:160/600: open fd = 3 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.908 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:161/600: open fd = 3 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:162/600: open fd = 3 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.909 INFO:tasks.workunit.client.0.smithi078.stdout:163/600: open fd = 3 2024-04-25T14:03:50.910 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.910 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.910 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.910 INFO:tasks.workunit.client.0.smithi078.stdout:164/600: open fd = 3 2024-04-25T14:03:50.910 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.910 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.910 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:165/600: open fd = 3 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:166/600: open fd = 3 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:167/600: open fd = 3 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.911 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:168/600: open fd = 3 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:169/600: open fd = 3 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:170/600: open fd = 3 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:171/600: open fd = 3 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:172/600: open fd = 3 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:173/600: open fd = 3 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:174/600: open fd = 3 2024-04-25T14:03:50.913 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:175/600: open fd = 3 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:176/600: open fd = 3 2024-04-25T14:03:50.914 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:177/600: open fd = 3 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:178/600: open fd = 3 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:179/600: open fd = 3 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.915 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:180/600: open fd = 3 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:181/600: open fd = 3 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:182/600: open fd = 3 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.916 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:183/600: open fd = 3 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:184/600: open fd = 3 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:185/600: open fd = 3 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.917 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:186/600: open fd = 3 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:187/600: open fd = 3 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:188/600: open fd = 3 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:50.918 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:03:52.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:52 smithi112 ceph-mon[32617]: pgmap v479: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:03:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:52 smithi129 ceph-mon[32569]: pgmap v479: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:03:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:52 smithi078 ceph-mon[29163]: pgmap v479: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.713+0000 7f948efab640 1 -- 172.21.15.78:0/2485882014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 msgr2=0x7f948810be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.713+0000 7f948efab640 1 --2- 172.21.15.78:0/2485882014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f948810be80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9480009ca0 tx=0x7f948002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.713+0000 7f948efab640 1 -- 172.21.15.78:0/2485882014 shutdown_connections 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.713+0000 7f948efab640 1 --2- 172.21.15.78:0/2485882014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f948810be80 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.713+0000 7f948efab640 1 --2- 172.21.15.78:0/2485882014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488104d60 0x7f9488105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.713+0000 7f948efab640 1 --2- 172.21.15.78:0/2485882014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488103b20 0x7f9488104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.713+0000 7f948efab640 1 -- 172.21.15.78:0/2485882014 >> 172.21.15.78:0/2485882014 conn(0x7f94880fed10 msgr2=0x7f94881011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 -- 172.21.15.78:0/2485882014 shutdown_connections 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 -- 172.21.15.78:0/2485882014 wait complete. 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 Processor -- start 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 -- start start 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488103b20 0x7f948819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488104d60 0x7f948819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f94881a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f948810f650 con 0x7f9488109a60 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f948810f4d0 con 0x7f9488104d60 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948efab640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f948810f7d0 con 0x7f9488103b20 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948d548640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f94881a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948d548640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f94881a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49928/0 (socket says 172.21.15.78:49928) 2024-04-25T14:03:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.714+0000 7f948d548640 1 -- 172.21.15.78:0/3995376442 learned_addr learned my addr 172.21.15.78:0/3995376442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f947ffff640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488104d60 0x7f948819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948cd47640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488103b20 0x7f948819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948d548640 1 -- 172.21.15.78:0/3995376442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488103b20 msgr2=0x7f948819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948d548640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488103b20 0x7f948819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948d548640 1 -- 172.21.15.78:0/3995376442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488104d60 msgr2=0x7f948819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948d548640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488104d60 0x7f948819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948d548640 1 -- 172.21.15.78:0/3995376442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94881a1d00 con 0x7f9488109a60 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948d548640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f94881a1690 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f94800097b0 tx=0x7f9480003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f947dffb640 1 -- 172.21.15.78:0/3995376442 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480004240 con 0x7f9488109a60 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f947dffb640 1 -- 172.21.15.78:0/3995376442 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948002f380 con 0x7f9488109a60 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94881a1f90 con 0x7f9488109a60 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f947dffb640 1 -- 172.21.15.78:0/3995376442 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948002fc30 con 0x7f9488109a60 2024-04-25T14:03:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.715+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94881a23f0 con 0x7f9488109a60 2024-04-25T14:03:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f947dffb640 1 -- 172.21.15.78:0/3995376442 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f94800043e0 con 0x7f9488109a60 2024-04-25T14:03:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 Processor -- start 2024-04-25T14:03:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 -- start start 2024-04-25T14:03:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355dc81b0 0x556356168970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356168ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355da74d0 0x55635616fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563560ce460 con 0x556355dc81b0 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556356088590 con 0x556355d9f2f0 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f9495e58740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563560ca420 con 0x556355da74d0 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f947ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355dc81b0 0x556356168970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f947ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355dc81b0 0x556356168970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49932/0 (socket says 172.21.15.78:49932) 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f947ffff640 1 -- 172.21.15.78:0/282455934 learned_addr learned my addr 172.21.15.78:0/282455934 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f947dffb640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94700afae0 0x7f94700b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.718+0000 7f948cd47640 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356168ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947ffff640 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355da74d0 0x55635616fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947ffff640 1 -- 172.21.15.78:0/282455934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355da74d0 msgr2=0x55635616fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947ffff640 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355da74d0 0x55635616fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947ffff640 1 -- 172.21.15.78:0/282455934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 msgr2=0x556356168ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947ffff640 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356168ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f948d548640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94700afae0 0x7f94700b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947ffff640 1 -- 172.21.15.78:0/282455934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563560950d0 con 0x556355dc81b0 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947dffb640 1 -- 172.21.15.78:0/3995376442 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9480100300 con 0x7f9488109a60 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f947ffff640 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355dc81b0 0x556356168970 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f947800b620 tx=0x7f9478004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3e71b13c3e89c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f946effd640 1 -- 172.21.15.78:0/282455934 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947800be80 con 0x556355dc81b0 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f946effd640 1 -- 172.21.15.78:0/282455934 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94780054d0 con 0x556355dc81b0 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.719+0000 7f946effd640 1 -- 172.21.15.78:0/282455934 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94780106c0 con 0x556355dc81b0 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 -- 172.21.15.78:0/282455934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355dc81b0 msgr2=0x556356168970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f948d548640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94700afae0 0x7f94700b1ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f94800c8760 tx=0x7f94800f6f20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355dc81b0 0x556356168970 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f947800b620 tx=0x7f9478004270 comp rx=0 tx=0).stop 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 -- 172.21.15.78:0/282455934 shutdown_connections 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355da74d0 0x55635616fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356168ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 --2- 172.21.15.78:0/282455934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355dc81b0 0x556356168970 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 -- 172.21.15.78:0/282455934 >> 172.21.15.78:0/282455934 conn(0x556355dbc300 msgr2=0x556356164300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 -- 172.21.15.78:0/282455934 shutdown_connections 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 -- 172.21.15.78:0/282455934 wait complete. 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.720+0000 7f9495e58740 1 -- start start 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 1 Processor -- start 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355d9f2f0 0x5563561725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356172b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355dc81b0 0x556356179a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5563561707e0 con 0x556355da74d0 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556356055b60 con 0x556355dc81b0 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556356170960 con 0x556355d9f2f0 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f9495e58740 10 client.0 _ll_register_callbacks cb 0x55635607a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356172b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356172b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49948/0 (socket says 172.21.15.78:49948) 2024-04-25T14:03:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 -- 172.21.15.78:0/2610311783 learned_addr learned my addr 172.21.15.78:0/2610311783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f947ffff640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355dc81b0 0x556356179a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f947ffff640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355d9f2f0 0x5563561725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 -- 172.21.15.78:0/2610311783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355d9f2f0 msgr2=0x5563561725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355d9f2f0 0x5563561725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 -- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355dc81b0 msgr2=0x556356179a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355dc81b0 0x556356179a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f948cd47640 1 -- 172.21.15.78:0/2610311783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556355de0210 con 0x556355da74d0 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.721+0000 7f947ffff640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355d9f2f0 0x5563561725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:03:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f948cd47640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356172b00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f948400bec0 tx=0x7f9484012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19838c6fb11a601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9467fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9467fff640 1 -- 172.21.15.78:0/2610311783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484004220 con 0x556355da74d0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9467fff640 1 -- 172.21.15.78:0/2610311783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f94840043c0 con 0x556355da74d0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94780054d0 con 0x556355da74d0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9467fff640 1 -- 172.21.15.78:0/2610311783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9484005020 con 0x556355da74d0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9495e58740 10 client.16716 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556355eac620 con 0x556355da74d0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9467fff640 1 -- 172.21.15.78:0/2610311783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94780054d0 con 0x556355da74d0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9467fff640 1 -- 172.21.15.78:0/2610311783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f94840035f0 con 0x556355da74d0 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9495e58740 10 client.16716 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.722+0000 7f9495e58740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.723+0000 7f9495e58740 20 client.16716 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.723+0000 7f9495e58740 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d86d80 0x556356180320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.723+0000 7f9495e58740 4 client.16716 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.723+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563560dc070 con 0x556355d86d80 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.723+0000 7f947ffff640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d86d80 0x556356180320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.723+0000 7f947ffff640 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d86d80 0x556356180320 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbcc01032d80e9ee server_cookie=69a9f8e805825fb1 in_seq=0 out_seq=0 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.723+0000 7f9467fff640 10 client.16716 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9467fff640 1 -- 172.21.15.78:0/2610311783 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563560dc070 con 0x556355d86d80 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9467fff640 10 client.16716 handle_command_reply: tid=0 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 client.16716 shutdown 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 msgr2=0x556356172b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356172b00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f948400bec0 tx=0x7f9484012820 comp rx=0 tx=0).stop 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 shutdown_connections 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355dc81b0 0x556356179a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356172b00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d86d80 0x556356180320 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 --2- 172.21.15.78:0/2610311783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355d9f2f0 0x5563561725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9467fff640 0 client.16716 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 >> 172.21.15.78:0/2610311783 conn(0x556355dbc300 msgr2=0x556356167210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 shutdown_connections 2024-04-25T14:03:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 1 -- 172.21.15.78:0/2610311783 wait complete. 2024-04-25T14:03:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.724+0000 7f9495e58740 20 client.16716 trim_cache size 0 max 16384 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.739+0000 7f9495e58740 1 Processor -- start 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.739+0000 7f9495e58740 1 -- start start 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f9495e58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f9495e58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f9495e58740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 0x5563561640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f9495e58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94881adb10 con 0x556355da74d0 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f9495e58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94881ab3e0 con 0x556355d9f2f0 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f9495e58740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94881a9230 con 0x556355dc81b0 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49962/0 (socket says 172.21.15.78:49962) 2024-04-25T14:03:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 -- 172.21.15.78:0/1098541221 learned_addr learned my addr 172.21.15.78:0/1098541221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f947ffff640 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 -- 172.21.15.78:0/1098541221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 msgr2=0x5563561640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 0x5563561640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 -- 172.21.15.78:0/1098541221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 msgr2=0x556356163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356163bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 -- 172.21.15.78:0/1098541221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563561701b0 con 0x556355da74d0 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f947ffff640 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f948cd47640 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179980 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9484000c00 tx=0x7f948400bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66445c3b8af6ccd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f946e7fc640 1 -- 172.21.15.78:0/1098541221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94840051c0 con 0x556355da74d0 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f946e7fc640 1 -- 172.21.15.78:0/1098541221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948402c070 con 0x556355da74d0 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.740+0000 7f946e7fc640 1 -- 172.21.15.78:0/1098541221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484028470 con 0x556355da74d0 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 -- 172.21.15.78:0/1098541221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 msgr2=0x556356179980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179980 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9484000c00 tx=0x7f948400bb30 comp rx=0 tx=0).stop 2024-04-25T14:03:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 -- 172.21.15.78:0/1098541221 shutdown_connections 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 0x5563561640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179980 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 --2- 172.21.15.78:0/1098541221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 -- 172.21.15.78:0/1098541221 >> 172.21.15.78:0/1098541221 conn(0x556355dbc300 msgr2=0x55635616bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 -- 172.21.15.78:0/1098541221 shutdown_connections 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 -- 172.21.15.78:0/1098541221 wait complete. 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.741+0000 7f9495e58740 1 -- start start 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 1 Processor -- start 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356179230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 0x556356177910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94881a98a0 con 0x556355da74d0 2024-04-25T14:03:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94881aad10 con 0x556355d9f2f0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94881a9f70 con 0x556355dc81b0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f9495e58740 10 client.0 _ll_register_callbacks cb 0x7f948400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49964/0 (socket says 172.21.15.78:49964) 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 -- 172.21.15.78:0/79070212 learned_addr learned my addr 172.21.15.78:0/79070212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f947ffff640 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356179230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f947ffff640 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 0x556356177910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 -- 172.21.15.78:0/79070212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 msgr2=0x556356177910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 0x556356177910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 -- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 msgr2=0x556356179230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356179230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.742+0000 7f948cd47640 1 -- 172.21.15.78:0/79070212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556356179ec0 con 0x556355da74d0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f948cd47640 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179770 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f94840038c0 tx=0x7f94840038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cd4a996686906f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:03:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9467fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9467fff640 1 -- 172.21.15.78:0/79070212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948401d480 con 0x556355da74d0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9467fff640 1 -- 172.21.15.78:0/79070212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f948401d620 con 0x556355da74d0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f948402c070 con 0x556355da74d0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9467fff640 1 -- 172.21.15.78:0/79070212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f948401d820 con 0x556355da74d0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 10 client.16722 fetch_fsmap learned FSMap version 22 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55635616d4a0 con 0x556355da74d0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9467fff640 1 -- 172.21.15.78:0/79070212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948402c070 con 0x556355da74d0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9467fff640 1 -- 172.21.15.78:0/79070212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9484010520 con 0x556355da74d0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 10 client.16722 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 20 client.16722 populate_metadata read hostname 'smithi078' 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d5c1c0 0x5563561eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 4 client.16722 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.743+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563560d5ee0 con 0x556355d5c1c0 2024-04-25T14:03:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.744+0000 7f947ffff640 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d5c1c0 0x5563561eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:03:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.744+0000 7f947ffff640 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d5c1c0 0x5563561eb4f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=699a953021086471 server_cookie=33ca2726ea78feb2 in_seq=0 out_seq=0 2024-04-25T14:03:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.744+0000 7f9467fff640 10 client.16722 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9467fff640 1 -- 172.21.15.78:0/79070212 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563560d5ee0 con 0x556355d5c1c0 2024-04-25T14:03:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9467fff640 10 client.16722 handle_command_reply: tid=0 2024-04-25T14:03:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 client.16722 shutdown 2024-04-25T14:03:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 msgr2=0x556356179770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179770 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f94840038c0 tx=0x7f94840038f0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 shutdown_connections 2024-04-25T14:03:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556355d5c1c0 0x5563561eb4f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556355dc81b0 0x556356177910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556355da74d0 0x556356179770 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 --2- 172.21.15.78:0/79070212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556355d9f2f0 0x556356179230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9467fff640 0 client.16722 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 >> 172.21.15.78:0/79070212 conn(0x556355dbc300 msgr2=0x556356167100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 shutdown_connections 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 1 -- 172.21.15.78:0/79070212 wait complete. 2024-04-25T14:03:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.759+0000 7f9495e58740 20 client.16722 trim_cache size 0 max 16384 2024-04-25T14:03:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:03:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:03:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:03:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:03:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94700afae0 msgr2=0x7f94700b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94700afae0 0x7f94700b1ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f94800c8760 tx=0x7f94800f6f20 comp rx=0 tx=0).stop 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 msgr2=0x7f94881a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f94881a1690 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f94800097b0 tx=0x7f9480003c20 comp rx=0 tx=0).stop 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 shutdown_connections 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94700afae0 0x7f94700b1ff0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a60 0x7f94881a1690 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488104d60 0x7f948819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 --2- 172.21.15.78:0/3995376442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488103b20 0x7f948819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:03:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 >> 172.21.15.78:0/3995376442 conn(0x7f94880fed10 msgr2=0x7f9488105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:03:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 shutdown_connections 2024-04-25T14:03:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:03:53.760+0000 7f948efab640 1 -- 172.21.15.78:0/3995376442 wait complete. 2024-04-25T14:03:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:54 smithi129 ceph-mon[32569]: pgmap v480: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:54 smithi078 ceph-mon[29163]: pgmap v480: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:54.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:54 smithi112 ceph-mon[32617]: pgmap v480: 97 pgs: 97 active+clean; 2.9 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:03:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:56 smithi129 ceph-mon[32569]: pgmap v481: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:03:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:56 smithi078 ceph-mon[29163]: pgmap v481: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:03:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:56 smithi112 ceph-mon[32617]: pgmap v481: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:03:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:03:58 smithi129 ceph-mon[32569]: pgmap v482: 97 pgs: 97 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:03:58 smithi078 ceph-mon[29163]: pgmap v482: 97 pgs: 97 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:03:58.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:03:58 smithi112 ceph-mon[32617]: pgmap v482: 97 pgs: 97 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:00.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:00 smithi129 ceph-mon[32569]: pgmap v483: 97 pgs: 97 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:00 smithi078 ceph-mon[29163]: pgmap v483: 97 pgs: 97 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:00.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:00 smithi112 ceph-mon[32617]: pgmap v483: 97 pgs: 97 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:02 smithi129 ceph-mon[32569]: pgmap v484: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:02 smithi078 ceph-mon[29163]: pgmap v484: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:02.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:02 smithi112 ceph-mon[32617]: pgmap v484: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:04 smithi129 ceph-mon[32569]: pgmap v485: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:04 smithi078 ceph-mon[29163]: pgmap v485: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:04.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:04 smithi112 ceph-mon[32617]: pgmap v485: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:06.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:06.658 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:06 smithi129 ceph-mon[32569]: pgmap v486: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:06.683 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:06 smithi078 ceph-mon[29163]: pgmap v486: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:06 smithi112 ceph-mon[32617]: pgmap v486: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.784+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3115768240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a941029e0 msgr2=0x7f4a941036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.784+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3115768240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a941029e0 0x7f4a941036e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009900 tx=0x7f4a9002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.784+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3115768240 shutdown_connections 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3115768240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a94108920 0x7f4a9410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3115768240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a94103c20 0x7f4a94104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3115768240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a941029e0 0x7f4a941036e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3115768240 >> 172.21.15.78:0/3115768240 conn(0x7f4a940fdbd0 msgr2=0x7f4a94100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3115768240 shutdown_connections 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3115768240 wait complete. 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 Processor -- start 2024-04-25T14:04:06.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.785+0000 7f4a9b6e2640 1 -- start start 2024-04-25T14:04:06.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9b6e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a941029e0 0x7f4a94075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9b6e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a94103c20 0x7f4a9419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9b6e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a94108920 0x7f4a941a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9b6e2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9410e4c0 con 0x7f4a94108920 2024-04-25T14:04:06.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9b6e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9410e340 con 0x7f4a94103c20 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9b6e2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9410e640 con 0x7f4a941029e0 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a94108920 0x7f4a941a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a94108920 0x7f4a941a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35026/0 (socket says 172.21.15.78:35026) 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 -- 172.21.15.78:0/3443351844 learned_addr learned my addr 172.21.15.78:0/3443351844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9947e640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a941029e0 0x7f4a94075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a94103c20 0x7f4a9419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 -- 172.21.15.78:0/3443351844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a941029e0 msgr2=0x7f4a94075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a941029e0 0x7f4a94075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 -- 172.21.15.78:0/3443351844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a94103c20 msgr2=0x7f4a9419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a94103c20 0x7f4a9419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 -- 172.21.15.78:0/3443351844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a941a1da0 con 0x7f4a94108920 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a94108920 0x7f4a941a1580 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00eaa0 tx=0x7f4a8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a8a7fc640 1 -- 172.21.15.78:0/3443351844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c00cdb0 con 0x7f4a94108920 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a941a2090 con 0x7f4a94108920 2024-04-25T14:04:06.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a8a7fc640 1 -- 172.21.15.78:0/3443351844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a8c005020 con 0x7f4a94108920 2024-04-25T14:04:06.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.786+0000 7f4a8a7fc640 1 -- 172.21.15.78:0/3443351844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0116d0 con 0x7f4a94108920 2024-04-25T14:04:06.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.787+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a941a2590 con 0x7f4a94108920 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.789+0000 7f4aa258e740 1 Processor -- start 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.789+0000 7f4a8a7fc640 1 -- 172.21.15.78:0/3443351844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4a8c004b20 con 0x7f4a94108920 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.789+0000 7f4aa258e740 1 -- start start 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4aa258e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be52138d80 con 0x55be51e32390 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4aa258e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be520f28b0 con 0x55be51e094d0 2024-04-25T14:04:06.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4aa258e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55be52134d40 con 0x55be51e116b0 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521d3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35036/0 (socket says 172.21.15.78:35036) 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2921865862 learned_addr learned my addr 172.21.15.78:0/2921865862 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a9947e640 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2921865862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 msgr2=0x55be521d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521d4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2921865862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 msgr2=0x55be521d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521d3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a8a7fc640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a800afaf0 0x7f4a800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2921865862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be520c3a40 con 0x55be51e32390 2024-04-25T14:04:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a8a7fc640 1 -- 172.21.15.78:0/3443351844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4a8c0ded80 con 0x7f4a94108920 2024-04-25T14:04:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.790+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a800afaf0 0x7f4a800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.791+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521d3a60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4a84012560 tx=0x7f4a8400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e2eed033ce7458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.791+0000 7f4a7f7fe640 1 -- 172.21.15.78:0/2921865862 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a840106c0 con 0x55be51e32390 2024-04-25T14:04:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.791+0000 7f4a7f7fe640 1 -- 172.21.15.78:0/2921865862 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a8400e940 con 0x55be51e32390 2024-04-25T14:04:06.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.791+0000 7f4a7f7fe640 1 -- 172.21.15.78:0/2921865862 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84021680 con 0x55be51e32390 2024-04-25T14:04:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 -- 172.21.15.78:0/2921865862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 msgr2=0x55be521d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a800afaf0 0x7f4a800b2000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4a84020000 tx=0x7f4a8401d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521d3a60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4a84012560 tx=0x7f4a8400b820 comp rx=0 tx=0).stop 2024-04-25T14:04:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 -- 172.21.15.78:0/2921865862 shutdown_connections 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521d4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521d3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2921865862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521d3a60 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 -- 172.21.15.78:0/2921865862 >> 172.21.15.78:0/2921865862 conn(0x55be51e264e0 msgr2=0x55be521cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 -- 172.21.15.78:0/2921865862 shutdown_connections 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 -- 172.21.15.78:0/2921865862 wait complete. 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 -- start start 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.792+0000 7f4aa258e740 1 Processor -- start 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521dd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521dde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4aa258e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be521d5780 con 0x55be51e32390 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4aa258e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be520bfd10 con 0x55be51e094d0 2024-04-25T14:04:06.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4aa258e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55be521d5900 con 0x55be51e116b0 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4aa258e740 10 client.0 _ll_register_callbacks cb 0x55be520ec210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521e4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35052/0 (socket says 172.21.15.78:35052) 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 -- 172.21.15.78:0/1622466739 learned_addr learned my addr 172.21.15.78:0/1622466739 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a9947e640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521dd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521dde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 -- 172.21.15.78:0/1622466739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 msgr2=0x55be521dde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521dde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 -- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 msgr2=0x55be521dd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521dd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a99c7f640 1 -- 172.21.15.78:0/1622466739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be51f147d0 con 0x55be51e32390 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.793+0000 7f4a9947e640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521dd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521e4d70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0a4570 tx=0x7f4a8c0a6e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a973547dff95a16d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4a7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4a7cff9640 1 -- 172.21.15.78:0/1622466739 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c01f070 con 0x55be51e32390 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4a7cff9640 1 -- 172.21.15.78:0/1622466739 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4a8c0054f0 con 0x55be51e32390 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a8400e940 con 0x55be51e32390 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4a7cff9640 1 -- 172.21.15.78:0/1622466739 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c0b00b0 con 0x55be51e32390 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 10 client.16734 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55be51e4a3f0 con 0x55be51e32390 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4a7cff9640 1 -- 172.21.15.78:0/1622466739 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a8400e940 con 0x55be51e32390 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4a7cff9640 1 -- 172.21.15.78:0/1622466739 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4a8c00f040 con 0x55be51e32390 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 10 client.16734 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 20 client.16734 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51df0f60 0x55be521eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 4 client.16734 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.794+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be52146990 con 0x55be51df0f60 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.795+0000 7f4a9947e640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51df0f60 0x55be521eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.795+0000 7f4a7cff9640 10 client.16734 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.795+0000 7f4a9947e640 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51df0f60 0x55be521eb2c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43d6c7e28493034 server_cookie=5d63ff36aa82514b in_seq=0 out_seq=0 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4a7cff9640 1 -- 172.21.15.78:0/1622466739 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be52146990 con 0x55be51df0f60 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4a7cff9640 10 client.16734 handle_command_reply: tid=0 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 client.16734 shutdown 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 msgr2=0x55be521e4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521e4d70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0a4570 tx=0x7f4a8c0a6e10 comp rx=0 tx=0).stop 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 shutdown_connections 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be521e4d70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51df0f60 0x55be521eb2c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e116b0 0x55be521dde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 --2- 172.21.15.78:0/1622466739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521dd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4a7cff9640 0 client.16734 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 >> 172.21.15.78:0/1622466739 conn(0x55be51e264e0 msgr2=0x55be521d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 shutdown_connections 2024-04-25T14:04:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 1 -- 172.21.15.78:0/1622466739 wait complete. 2024-04-25T14:04:06.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.796+0000 7f4aa258e740 20 client.16734 trim_cache size 0 max 16384 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 Processor -- start 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 -- start start 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e116b0 0x55be521cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e32390 0x55be521dbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941adbb0 con 0x55be51e116b0 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941ab480 con 0x55be51e094d0 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.814+0000 7f4aa258e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941a92d0 con 0x55be51e32390 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e116b0 0x55be521cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e116b0 0x55be521cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35068/0 (socket says 172.21.15.78:35068) 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2780547392 learned_addr learned my addr 172.21.15.78:0/2780547392 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a9947e640 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e32390 0x55be521dbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2780547392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e32390 msgr2=0x55be521dbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e32390 0x55be521dbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2780547392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 msgr2=0x55be521cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 -- 172.21.15.78:0/2780547392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be51e0a0f0 con 0x55be51e116b0 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e116b0 0x55be521cf170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4a84012eb0 tx=0x7f4a84026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675e81a4bdaa5953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a7effd640 1 -- 172.21.15.78:0/2780547392 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84038070 con 0x55be51e116b0 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a7effd640 1 -- 172.21.15.78:0/2780547392 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84021e30 con 0x55be51e116b0 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.815+0000 7f4a7effd640 1 -- 172.21.15.78:0/2780547392 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a840052a0 con 0x55be51e116b0 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 -- 172.21.15.78:0/2780547392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e116b0 msgr2=0x55be521cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e116b0 0x55be521cf170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4a84012eb0 tx=0x7f4a84026000 comp rx=0 tx=0).stop 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 -- 172.21.15.78:0/2780547392 shutdown_connections 2024-04-25T14:04:06.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e32390 0x55be521dbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e116b0 0x55be521cf170 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2780547392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e094d0 0x55be521cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 -- 172.21.15.78:0/2780547392 >> 172.21.15.78:0/2780547392 conn(0x55be51e264e0 msgr2=0x55be521d8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 -- 172.21.15.78:0/2780547392 shutdown_connections 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.816+0000 7f4aa258e740 1 -- 172.21.15.78:0/2780547392 wait complete. 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 -- start start 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 Processor -- start 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e094d0 0x55be521e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e116b0 0x55be521e1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be522515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941a9940 con 0x55be51e32390 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941aadb0 con 0x55be51e116b0 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941aa010 con 0x55be51e094d0 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4aa258e740 10 client.0 _ll_register_callbacks cb 0x7f4a8c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:06.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4a99c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be522515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4a99c7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be522515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35080/0 (socket says 172.21.15.78:35080) 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4a99c7f640 1 -- 172.21.15.78:0/2158599373 learned_addr learned my addr 172.21.15.78:0/2158599373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4a98c7d640 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e116b0 0x55be521e1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.817+0000 7f4a9947e640 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e094d0 0x55be521e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a99c7f640 1 -- 172.21.15.78:0/2158599373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e094d0 msgr2=0x55be521e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e094d0 0x55be521e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a99c7f640 1 -- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e116b0 msgr2=0x55be521e1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e116b0 0x55be521e1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a99c7f640 1 -- 172.21.15.78:0/2158599373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be520f94b0 con 0x55be51e32390 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a99c7f640 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be522515d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0a2f50 tx=0x7f4a8c004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c34684cf887f907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a7cff9640 1 -- 172.21.15.78:0/2158599373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c01f070 con 0x55be51e32390 2024-04-25T14:04:06.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a7cff9640 1 -- 172.21.15.78:0/2158599373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4a8c0ab160 con 0x55be51e32390 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a84021e30 con 0x55be51e32390 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a7cff9640 1 -- 172.21.15.78:0/2158599373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c0a70d0 con 0x55be51e32390 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4aa258e740 10 client.16740 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55be52252a50 con 0x55be51e32390 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a7cff9640 1 -- 172.21.15.78:0/2158599373 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a84021e30 con 0x55be51e32390 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4a7cff9640 1 -- 172.21.15.78:0/2158599373 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4a8c0a66f0 con 0x55be51e32390 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.818+0000 7f4aa258e740 10 client.16740 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4aa258e740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4aa258e740 20 client.16740 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51dc63a0 0x55be522552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4aa258e740 4 client.16740 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be52140800 con 0x55be51dc63a0 2024-04-25T14:04:06.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4a9947e640 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51dc63a0 0x55be522552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4a7cff9640 10 client.16740 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.819+0000 7f4a9947e640 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51dc63a0 0x55be522552b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6a5dcdd8c247a5 server_cookie=318da5ce4ecb4f52 in_seq=0 out_seq=0 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4a7cff9640 1 -- 172.21.15.78:0/2158599373 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be52140800 con 0x55be51dc63a0 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4a7cff9640 10 client.16740 handle_command_reply: tid=0 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 client.16740 shutdown 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 msgr2=0x55be522515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be522515d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0a2f50 tx=0x7f4a8c004e10 comp rx=0 tx=0).stop 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 shutdown_connections 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be51e32390 0x55be522515d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be51e116b0 0x55be521e1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55be51dc63a0 0x55be522552b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 --2- 172.21.15.78:0/2158599373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be51e094d0 0x55be521e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4a7cff9640 0 client.16740 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 >> 172.21.15.78:0/2158599373 conn(0x55be51e264e0 msgr2=0x55be521cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 shutdown_connections 2024-04-25T14:04:06.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 1 -- 172.21.15.78:0/2158599373 wait complete. 2024-04-25T14:04:06.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.820+0000 7f4aa258e740 20 client.16740 trim_cache size 0 max 16384 2024-04-25T14:04:06.827 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:04:06.827 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:04:06.827 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:04:06.827 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a800afaf0 msgr2=0x7f4a800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a800afaf0 0x7f4a800b2000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4a84020000 tx=0x7f4a8401d000 comp rx=0 tx=0).stop 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a94108920 msgr2=0x7f4a941a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a94108920 0x7f4a941a1580 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00eaa0 tx=0x7f4a8c00ef60 comp rx=0 tx=0).stop 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 shutdown_connections 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4a800afaf0 0x7f4a800b2000 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a94108920 0x7f4a941a1580 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a94103c20 0x7f4a9419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 --2- 172.21.15.78:0/3443351844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a941029e0 0x7f4a94075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 >> 172.21.15.78:0/3443351844 conn(0x7f4a940fdbd0 msgr2=0x7f4a94104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 shutdown_connections 2024-04-25T14:04:06.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.821+0000 7f4a9b6e2640 1 -- 172.21.15.78:0/3443351844 wait complete. 2024-04-25T14:04:06.834 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:04:06.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 -- 172.21.15.78:0/3363666670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70102a30 msgr2=0x7f7f70103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 --2- 172.21.15.78:0/3363666670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70102a30 0x7f7f70103730 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009900 tx=0x7f7f6c02a410 comp rx=0 tx=0).stop 2024-04-25T14:04:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 -- 172.21.15.78:0/3363666670 shutdown_connections 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 --2- 172.21.15.78:0/3363666670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f70108970 0x7f7f7010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 --2- 172.21.15.78:0/3363666670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f70103c70 0x7f7f70104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 --2- 172.21.15.78:0/3363666670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70102a30 0x7f7f70103730 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 -- 172.21.15.78:0/3363666670 >> 172.21.15.78:0/3363666670 conn(0x7f7f700fdc20 msgr2=0x7f7f701000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 -- 172.21.15.78:0/3363666670 shutdown_connections 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.863+0000 7f7f78661640 1 -- 172.21.15.78:0/3363666670 wait complete. 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 Processor -- start 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 -- start start 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f70102a30 0x7f7f7019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70103c70 0x7f7f7019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f70108970 0x7f7f701a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010f660 con 0x7f7f70103c70 2024-04-25T14:04:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010f4e0 con 0x7f7f70102a30 2024-04-25T14:04:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f78661640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7010f7e0 con 0x7f7f70108970 2024-04-25T14:04:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70103c70 0x7f7f7019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70103c70 0x7f7f7019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35092/0 (socket says 172.21.15.78:35092) 2024-04-25T14:04:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 -- 172.21.15.78:0/3453077847 learned_addr learned my addr 172.21.15.78:0/3453077847 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f763fd640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f70102a30 0x7f7f7019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f70108970 0x7f7f701a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 -- 172.21.15.78:0/3453077847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f70108970 msgr2=0x7f7f701a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f70108970 0x7f7f701a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 -- 172.21.15.78:0/3453077847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f70102a30 msgr2=0x7f7f7019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f70102a30 0x7f7f7019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f75bfc640 1 -- 172.21.15.78:0/3453077847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f701a1db0 con 0x7f7f70103c70 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.864+0000 7f7f763fd640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f70102a30 0x7f7f7019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.865+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70103c70 0x7f7f7019a7b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d770 tx=0x7f7f6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.865+0000 7f7f677fe640 1 -- 172.21.15.78:0/3453077847 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60017070 con 0x7f7f70103c70 2024-04-25T14:04:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.865+0000 7f7f677fe640 1 -- 172.21.15.78:0/3453077847 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f600054d0 con 0x7f7f70103c70 2024-04-25T14:04:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.865+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f701a20a0 con 0x7f7f70103c70 2024-04-25T14:04:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.865+0000 7f7f677fe640 1 -- 172.21.15.78:0/3453077847 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60004c30 con 0x7f7f70103c70 2024-04-25T14:04:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.865+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701a25a0 con 0x7f7f70103c70 2024-04-25T14:04:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 Processor -- start 2024-04-25T14:04:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 -- start start 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed06970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed06ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9454d0 0x55982ed0ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55982ec6c460 con 0x55982e9661b0 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55982ec26590 con 0x55982e9454d0 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55982ec68420 con 0x55982e93d2f0 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f677fe640 1 -- 172.21.15.78:0/3453077847 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7f60004e10 con 0x7f7f70103c70 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f76bfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed06970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f76bfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed06970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35094/0 (socket says 172.21.15.78:35094) 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f76bfe640 1 -- 172.21.15.78:0/3489143232 learned_addr learned my addr 172.21.15.78:0/3489143232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.868+0000 7f7f763fd640 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed06ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9454d0 0x55982ed0ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f76bfe640 1 -- 172.21.15.78:0/3489143232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 msgr2=0x55982ed06ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed06ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f76bfe640 1 -- 172.21.15.78:0/3489143232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9454d0 msgr2=0x55982ed0ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9454d0 0x55982ed0ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f76bfe640 1 -- 172.21.15.78:0/3489143232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982ec330d0 con 0x55982e9661b0 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9454d0 0x55982ed0ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed06970 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009fc0 tx=0x7f7f68010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf62a2b2cbfb40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f64bfe640 1 -- 172.21.15.78:0/3489143232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68003b70 con 0x55982e9661b0 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f677fe640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7f5c0afae0 0x7f7f5c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f64bfe640 1 -- 172.21.15.78:0/3489143232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f68003d10 con 0x55982e9661b0 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f64bfe640 1 -- 172.21.15.78:0/3489143232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68011a40 con 0x55982e9661b0 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f763fd640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7f5c0afae0 0x7f7f5c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.869+0000 7f7f677fe640 1 -- 172.21.15.78:0/3453077847 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7f600de7a0 con 0x7f7f70103c70 2024-04-25T14:04:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/3489143232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 msgr2=0x55982ed06970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f763fd640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7f5c0afae0 0x7f7f5c0b1ff0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009230 tx=0x7f7f6c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed06970 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009fc0 tx=0x7f7f68010860 comp rx=0 tx=0).stop 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/3489143232 shutdown_connections 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9454d0 0x55982ed0ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed06ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/3489143232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed06970 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/3489143232 >> 172.21.15.78:0/3489143232 conn(0x55982e95a300 msgr2=0x55982ed02300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/3489143232 shutdown_connections 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.870+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/3489143232 wait complete. 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 -- start start 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 Processor -- start 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed10710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed10c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9661b0 0x55982ed17bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55982ed0e840 con 0x55982e9454d0 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55982ebf3b60 con 0x55982e9661b0 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55982ed0e9c0 con 0x55982e93d2f0 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f7f5ed740 10 client.0 _ll_register_callbacks cb 0x55982ec182b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed10c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed10c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35102/0 (socket says 172.21.15.78:35102) 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 -- 172.21.15.78:0/4102326017 learned_addr learned my addr 172.21.15.78:0/4102326017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f763fd640 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed10710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 -- 172.21.15.78:0/4102326017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 msgr2=0x55982ed10710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed10710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 -- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9661b0 msgr2=0x55982ed17bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9661b0 0x55982ed17bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9661b0 0x55982ed17bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.871+0000 7f7f75bfc640 1 -- 172.21.15.78:0/4102326017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982e97e210 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed10c50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009610 tx=0x7f7f600af410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dc0f762d89ea15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f59ffb640 1 -- 172.21.15.78:0/4102326017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60020050 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f59ffb640 1 -- 172.21.15.78:0/4102326017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7f600a6ed0 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f68003d10 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f59ffb640 1 -- 172.21.15.78:0/4102326017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7f600a7370 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 10 client.16752 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55982ea4a620 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f59ffb640 1 -- 172.21.15.78:0/4102326017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f68003d10 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f59ffb640 1 -- 172.21.15.78:0/4102326017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7f600af640 con 0x55982e9454d0 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 10 client.16752 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 20 client.16752 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e924d80 0x55982ed1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 4 client.16752 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.872+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55982ec7a070 con 0x55982e924d80 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.873+0000 7f7f763fd640 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e924d80 0x55982ed1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.873+0000 7f7f763fd640 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e924d80 0x55982ed1e350 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da138f4d8bff8515 server_cookie=693ecea98fea2db2 in_seq=0 out_seq=0 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.873+0000 7f7f59ffb640 10 client.16752 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f59ffb640 1 -- 172.21.15.78:0/4102326017 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55982ec7a070 con 0x55982e924d80 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f59ffb640 10 client.16752 handle_command_reply: tid=0 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 client.16752 shutdown 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 msgr2=0x55982ed10c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed10c50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009610 tx=0x7f7f600af410 comp rx=0 tx=0).stop 2024-04-25T14:04:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 shutdown_connections 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e9661b0 0x55982ed17bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed10c50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e924d80 0x55982ed1e350 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/4102326017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e93d2f0 0x55982ed10710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 >> 172.21.15.78:0/4102326017 conn(0x55982e95a300 msgr2=0x55982ed05210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f59ffb640 0 client.16752 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 shutdown_connections 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/4102326017 wait complete. 2024-04-25T14:04:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.874+0000 7f7f7f5ed740 20 client.16752 trim_cache size 0 max 16384 2024-04-25T14:04:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.889+0000 7f7f7f5ed740 1 Processor -- start 2024-04-25T14:04:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.889+0000 7f7f7f5ed740 1 -- start start 2024-04-25T14:04:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.889+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed23b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.889+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9454d0 0x55982ed17ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.889+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.889+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701adb40 con 0x55982e9661b0 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701ab410 con 0x55982e93d2f0 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701a9260 con 0x55982e9454d0 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed240b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35110/0 (socket says 172.21.15.78:35110) 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 -- 172.21.15.78:0/1197300281 learned_addr learned my addr 172.21.15.78:0/1197300281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f763fd640 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed23b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9454d0 0x55982ed17ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 -- 172.21.15.78:0/1197300281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9454d0 msgr2=0x55982ed17ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9454d0 0x55982ed17ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 -- 172.21.15.78:0/1197300281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 msgr2=0x55982ed23b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed23b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 -- 172.21.15.78:0/1197300281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982ed0b5c0 con 0x55982e9661b0 2024-04-25T14:04:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9454d0 0x55982ed17ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed240b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006010 tx=0x7f7f68009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2690851819e26bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f5bfff640 1 -- 172.21.15.78:0/1197300281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f680049e0 con 0x55982e9661b0 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f5bfff640 1 -- 172.21.15.78:0/1197300281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f68003e20 con 0x55982e9661b0 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.890+0000 7f7f5bfff640 1 -- 172.21.15.78:0/1197300281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f680045e0 con 0x55982e9661b0 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/1197300281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 msgr2=0x55982ed240b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed240b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006010 tx=0x7f7f68009d60 comp rx=0 tx=0).stop 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/1197300281 shutdown_connections 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9661b0 0x55982ed240b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9454d0 0x55982ed17ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/1197300281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed23b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/1197300281 >> 172.21.15.78:0/1197300281 conn(0x55982e95a300 msgr2=0x55982ed0ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/1197300281 shutdown_connections 2024-04-25T14:04:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.891+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/1197300281 wait complete. 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 -- start start 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 Processor -- start 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9661b0 0x55982ed84c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701a98d0 con 0x55982e9454d0 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701aad40 con 0x55982e93d2f0 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701a9fa0 con 0x55982e9661b0 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f7f5ed740 10 client.0 _ll_register_callbacks cb 0x7f7f60009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35112/0 (socket says 172.21.15.78:35112) 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 -- 172.21.15.78:0/2314983469 learned_addr learned my addr 172.21.15.78:0/2314983469 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f76bfe640 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9661b0 0x55982ed84c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f763fd640 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 -- 172.21.15.78:0/2314983469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9661b0 msgr2=0x55982ed84c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9661b0 0x55982ed84c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 -- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 msgr2=0x55982ed171b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed171b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.892+0000 7f7f75bfc640 1 -- 172.21.15.78:0/2314983469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982ed85230 con 0x55982e9454d0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f75bfc640 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed176f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7f600a6970 tx=0x7f7f600a25e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd8fecc171db16d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f59ffb640 1 -- 172.21.15.78:0/2314983469 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60020050 con 0x55982e9454d0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f59ffb640 1 -- 172.21.15.78:0/2314983469 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7f600a3260 con 0x55982e9454d0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f68003e20 con 0x55982e9454d0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f59ffb640 1 -- 172.21.15.78:0/2314983469 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7f600a65c0 con 0x55982e9454d0 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 10 client.16758 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55982ed864a0 con 0x55982e9454d0 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f59ffb640 1 -- 172.21.15.78:0/2314983469 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f68003e20 con 0x55982e9454d0 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f59ffb640 1 -- 172.21.15.78:0/2314983469 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7f6006d340 con 0x55982e9454d0 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 10 client.16758 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 20 client.16758 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e8fa1c0 0x55982ed88e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 4 client.16758 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.893+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55982ec73ee0 con 0x55982e8fa1c0 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.894+0000 7f7f763fd640 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e8fa1c0 0x55982ed88e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.894+0000 7f7f59ffb640 10 client.16758 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.894+0000 7f7f763fd640 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e8fa1c0 0x55982ed88e70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e19e09351792c0 server_cookie=b592e0b2a5731289 in_seq=0 out_seq=0 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.909+0000 7f7f59ffb640 1 -- 172.21.15.78:0/2314983469 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55982ec73ee0 con 0x55982e8fa1c0 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.909+0000 7f7f59ffb640 10 client.16758 handle_command_reply: tid=0 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.909+0000 7f7f7f5ed740 1 client.16758 shutdown 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 msgr2=0x55982ed176f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed176f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7f600a6970 tx=0x7f7f600a25e0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 shutdown_connections 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55982e8fa1c0 0x55982ed88e70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55982e9661b0 0x55982ed84c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55982e9454d0 0x55982ed176f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 --2- 172.21.15.78:0/2314983469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55982e93d2f0 0x55982ed171b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f59ffb640 0 client.16758 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 >> 172.21.15.78:0/2314983469 conn(0x55982e95a300 msgr2=0x55982ed05100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 shutdown_connections 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 1 -- 172.21.15.78:0/2314983469 wait complete. 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f7f5ed740 20 client.16758 trim_cache size 0 max 16384 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:04:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:04:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7f5c0afae0 msgr2=0x7f7f5c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7f5c0afae0 0x7f7f5c0b1ff0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009230 tx=0x7f7f6c005a20 comp rx=0 tx=0).stop 2024-04-25T14:04:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70103c70 msgr2=0x7f7f7019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70103c70 0x7f7f7019a7b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000d770 tx=0x7f7f6000dc30 comp rx=0 tx=0).stop 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 shutdown_connections 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f70108970 0x7f7f701a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7f5c0afae0 0x7f7f5c0b1ff0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f70103c70 0x7f7f7019a7b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 --2- 172.21.15.78:0/3453077847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f70102a30 0x7f7f7019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.910+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 >> 172.21.15.78:0/3453077847 conn(0x7f7f700fdc20 msgr2=0x7f7f70104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.911+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 shutdown_connections 2024-04-25T14:04:06.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:06.911+0000 7f7f78661640 1 -- 172.21.15.78:0/3453077847 wait complete. 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 -- 172.21.15.78:0/2887709833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204104470 msgr2=0x7fd20410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 --2- 172.21.15.78:0/2887709833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204104470 0x7fd20410abb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009ca0 tx=0x7fd1fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 -- 172.21.15.78:0/2887709833 shutdown_connections 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 --2- 172.21.15.78:0/2887709833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204104470 0x7fd20410abb0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 --2- 172.21.15.78:0/2887709833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd204103930 0x7fd204103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 --2- 172.21.15.78:0/2887709833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd204102f50 0x7fd204103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 -- 172.21.15.78:0/2887709833 >> 172.21.15.78:0/2887709833 conn(0x7fd2040fd370 msgr2=0x7fd2040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 -- 172.21.15.78:0/2887709833 shutdown_connections 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 -- 172.21.15.78:0/2887709833 wait complete. 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 Processor -- start 2024-04-25T14:04:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.976+0000 7fd20b44a640 1 -- start start 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd20b44a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102f50 0x7fd204199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd20b44a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd204103930 0x7fd2041995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd20b44a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd204104470 0x7fd2041a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd20b44a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd20410e540 con 0x7fd204102f50 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd20b44a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd20410e3c0 con 0x7fd204104470 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd20b44a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd20410e6c0 con 0x7fd204103930 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102f50 0x7fd204199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102f50 0x7fd204199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35120/0 (socket says 172.21.15.78:35120) 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 -- 172.21.15.78:0/3227484597 learned_addr learned my addr 172.21.15.78:0/3227484597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2089e5640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd204103930 0x7fd2041995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 -- 172.21.15.78:0/3227484597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd204103930 msgr2=0x7fd2041995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd204103930 0x7fd2041995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 -- 172.21.15.78:0/3227484597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd204104470 msgr2=0x7fd2041a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd204104470 0x7fd2041a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 -- 172.21.15.78:0/3227484597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041a0bc0 con 0x7fd204102f50 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2089e5640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd204103930 0x7fd2041995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd2091e6640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102f50 0x7fd204199080 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d750 tx=0x7fd20000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd1fa7fc640 1 -- 172.21.15.78:0/3227484597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200017070 con 0x7fd204102f50 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd1fa7fc640 1 -- 172.21.15.78:0/3227484597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2000054d0 con 0x7fd204102f50 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2041a0eb0 con 0x7fd204102f50 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.977+0000 7fd1fa7fc640 1 -- 172.21.15.78:0/3227484597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200004c30 con 0x7fd204102f50 2024-04-25T14:04:06.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.978+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a12c0 con 0x7fd204102f50 2024-04-25T14:04:06.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.979+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20406a1c0 con 0x7fd204102f50 2024-04-25T14:04:06.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.980+0000 7fd1fa7fc640 1 -- 172.21.15.78:0/3227484597 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd200004dd0 con 0x7fd204102f50 2024-04-25T14:04:06.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.981+0000 7fd1fa7fc640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1f00afaf0 0x7fd1f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:06.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.981+0000 7fd2089e5640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1f00afaf0 0x7fd1f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:06.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.981+0000 7fd1fa7fc640 1 -- 172.21.15.78:0/3227484597 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd2000deae0 con 0x7fd204102f50 2024-04-25T14:04:06.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.981+0000 7fd2089e5640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1f00afaf0 0x7fd1f00b2000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4005d00 tx=0x7fd1f4005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:06.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:06.984+0000 7fd1fa7fc640 1 -- 172.21.15.78:0/3227484597 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd20406a1c0 con 0x7fd204102f50 2024-04-25T14:04:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.201+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd20402dc30 con 0x7fd204102f50 2024-04-25T14:04:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.202+0000 7fd1fa7fc640 1 -- 172.21.15.78:0/3227484597 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fd2000a76b0 con 0x7fd204102f50 2024-04-25T14:04:07.202 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:04:07.202 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:04:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.204+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1f00afaf0 msgr2=0x7fd1f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.204+0000 7fd20b44a640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1f00afaf0 0x7fd1f00b2000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4005d00 tx=0x7fd1f4005ed0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.204+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102f50 msgr2=0x7fd204199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.204+0000 7fd20b44a640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102f50 0x7fd204199080 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d750 tx=0x7fd20000dc10 comp rx=0 tx=0).stop 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 shutdown_connections 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd204104470 0x7fd2041a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd204103930 0x7fd2041995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd1f00afaf0 0x7fd1f00b2000 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 --2- 172.21.15.78:0/3227484597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204102f50 0x7fd204199080 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 >> 172.21.15.78:0/3227484597 conn(0x7fd2040fd370 msgr2=0x7fd204106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 shutdown_connections 2024-04-25T14:04:07.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.205+0000 7fd20b44a640 1 -- 172.21.15.78:0/3227484597 wait complete. 2024-04-25T14:04:07.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:04:07.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 -- 172.21.15.78:0/2863800321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481050b0 msgr2=0x7fa2481054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 --2- 172.21.15.78:0/2863800321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481050b0 0x7fa2481054c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fa244009900 tx=0x7fa24402a410 comp rx=0 tx=0).stop 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 -- 172.21.15.78:0/2863800321 shutdown_connections 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 --2- 172.21.15.78:0/2863800321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa248102260 0x7fa24810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 --2- 172.21.15.78:0/2863800321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa248101760 0x7fa248101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 --2- 172.21.15.78:0/2863800321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2481050b0 0x7fa2481054c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 -- 172.21.15.78:0/2863800321 >> 172.21.15.78:0/2863800321 conn(0x7fa2480fd380 msgr2=0x7fa2480ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 -- 172.21.15.78:0/2863800321 shutdown_connections 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.367+0000 7fa2501e3640 1 -- 172.21.15.78:0/2863800321 wait complete. 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 Processor -- start 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 -- start start 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa248101760 0x7fa24819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa248102260 0x7fa24819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2481050b0 0x7fa2481a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa24810f600 con 0x7fa248101760 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa24810f480 con 0x7fa2481050b0 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa2501e3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa24810f780 con 0x7fa248102260 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa248101760 0x7fa24819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa248101760 0x7fa24819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35138/0 (socket says 172.21.15.78:35138) 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 -- 172.21.15.78:0/4117509456 learned_addr learned my addr 172.21.15.78:0/4117509456 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 -- 172.21.15.78:0/4117509456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa248102260 msgr2=0x7fa24819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24d77e640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa248102260 0x7fa24819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa248102260 0x7fa24819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 -- 172.21.15.78:0/4117509456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2481050b0 msgr2=0x7fa2481a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24e780640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2481050b0 0x7fa2481a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2481050b0 0x7fa2481a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24df7f640 1 -- 172.21.15.78:0/4117509456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2481a1dd0 con 0x7fa248101760 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.368+0000 7fa24d77e640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa248102260 0x7fa24819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.369+0000 7fa24df7f640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa248101760 0x7fa24819a290 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa244006010 tx=0x7fa244031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.369+0000 7fa23effd640 1 -- 172.21.15.78:0/4117509456 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24403f030 con 0x7fa248101760 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.369+0000 7fa23effd640 1 -- 172.21.15.78:0/4117509456 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa244003aa0 con 0x7fa248101760 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.369+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2481a2060 con 0x7fa248101760 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.369+0000 7fa23effd640 1 -- 172.21.15.78:0/4117509456 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244035070 con 0x7fa248101760 2024-04-25T14:04:07.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.369+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a2550 con 0x7fa248101760 2024-04-25T14:04:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.371+0000 7fa257169740 1 Processor -- start 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.371+0000 7fa23effd640 1 -- 172.21.15.78:0/4117509456 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa244042030 con 0x7fa248101760 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.371+0000 7fa257169740 1 -- start start 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa257169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b5382810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa257169740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b5382d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa257169740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b5383a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa257169740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605b52e6d80 con 0x5605b4fbf6b0 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa257169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605b52a0a20 con 0x5605b4fe0390 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa257169740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5605b52e2d40 con 0x5605b4fb74d0 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b5383a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b5383a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35152/0 (socket says 172.21.15.78:35152) 2024-04-25T14:04:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 -- 172.21.15.78:0/4135094648 learned_addr learned my addr 172.21.15.78:0/4135094648 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24e780640 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b5382810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24df7f640 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b5382d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa23effd640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2340afaf0 0x7fa2340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 -- 172.21.15.78:0/4135094648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 msgr2=0x5605b5382d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b5382d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 -- 172.21.15.78:0/4135094648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 msgr2=0x5605b5382810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b5382810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 -- 172.21.15.78:0/4135094648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b52abc30 con 0x5605b4fbf6b0 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24df7f640 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b5382d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24e780640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2340afaf0 0x7fa2340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa23effd640 1 -- 172.21.15.78:0/4117509456 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa2440ff270 con 0x7fa248101760 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.372+0000 7fa24d77e640 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b5383a20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa238009cb0 tx=0x7fa23800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85384eb5d28ff528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.373+0000 7fa233fff640 1 -- 172.21.15.78:0/4135094648 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238016070 con 0x5605b4fbf6b0 2024-04-25T14:04:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.373+0000 7fa233fff640 1 -- 172.21.15.78:0/4135094648 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2380035f0 con 0x5605b4fbf6b0 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.373+0000 7fa233fff640 1 -- 172.21.15.78:0/4135094648 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2380116d0 con 0x5605b4fbf6b0 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.373+0000 7fa257169740 1 -- 172.21.15.78:0/4135094648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 msgr2=0x5605b5383a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.373+0000 7fa24e780640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2340afaf0 0x7fa2340b2000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5605b5383860 tx=0x7fa240004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.373+0000 7fa257169740 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b5383a20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa238009cb0 tx=0x7fa23800e6c0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- 172.21.15.78:0/4135094648 shutdown_connections 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b5383a20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b5382d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 --2- 172.21.15.78:0/4135094648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b5382810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- 172.21.15.78:0/4135094648 >> 172.21.15.78:0/4135094648 conn(0x5605b4fd44e0 msgr2=0x5605b537e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- 172.21.15.78:0/4135094648 shutdown_connections 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- 172.21.15.78:0/4135094648 wait complete. 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- start start 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 Processor -- start 2024-04-25T14:04:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b538ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b53939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605b5384530 con 0x5605b4fbf6b0 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605b526ddc0 con 0x5605b4fe0390 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.374+0000 7fa257169740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5605b53846b0 con 0x5605b4fb74d0 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa257169740 10 client.0 _ll_register_callbacks cb 0x5605b5289600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b538ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b538ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35164/0 (socket says 172.21.15.78:35164) 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 -- 172.21.15.78:0/1164326013 learned_addr learned my addr 172.21.15.78:0/1164326013 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24e780640 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b53939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24df7f640 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 -- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 msgr2=0x5605b538c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 -- 172.21.15.78:0/1164326013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 msgr2=0x5605b53939d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b53939d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 -- 172.21.15.78:0/1164326013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b4ff83f0 con 0x5605b4fbf6b0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa24d77e640 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b538ca70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa23800ebc0 tx=0x7fa238003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8887e12affaf4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa2317fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa2317fa640 1 -- 172.21.15.78:0/1164326013 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238028060 con 0x5605b4fbf6b0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa2317fa640 1 -- 172.21.15.78:0/1164326013 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa238004210 con 0x5605b4fbf6b0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2380035f0 con 0x5605b4fbf6b0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa2317fa640 1 -- 172.21.15.78:0/1164326013 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa238016040 con 0x5605b4fbf6b0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa257169740 10 client.16776 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.375+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5605b50c27d0 con 0x5605b4fbf6b0 2024-04-25T14:04:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa2317fa640 1 -- 172.21.15.78:0/1164326013 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2380035f0 con 0x5605b4fbf6b0 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa2317fa640 1 -- 172.21.15.78:0/1164326013 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa23800f040 con 0x5605b4fbf6b0 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa257169740 10 client.16776 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa257169740 10 client.16776 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa257169740 20 client.16776 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa257169740 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f9ef60 0x5605b5399e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa257169740 4 client.16776 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b52f4990 con 0x5605b4f9ef60 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.376+0000 7fa24df7f640 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f9ef60 0x5605b5399e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.377+0000 7fa2317fa640 10 client.16776 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.377+0000 7fa24df7f640 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f9ef60 0x5605b5399e80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6dbe723aad82ad39 server_cookie=be842e8d360aae04 in_seq=0 out_seq=0 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.377+0000 7fa2317fa640 1 -- 172.21.15.78:0/1164326013 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5605b52f4990 con 0x5605b4f9ef60 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.377+0000 7fa2317fa640 10 client.16776 handle_command_reply: tid=0 2024-04-25T14:04:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.377+0000 7fa257169740 1 client.16776 shutdown 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.377+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 msgr2=0x5605b538ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.377+0000 7fa257169740 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b538ca70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa23800ebc0 tx=0x7fa238003ca0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 shutdown_connections 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f9ef60 0x5605b5399e80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fe0390 0x5605b53939d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fbf6b0 0x5605b538ca70 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 --2- 172.21.15.78:0/1164326013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa2317fa640 0 client.16776 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 >> 172.21.15.78:0/1164326013 conn(0x5605b4fd44e0 msgr2=0x5605b537d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 shutdown_connections 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 1 -- 172.21.15.78:0/1164326013 wait complete. 2024-04-25T14:04:07.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.378+0000 7fa257169740 20 client.16776 trim_cache size 0 max 16384 2024-04-25T14:04:07.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 Processor -- start 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 -- start start 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fbf6b0 0x5605b53938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b539e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481adc00 con 0x5605b4fe0390 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ab4d0 con 0x5605b4fbf6b0 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa257169740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481a9320 con 0x5605b4fb74d0 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa24e780640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b539e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa24e780640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b539e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35178/0 (socket says 172.21.15.78:35178) 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa24e780640 1 -- 172.21.15.78:0/3991836272 learned_addr learned my addr 172.21.15.78:0/3991836272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa24df7f640 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa24d77e640 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fbf6b0 0x5605b53938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.391+0000 7fa24e780640 1 -- 172.21.15.78:0/3991836272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 msgr2=0x5605b538b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa24e780640 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa24e780640 1 -- 172.21.15.78:0/3991836272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fbf6b0 msgr2=0x5605b53938f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa24e780640 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fbf6b0 0x5605b53938f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa24e780640 1 -- 172.21.15.78:0/3991836272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b5387cc0 con 0x5605b4fe0390 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa24e780640 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b539e560 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa24001c7b0 tx=0x7fa240017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d37f157d2ad9420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa2337fe640 1 -- 172.21.15.78:0/3991836272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24001c190 con 0x5605b4fe0390 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa2337fe640 1 -- 172.21.15.78:0/3991836272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa240010590 con 0x5605b4fe0390 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.392+0000 7fa2337fe640 1 -- 172.21.15.78:0/3991836272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240005020 con 0x5605b4fe0390 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- 172.21.15.78:0/3991836272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 msgr2=0x5605b539e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b539e560 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa24001c7b0 tx=0x7fa240017000 comp rx=0 tx=0).stop 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- 172.21.15.78:0/3991836272 shutdown_connections 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b539e560 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fbf6b0 0x5605b53938f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 --2- 172.21.15.78:0/3991836272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fb74d0 0x5605b538b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- 172.21.15.78:0/3991836272 >> 172.21.15.78:0/3991836272 conn(0x5605b4fd44e0 msgr2=0x5605b538acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- 172.21.15.78:0/3991836272 shutdown_connections 2024-04-25T14:04:07.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- 172.21.15.78:0/3991836272 wait complete. 2024-04-25T14:04:07.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- start start 2024-04-25T14:04:07.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 Processor -- start 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fb74d0 0x5605b5390ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fbf6b0 0x5605b5391220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b5392150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481a9990 con 0x5605b4fe0390 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481aae00 con 0x5605b4fb74d0 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.393+0000 7fa257169740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481aa060 con 0x5605b4fbf6b0 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa257169740 10 client.0 _ll_register_callbacks cb 0x7fa240009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b5392150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b5392150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35188/0 (socket says 172.21.15.78:35188) 2024-04-25T14:04:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 -- 172.21.15.78:0/306492635 learned_addr learned my addr 172.21.15.78:0/306492635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24df7f640 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fb74d0 0x5605b5390ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24d77e640 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fbf6b0 0x5605b5391220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 -- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fbf6b0 msgr2=0x5605b5391220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fbf6b0 0x5605b5391220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 -- 172.21.15.78:0/306492635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fb74d0 msgr2=0x5605b5390ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fb74d0 0x5605b5390ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 -- 172.21.15.78:0/306492635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b5393e30 con 0x5605b4fe0390 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa24e780640 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b5392150 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa24001cfa0 tx=0x7fa240004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c59123126288a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa2317fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa2317fa640 1 -- 172.21.15.78:0/306492635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240033070 con 0x5605b4fe0390 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa2317fa640 1 -- 172.21.15.78:0/306492635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa240010950 con 0x5605b4fe0390 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa240010590 con 0x5605b4fe0390 2024-04-25T14:04:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa2317fa640 1 -- 172.21.15.78:0/306492635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa24001c410 con 0x5605b4fe0390 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa257169740 10 client.16782 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.394+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5605b53937c0 con 0x5605b4fe0390 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa2317fa640 1 -- 172.21.15.78:0/306492635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa240010590 con 0x5605b4fe0390 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa2317fa640 1 -- 172.21.15.78:0/306492635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa24002f440 con 0x5605b4fe0390 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa257169740 10 client.16782 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa257169740 10 client.16782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa257169740 20 client.16782 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa257169740 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f743a0 0x5605b54044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa257169740 4 client.16782 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605b52ee800 con 0x5605b4f743a0 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa24df7f640 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f743a0 0x5605b54044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa24df7f640 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f743a0 0x5605b54044d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68448dc7b85fc55d server_cookie=f74c407b810da600 in_seq=0 out_seq=0 2024-04-25T14:04:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.395+0000 7fa2317fa640 10 client.16782 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa2317fa640 1 -- 172.21.15.78:0/306492635 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605b52ee800 con 0x5605b4f743a0 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa2317fa640 10 client.16782 handle_command_reply: tid=0 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 client.16782 shutdown 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 msgr2=0x5605b5392150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b5392150 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa24001cfa0 tx=0x7fa240004ce0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 shutdown_connections 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5605b4f743a0 0x5605b54044d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605b4fe0390 0x5605b5392150 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa2317fa640 0 client.16782 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605b4fbf6b0 0x5605b5391220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 --2- 172.21.15.78:0/306492635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605b4fb74d0 0x5605b5390ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 >> 172.21.15.78:0/306492635 conn(0x5605b4fd44e0 msgr2=0x5605b5380e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 shutdown_connections 2024-04-25T14:04:07.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 1 -- 172.21.15.78:0/306492635 wait complete. 2024-04-25T14:04:07.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.396+0000 7fa257169740 20 client.16782 trim_cache size 0 max 16384 2024-04-25T14:04:07.403 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2340afaf0 msgr2=0x7fa2340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2340afaf0 0x7fa2340b2000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x5605b5383860 tx=0x7fa240004a40 comp rx=0 tx=0).stop 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa248101760 msgr2=0x7fa24819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa248101760 0x7fa24819a290 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa244006010 tx=0x7fa244031a50 comp rx=0 tx=0).stop 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 shutdown_connections 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2481050b0 0x7fa2481a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa248102260 0x7fa24819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa2340afaf0 0x7fa2340b2000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 --2- 172.21.15.78:0/4117509456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa248101760 0x7fa24819a290 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 >> 172.21.15.78:0/4117509456 conn(0x7fa2480fd380 msgr2=0x7fa248105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 shutdown_connections 2024-04-25T14:04:07.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.397+0000 7fa2501e3640 1 -- 172.21.15.78:0/4117509456 wait complete. 2024-04-25T14:04:07.407 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:04:07.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:07 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3227484597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:07.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 -- 172.21.15.78:0/2700082481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103c20 msgr2=0x7f34a0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 --2- 172.21.15.78:0/2700082481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103c20 0x7f34a0104030 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3488009900 tx=0x7f348802a410 comp rx=0 tx=0).stop 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 -- 172.21.15.78:0/2700082481 shutdown_connections 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 --2- 172.21.15.78:0/2700082481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a0108920 0x7f34a010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 --2- 172.21.15.78:0/2700082481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103c20 0x7f34a0104030 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 --2- 172.21.15.78:0/2700082481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a01029e0 0x7f34a01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 -- 172.21.15.78:0/2700082481 >> 172.21.15.78:0/2700082481 conn(0x7f34a00fdbd0 msgr2=0x7f34a0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 -- 172.21.15.78:0/2700082481 shutdown_connections 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.517+0000 7f34a5696640 1 -- 172.21.15.78:0/2700082481 wait complete. 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 Processor -- start 2024-04-25T14:04:07.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 -- start start 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a01029e0 0x7f34a01991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a0103c20 0x7f34a0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a0108920 0x7f34a01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010e500 con 0x7f34a01029e0 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010e380 con 0x7f34a0103c20 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f34a5696640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010e680 con 0x7f34a0108920 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a01029e0 0x7f34a01991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a01029e0 0x7f34a01991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35212/0 (socket says 172.21.15.78:35212) 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f349effd640 1 -- 172.21.15.78:0/3120543845 learned_addr learned my addr 172.21.15.78:0/3120543845 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.518+0000 7f3497fff640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a0103c20 0x7f34a0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349f7fe640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a0108920 0x7f34a01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349effd640 1 -- 172.21.15.78:0/3120543845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a0108920 msgr2=0x7f34a01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349effd640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a0108920 0x7f34a01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349effd640 1 -- 172.21.15.78:0/3120543845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a0103c20 msgr2=0x7f34a0199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349effd640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a0103c20 0x7f34a0199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349effd640 1 -- 172.21.15.78:0/3120543845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a01a0c90 con 0x7f34a01029e0 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349f7fe640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a0108920 0x7f34a01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349effd640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a01029e0 0x7f34a01991c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3490005d00 tx=0x7f349000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349cff9640 1 -- 172.21.15.78:0/3120543845 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490017070 con 0x7f34a01029e0 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01a0f80 con 0x7f34a01029e0 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349cff9640 1 -- 172.21.15.78:0/3120543845 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3490009c30 con 0x7f34a01029e0 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f349cff9640 1 -- 172.21.15.78:0/3120543845 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349000f620 con 0x7f34a01029e0 2024-04-25T14:04:07.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.519+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a13e0 con 0x7f34a01029e0 2024-04-25T14:04:07.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.521+0000 7f34ac626740 1 Processor -- start 2024-04-25T14:04:07.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.521+0000 7f34ac626740 1 -- start start 2024-04-25T14:04:07.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f34ac626740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x56141348f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f34ac626740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130c44d0 0x56141348fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f34ac626740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130cc6b0 0x561413490a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f34ac626740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5614133f3d80 con 0x5614130cc6b0 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f34ac626740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614133ada20 con 0x5614130ed390 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f34ac626740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5614133efd40 con 0x5614130c44d0 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f349cff9640 1 -- 172.21.15.78:0/3120543845 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f349001a070 con 0x7f34a01029e0 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130cc6b0 0x561413490a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130cc6b0 0x561413490a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35220/0 (socket says 172.21.15.78:35220) 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 -- 172.21.15.78:0/1332236154 learned_addr learned my addr 172.21.15.78:0/1332236154 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f349f7fe640 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x56141348f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 -- 172.21.15.78:0/1332236154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130c44d0 msgr2=0x56141348fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f349effd640 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130c44d0 0x56141348fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130c44d0 0x56141348fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 -- 172.21.15.78:0/1332236154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 msgr2=0x56141348f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x56141348f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 -- 172.21.15.78:0/1332236154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614133b8c30 con 0x5614130cc6b0 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f349f7fe640 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x56141348f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f349effd640 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130c44d0 0x56141348fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f3497fff640 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130cc6b0 0x561413490a20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f348802a930 tx=0x7f3488032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b12a457bdb2f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.522+0000 7f349cff9640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f348c0afaf0 0x7f348c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f349f7fe640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f348c0afaf0 0x7f348c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34957fa640 1 -- 172.21.15.78:0/1332236154 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348803f070 con 0x5614130cc6b0 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f349cff9640 1 -- 172.21.15.78:0/3120543845 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f34900de730 con 0x7f34a01029e0 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34957fa640 1 -- 172.21.15.78:0/1332236154 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3488004040 con 0x5614130cc6b0 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34957fa640 1 -- 172.21.15.78:0/1332236154 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488035070 con 0x5614130cc6b0 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34ac626740 1 -- 172.21.15.78:0/1332236154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130cc6b0 msgr2=0x561413490a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34ac626740 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130cc6b0 0x561413490a20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f348802a930 tx=0x7f3488032ba0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f349f7fe640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f348c0afaf0 0x7f348c0b2000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x561413490860 tx=0x7f34980095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34ac626740 1 -- 172.21.15.78:0/1332236154 shutdown_connections 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34ac626740 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130cc6b0 0x561413490a20 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.523+0000 7f34ac626740 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130c44d0 0x56141348fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 --2- 172.21.15.78:0/1332236154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x56141348f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 -- 172.21.15.78:0/1332236154 >> 172.21.15.78:0/1332236154 conn(0x5614130e14e0 msgr2=0x56141348b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 -- 172.21.15.78:0/1332236154 shutdown_connections 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 -- 172.21.15.78:0/1332236154 wait complete. 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 -- start start 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 Processor -- start 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130c44d0 0x56141349b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x5614134a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5614134914d0 con 0x5614130c44d0 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56141337adc0 con 0x5614130ed390 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561413491650 con 0x5614130cc6b0 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f34ac626740 10 client.0 _ll_register_callbacks cb 0x561413396600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130c44d0 0x56141349b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130c44d0 0x56141349b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35224/0 (socket says 172.21.15.78:35224) 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.524+0000 7f349effd640 1 -- 172.21.15.78:0/105145254 learned_addr learned my addr 172.21.15.78:0/105145254 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f349f7fe640 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x5614134a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f349effd640 1 -- 172.21.15.78:0/105145254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 msgr2=0x56141349bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f349effd640 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f349effd640 1 -- 172.21.15.78:0/105145254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 msgr2=0x5614134a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f349effd640 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x5614134a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f349effd640 1 -- 172.21.15.78:0/105145254 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614131053f0 con 0x5614130c44d0 2024-04-25T14:04:07.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f349effd640 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130c44d0 0x56141349b700 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f34900a6b90 tx=0x7f349000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e2211de25040ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34867fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34867fc640 1 -- 172.21.15.78:0/105145254 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490021070 con 0x5614130c44d0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34867fc640 1 -- 172.21.15.78:0/105145254 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f34900dedf0 con 0x5614130c44d0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3488004040 con 0x5614130c44d0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34867fc640 1 -- 172.21.15.78:0/105145254 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f34900af3a0 con 0x5614130c44d0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34ac626740 10 client.16794 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5614131cf7d0 con 0x5614130c44d0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34867fc640 1 -- 172.21.15.78:0/105145254 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3488004040 con 0x5614130c44d0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34867fc640 1 -- 172.21.15.78:0/105145254 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f34900a2570 con 0x5614130c44d0 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34ac626740 10 client.16794 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34ac626740 10 client.16794 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:04:07.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.525+0000 7f34ac626740 20 client.16794 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.526+0000 7f34ac626740 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130abf60 0x5614134a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.526+0000 7f34ac626740 4 client.16794 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.526+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561413401990 con 0x5614130abf60 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.526+0000 7f3497fff640 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130abf60 0x5614134a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.526+0000 7f34867fc640 10 client.16794 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.526+0000 7f3497fff640 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130abf60 0x5614134a93c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0a468c9b1292fb8 server_cookie=896dccfde9a593ef in_seq=0 out_seq=0 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34867fc640 1 -- 172.21.15.78:0/105145254 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561413401990 con 0x5614130abf60 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34867fc640 10 client.16794 handle_command_reply: tid=0 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 client.16794 shutdown 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130c44d0 msgr2=0x56141349b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130c44d0 0x56141349b700 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f34900a6b90 tx=0x7f349000bf60 comp rx=0 tx=0).stop 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 shutdown_connections 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130ed390 0x5614134a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130abf60 0x5614134a93c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 --2- 172.21.15.78:0/105145254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130c44d0 0x56141349b700 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34867fc640 0 client.16794 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 >> 172.21.15.78:0/105145254 conn(0x5614130e14e0 msgr2=0x56141348a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 shutdown_connections 2024-04-25T14:04:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 1 -- 172.21.15.78:0/105145254 wait complete. 2024-04-25T14:04:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.527+0000 7f34ac626740 20 client.16794 trim_cache size 0 max 16384 2024-04-25T14:04:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 Processor -- start 2024-04-25T14:04:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 -- start start 2024-04-25T14:04:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01ab7d0 con 0x5614130abf60 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01a90a0 con 0x5614130c44d0 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f34ac626740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01a6ef0 con 0x5614130cc6b0 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35240/0 (socket says 172.21.15.78:35240) 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 -- 172.21.15.78:0/2130983831 learned_addr learned my addr 172.21.15.78:0/2130983831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f3497fff640 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 -- 172.21.15.78:0/2130983831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 msgr2=0x56141349b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 -- 172.21.15.78:0/2130983831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 msgr2=0x56141349b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.541+0000 7f349effd640 1 -- 172.21.15.78:0/2130983831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56141348bbb0 con 0x5614130abf60 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.542+0000 7f349effd640 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134983c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f34900a6900 tx=0x7f34900c12c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf35cf366ebe56c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.542+0000 7f3494ff9640 1 -- 172.21.15.78:0/2130983831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490021070 con 0x5614130abf60 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.542+0000 7f3494ff9640 1 -- 172.21.15.78:0/2130983831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34900a6710 con 0x5614130abf60 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.542+0000 7f3494ff9640 1 -- 172.21.15.78:0/2130983831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900a5920 con 0x5614130abf60 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.542+0000 7f34ac626740 1 -- 172.21.15.78:0/2130983831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 msgr2=0x5614134983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.542+0000 7f34ac626740 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134983c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f34900a6900 tx=0x7f34900c12c0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- 172.21.15.78:0/2130983831 shutdown_connections 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 --2- 172.21.15.78:0/2130983831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134983c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- 172.21.15.78:0/2130983831 >> 172.21.15.78:0/2130983831 conn(0x5614130e14e0 msgr2=0x561413497cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- 172.21.15.78:0/2130983831 shutdown_connections 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- 172.21.15.78:0/2130983831 wait complete. 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- start start 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 Processor -- start 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01a7560 con 0x5614130abf60 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01a89d0 con 0x5614130c44d0 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01a7c30 con 0x5614130cc6b0 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f34ac626740 10 client.0 _ll_register_callbacks cb 0x7f349000df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35252/0 (socket says 172.21.15.78:35252) 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349effd640 1 -- 172.21.15.78:0/4184039607 learned_addr learned my addr 172.21.15.78:0/4184039607 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f3497fff640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349f7fe640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349effd640 1 -- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 msgr2=0x56141349f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349effd640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349effd640 1 -- 172.21.15.78:0/4184039607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 msgr2=0x56141349ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.543+0000 7f349effd640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f349effd640 1 -- 172.21.15.78:0/4184039607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56141350f750 con 0x5614130abf60 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f3497fff640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f349effd640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134a21d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34900a2910 tx=0x7f349000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af729af4dc6eeafa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34867fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34867fc640 1 -- 172.21.15.78:0/4184039607 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490021040 con 0x5614130abf60 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34867fc640 1 -- 172.21.15.78:0/4184039607 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f349006d2b0 con 0x5614130abf60 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34900a6710 con 0x5614130abf60 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34867fc640 1 -- 172.21.15.78:0/4184039607 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3490009de0 con 0x5614130abf60 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 10 client.16800 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56141350f750 con 0x5614130abf60 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34867fc640 1 -- 172.21.15.78:0/4184039607 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34900a6710 con 0x5614130abf60 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34867fc640 1 -- 172.21.15.78:0/4184039607 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f34900ac260 con 0x5614130abf60 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 10 client.16800 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 10 client.16800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 20 client.16800 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:07.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130ed390 0x561413512f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 4 client.16800 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.544+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614133fb800 con 0x5614130ed390 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f3497fff640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130ed390 0x561413512f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f3497fff640 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130ed390 0x561413512f30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=863d01b80585fc6d server_cookie=7b9d696d4875494 in_seq=0 out_seq=0 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34867fc640 10 client.16800 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34867fc640 1 -- 172.21.15.78:0/4184039607 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614133fb800 con 0x5614130ed390 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34867fc640 10 client.16800 handle_command_reply: tid=0 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 client.16800 shutdown 2024-04-25T14:04:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 msgr2=0x5614134a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134a21d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34900a2910 tx=0x7f349000bf60 comp rx=0 tx=0).stop 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 shutdown_connections 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5614130ed390 0x561413512f30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614130cc6b0 0x56141349f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34867fc640 0 client.16800 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614130c44d0 0x56141349ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 --2- 172.21.15.78:0/4184039607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614130abf60 0x5614134a21d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 >> 172.21.15.78:0/4184039607 conn(0x5614130e14e0 msgr2=0x56141349b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 shutdown_connections 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.545+0000 7f34ac626740 1 -- 172.21.15.78:0/4184039607 wait complete. 2024-04-25T14:04:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34ac626740 20 client.16800 trim_cache size 0 max 16384 2024-04-25T14:04:07.554 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f348c0afaf0 msgr2=0x7f348c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f348c0afaf0 0x7f348c0b2000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x561413490860 tx=0x7f34980095b0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a01029e0 msgr2=0x7f34a01991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a01029e0 0x7f34a01991c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3490005d00 tx=0x7f349000dde0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 shutdown_connections 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34a0108920 0x7f34a01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34a0103c20 0x7f34a0199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f348c0afaf0 0x7f348c0b2000 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 --2- 172.21.15.78:0/3120543845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a01029e0 0x7f34a01991c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 >> 172.21.15.78:0/3120543845 conn(0x7f34a00fdbd0 msgr2=0x7f34a0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 shutdown_connections 2024-04-25T14:04:07.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.546+0000 7f34a5696640 1 -- 172.21.15.78:0/3120543845 wait complete. 2024-04-25T14:04:07.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:04:07.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:07 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3227484597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 -- 172.21.15.78:0/3050652122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e0102260 msgr2=0x7fd6e010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 --2- 172.21.15.78:0/3050652122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e0102260 0x7fd6e010bcc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009ca0 tx=0x7fd6d802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 -- 172.21.15.78:0/3050652122 shutdown_connections 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 --2- 172.21.15.78:0/3050652122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e0102260 0x7fd6e010bcc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 --2- 172.21.15.78:0/3050652122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0101760 0x7fd6e0101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 --2- 172.21.15.78:0/3050652122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e01050b0 0x7fd6e01054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 -- 172.21.15.78:0/3050652122 >> 172.21.15.78:0/3050652122 conn(0x7fd6e00fd380 msgr2=0x7fd6e00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 -- 172.21.15.78:0/3050652122 shutdown_connections 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.666+0000 7fd6e6364640 1 -- 172.21.15.78:0/3050652122 wait complete. 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 Processor -- start 2024-04-25T14:04:07.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 -- start start 2024-04-25T14:04:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e0101760 0x7fd6e019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0102260 0x7fd6e019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e01050b0 0x7fd6e01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e010f640 con 0x7fd6e01050b0 2024-04-25T14:04:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e010f4c0 con 0x7fd6e0101760 2024-04-25T14:04:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e6364640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e010f7c0 con 0x7fd6e0102260 2024-04-25T14:04:07.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e4901640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e01050b0 0x7fd6e01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e4901640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e01050b0 0x7fd6e01a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35278/0 (socket says 172.21.15.78:35278) 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6e4901640 1 -- 172.21.15.78:0/1600326534 learned_addr learned my addr 172.21.15.78:0/1600326534 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.667+0000 7fd6dffff640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e0101760 0x7fd6e019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0102260 0x7fd6e019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6e4901640 1 -- 172.21.15.78:0/1600326534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0102260 msgr2=0x7fd6e019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6e4901640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0102260 0x7fd6e019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6e4901640 1 -- 172.21.15.78:0/1600326534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e0101760 msgr2=0x7fd6e019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6e4901640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e0101760 0x7fd6e019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6e4901640 1 -- 172.21.15.78:0/1600326534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e01a1d30 con 0x7fd6e01050b0 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6dffff640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e0101760 0x7fd6e019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6e4901640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e01050b0 0x7fd6e01a16c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80097b0 tx=0x7fd6d8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6dd7fa640 1 -- 172.21.15.78:0/1600326534 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8003bf0 con 0x7fd6e01050b0 2024-04-25T14:04:07.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e01a1fc0 con 0x7fd6e01050b0 2024-04-25T14:04:07.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6dd7fa640 1 -- 172.21.15.78:0/1600326534 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d8003d90 con 0x7fd6e01050b0 2024-04-25T14:04:07.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.668+0000 7fd6dd7fa640 1 -- 172.21.15.78:0/1600326534 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d802fc30 con 0x7fd6e01050b0 2024-04-25T14:04:07.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.669+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e01a2480 con 0x7fd6e01050b0 2024-04-25T14:04:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 Processor -- start 2024-04-25T14:04:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6dd7fa640 1 -- 172.21.15.78:0/1600326534 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd6d8004040 con 0x7fd6e01050b0 2024-04-25T14:04:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 -- start start 2024-04-25T14:04:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd2d390 0x55d24c0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd044d0 0x55d24c0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d24c033d80 con 0x55d24bd044d0 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d24bfeda20 con 0x55d24bd2d390 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6ed214740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d24c02fd40 con 0x55d24bd0c6b0 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd044d0 0x55d24c0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.671+0000 7fd6dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd044d0 0x55d24c0cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35292/0 (socket says 172.21.15.78:35292) 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dffff640 1 -- 172.21.15.78:0/1953653201 learned_addr learned my addr 172.21.15.78:0/1953653201 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd2d390 0x55d24c0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dd7fa640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd6c80afaf0 0x7fd6c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dffff640 1 -- 172.21.15.78:0/1953653201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 msgr2=0x55d24c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dffff640 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dffff640 1 -- 172.21.15.78:0/1953653201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd2d390 msgr2=0x55d24c0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dffff640 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd2d390 0x55d24c0cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dffff640 1 -- 172.21.15.78:0/1953653201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d24bff8c30 con 0x55d24bd044d0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6e4901640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd6c80afaf0 0x7fd6c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dd7fa640 1 -- 172.21.15.78:0/1600326534 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd6d8100290 con 0x7fd6e01050b0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6dffff640 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd044d0 0x55d24c0cfd90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0010480 tx=0x7fd6d0010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95571ffec0912171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6c67fc640 1 -- 172.21.15.78:0/1953653201 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0005220 con 0x55d24bd044d0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6c67fc640 1 -- 172.21.15.78:0/1953653201 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d0004c50 con 0x55d24bd044d0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.672+0000 7fd6c67fc640 1 -- 172.21.15.78:0/1953653201 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0011a00 con 0x55d24bd044d0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 -- 172.21.15.78:0/1953653201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd044d0 msgr2=0x55d24c0cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6e4901640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd6c80afaf0 0x7fd6c80b2000 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80c59d0 tx=0x7fd6d80972b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd044d0 0x55d24c0cfd90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0010480 tx=0x7fd6d0010940 comp rx=0 tx=0).stop 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 -- 172.21.15.78:0/1953653201 shutdown_connections 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd044d0 0x55d24c0cfd90 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 --2- 172.21.15.78:0/1953653201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd2d390 0x55d24c0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 -- 172.21.15.78:0/1953653201 >> 172.21.15.78:0/1953653201 conn(0x55d24bd214e0 msgr2=0x55d24c0cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 -- 172.21.15.78:0/1953653201 shutdown_connections 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 -- 172.21.15.78:0/1953653201 wait complete. 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.673+0000 7fd6ed214740 1 -- start start 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 1 Processor -- start 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d24c0d1530 con 0x55d24bd2d390 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d24bfbadc0 con 0x55d24bd0c6b0 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d24c0d16b0 con 0x55d24bd044d0 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6ed214740 10 client.0 _ll_register_callbacks cb 0x55d24bfd6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0e0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35304/0 (socket says 172.21.15.78:35304) 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6df7fe640 1 -- 172.21.15.78:0/3957618638 learned_addr learned my addr 172.21.15.78:0/3957618638 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6dffff640 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.674+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6df7fe640 1 -- 172.21.15.78:0/3957618638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 msgr2=0x55d24c0d9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0d9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6df7fe640 1 -- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 msgr2=0x55d24c0d9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6df7fe640 1 -- 172.21.15.78:0/3957618638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d24bd453f0 con 0x55d24bd2d390 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0e0a30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4010da0 tx=0x7fd6d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d3591077ad8202 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/3957618638 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400fe60 con 0x55d24bd2d390 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/3957618638 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd6d4004030 con 0x55d24bd2d390 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d0004c50 con 0x55d24bd2d390 2024-04-25T14:04:07.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/3957618638 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd6d4004230 con 0x55d24bd2d390 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6ed214740 10 client.16812 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d24be0f7d0 con 0x55d24bd2d390 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/3957618638 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d0004c50 con 0x55d24bd2d390 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.675+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/3957618638 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd6d4020420 con 0x55d24bd2d390 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6ed214740 10 client.16812 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6ed214740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6ed214740 20 client.16812 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6ed214740 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcebf60 0x55d24c0e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6ed214740 4 client.16812 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d24c041990 con 0x55d24bcebf60 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6dffff640 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcebf60 0x55d24c0e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6dffff640 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcebf60 0x55d24c0e71f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b301c047a3ec457 server_cookie=74234e67a3555fca in_seq=0 out_seq=0 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.676+0000 7fd6bf7fe640 10 client.16812 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/3957618638 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d24c041990 con 0x55d24bcebf60 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6bf7fe640 10 client.16812 handle_command_reply: tid=0 2024-04-25T14:04:07.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 client.16812 shutdown 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 msgr2=0x55d24c0e0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0e0a30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4010da0 tx=0x7fd6d4004730 comp rx=0 tx=0).stop 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 shutdown_connections 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0e0a30 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0d9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcebf60 0x55d24c0e71f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 --2- 172.21.15.78:0/3957618638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0d9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 >> 172.21.15.78:0/3957618638 conn(0x55d24bd214e0 msgr2=0x55d24c0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6bf7fe640 0 client.16812 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 shutdown_connections 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 1 -- 172.21.15.78:0/3957618638 wait complete. 2024-04-25T14:04:07.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.677+0000 7fd6ed214740 20 client.16812 trim_cache size 0 max 16384 2024-04-25T14:04:07.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 Processor -- start 2024-04-25T14:04:07.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 -- start start 2024-04-25T14:04:07.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd044d0 0x55d24c0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01adba0 con 0x55d24bd2d390 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01ab470 con 0x55d24bd044d0 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6ed214740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01a92c0 con 0x55d24bd0c6b0 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0ec9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35308/0 (socket says 172.21.15.78:35308) 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 -- 172.21.15.78:0/2294881095 learned_addr learned my addr 172.21.15.78:0/2294881095 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6dffff640 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 -- 172.21.15.78:0/2294881095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 msgr2=0x55d24c0e09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0e09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 -- 172.21.15.78:0/2294881095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd044d0 msgr2=0x55d24c0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd044d0 0x55d24c0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 -- 172.21.15.78:0/2294881095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d24c0da6d0 con 0x55d24bd2d390 2024-04-25T14:04:07.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0ec9b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4010b00 tx=0x7fd6d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d215dc9a6a383c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.691+0000 7fd6c5ffb640 1 -- 172.21.15.78:0/2294881095 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4009d00 con 0x55d24bd2d390 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6c5ffb640 1 -- 172.21.15.78:0/2294881095 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d40043d0 con 0x55d24bd2d390 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6c5ffb640 1 -- 172.21.15.78:0/2294881095 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d402b910 con 0x55d24bd2d390 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 -- 172.21.15.78:0/2294881095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 msgr2=0x55d24c0ec9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0ec9b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4010b00 tx=0x7fd6d4005f90 comp rx=0 tx=0).stop 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 -- 172.21.15.78:0/2294881095 shutdown_connections 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c0ec9b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd0c6b0 0x55d24c0e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 --2- 172.21.15.78:0/2294881095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd044d0 0x55d24c0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 -- 172.21.15.78:0/2294881095 >> 172.21.15.78:0/2294881095 conn(0x55d24bd214e0 msgr2=0x55d24c0cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 -- 172.21.15.78:0/2294881095 shutdown_connections 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.692+0000 7fd6ed214740 1 -- 172.21.15.78:0/2294881095 wait complete. 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 -- start start 2024-04-25T14:04:07.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 Processor -- start 2024-04-25T14:04:07.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0dfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c14d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01a9930 con 0x55d24bd2d390 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01aada0 con 0x55d24bd0c6b0 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01aa000 con 0x55d24bd044d0 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6ed214740 10 client.0 _ll_register_callbacks cb 0x7fd6d4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c14d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35314/0 (socket says 172.21.15.78:35314) 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 -- 172.21.15.78:0/4074911217 learned_addr learned my addr 172.21.15.78:0/4074911217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0dfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6dffff640 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 -- 172.21.15.78:0/4074911217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 msgr2=0x55d24c0dfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0dfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 -- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 msgr2=0x55d24c0dffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:07.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.693+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0dffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6df7fe640 1 -- 172.21.15.78:0/4074911217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d24c14dac0 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6df7fe640 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c14d550 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400fff0 tx=0x7fd6d4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471160266f3ace20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/4074911217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40204a0 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/4074911217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd6d402b6b0 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d40043d0 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/4074911217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd6d402b8b0 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 10 client.16818 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d24c14ed50 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/4074911217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d40043d0 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/4074911217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd6d4038470 con 0x55d24bd2d390 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 10 client.16818 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:07.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 20 client.16818 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcc13a0 0x55d24c151710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 4 client.16818 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.694+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d24c03b800 con 0x55d24bcc13a0 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6dffff640 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcc13a0 0x55d24c151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6dffff640 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcc13a0 0x55d24c151710 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb82d9ba49407f01 server_cookie=5023d1f5feccebe9 in_seq=0 out_seq=0 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6bf7fe640 10 client.16818 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6bf7fe640 1 -- 172.21.15.78:0/4074911217 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d24c03b800 con 0x55d24bcc13a0 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6bf7fe640 10 client.16818 handle_command_reply: tid=0 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 client.16818 shutdown 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 msgr2=0x55d24c14d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c14d550 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400fff0 tx=0x7fd6d4005740 comp rx=0 tx=0).stop 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 shutdown_connections 2024-04-25T14:04:07.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d24bd2d390 0x55d24c14d550 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d24bd0c6b0 0x55d24c0dffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d24bcc13a0 0x55d24c151710 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 --2- 172.21.15.78:0/4074911217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d24bd044d0 0x55d24c0dfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6bf7fe640 0 client.16818 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 >> 172.21.15.78:0/4074911217 conn(0x55d24bd214e0 msgr2=0x55d24c0d7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 shutdown_connections 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.695+0000 7fd6ed214740 1 -- 172.21.15.78:0/4074911217 wait complete. 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6ed214740 20 client.16818 trim_cache size 0 max 16384 2024-04-25T14:04:07.702 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:04:07.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd6c80afaf0 msgr2=0x7fd6c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd6c80afaf0 0x7fd6c80b2000 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80c59d0 tx=0x7fd6d80972b0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e01050b0 msgr2=0x7fd6e01a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e01050b0 0x7fd6e01a16c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80097b0 tx=0x7fd6d8031840 comp rx=0 tx=0).stop 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 shutdown_connections 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd6c80afaf0 0x7fd6c80b2000 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6e01050b0 0x7fd6e01a16c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6e0102260 0x7fd6e019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 --2- 172.21.15.78:0/1600326534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6e0101760 0x7fd6e019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 >> 172.21.15.78:0/1600326534 conn(0x7fd6e00fd380 msgr2=0x7fd6e0107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 shutdown_connections 2024-04-25T14:04:07.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:07.696+0000 7fd6e6364640 1 -- 172.21.15.78:0/1600326534 wait complete. 2024-04-25T14:04:07.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:07 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3227484597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:08 smithi129 ceph-mon[32569]: pgmap v487: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:08.708 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:04:08.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:08.708 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:04:08.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:08 smithi078 ceph-mon[29163]: pgmap v487: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:08.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:08 smithi112 ceph-mon[32617]: pgmap v487: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 -- 172.21.15.78:0/2213583700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 msgr2=0x7feed410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 --2- 172.21.15.78:0/2213583700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed410bcc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ca0 tx=0x7feecc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 -- 172.21.15.78:0/2213583700 shutdown_connections 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 --2- 172.21.15.78:0/2213583700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed410bcc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 --2- 172.21.15.78:0/2213583700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feed4101760 0x7feed4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 --2- 172.21.15.78:0/2213583700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed41050b0 0x7feed41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 -- 172.21.15.78:0/2213583700 >> 172.21.15.78:0/2213583700 conn(0x7feed40fd380 msgr2=0x7feed40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 -- 172.21.15.78:0/2213583700 shutdown_connections 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 -- 172.21.15.78:0/2213583700 wait complete. 2024-04-25T14:04:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 Processor -- start 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.829+0000 7feedac8c640 1 -- start start 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feedac8c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feed4101760 0x7feed419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feedac8c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feedac8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed41050b0 0x7feed41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feedac8c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feed410f620 con 0x7feed4102260 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feedac8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feed410f4a0 con 0x7feed41050b0 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feedac8c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feed410f7a0 con 0x7feed4101760 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35334/0 (socket says 172.21.15.78:35334) 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 -- 172.21.15.78:0/618734173 learned_addr learned my addr 172.21.15.78:0/618734173 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed8a28640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feed4101760 0x7feed419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed9229640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed41050b0 0x7feed41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 -- 172.21.15.78:0/618734173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feed4101760 msgr2=0x7feed419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feed4101760 0x7feed419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 -- 172.21.15.78:0/618734173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed41050b0 msgr2=0x7feed41a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed41050b0 0x7feed41a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 -- 172.21.15.78:0/618734173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed41a1c60 con 0x7feed4102260 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed9229640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed41050b0 0x7feed41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed3fff640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed419a710 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feec800c920 tx=0x7feec800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed1ffb640 1 -- 172.21.15.78:0/618734173 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8007c70 con 0x7feed4102260 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed1ffb640 1 -- 172.21.15.78:0/618734173 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec8007e10 con 0x7feed4102260 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41a1f50 con 0x7feed4102260 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.830+0000 7feed1ffb640 1 -- 172.21.15.78:0/618734173 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8018820 con 0x7feed4102260 2024-04-25T14:04:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.831+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41a23c0 con 0x7feed4102260 2024-04-25T14:04:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feed1ffb640 1 -- 172.21.15.78:0/618734173 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7feec800f450 con 0x7feed4102260 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 Processor -- start 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 -- start start 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027828b410 0x56027862c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x56027862ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x560278633920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560278591b00 con 0x560278262550 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56027852d920 con 0x56027828b410 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feee1c10740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56027858dac0 con 0x56027826a730 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feed1ffb640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feebc0afaf0 0x7feebc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feed8a28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x56027862ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feed8a28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x56027862ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35350/0 (socket says 172.21.15.78:35350) 2024-04-25T14:04:08.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feed8a28640 1 -- 172.21.15.78:0/1237473933 learned_addr learned my addr 172.21.15.78:0/1237473933 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.833+0000 7feed9229640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feebc0afaf0 0x7feebc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed9229640 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027828b410 0x56027862c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed1ffb640 1 -- 172.21.15.78:0/618734173 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7feec80dee30 con 0x7feed4102260 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed3fff640 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x560278633920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed8a28640 1 -- 172.21.15.78:0/1237473933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 msgr2=0x560278633920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed8a28640 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x560278633920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed8a28640 1 -- 172.21.15.78:0/1237473933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027828b410 msgr2=0x56027862c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed8a28640 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027828b410 0x56027862c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed8a28640 1 -- 172.21.15.78:0/1237473933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56027836b7e0 con 0x560278262550 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed3fff640 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x560278633920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed9229640 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027828b410 0x56027862c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed8a28640 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x56027862ca20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7feec4009bf0 tx=0x7feec40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d3ee71ee47fb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feed9229640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feebc0afaf0 0x7feebc0b2000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56027862d670 tx=0x7feecc031a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feebaffd640 1 -- 172.21.15.78:0/1237473933 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4004030 con 0x560278262550 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feebaffd640 1 -- 172.21.15.78:0/1237473933 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec40041d0 con 0x560278262550 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.834+0000 7feebaffd640 1 -- 172.21.15.78:0/1237473933 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4012670 con 0x560278262550 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 -- 172.21.15.78:0/1237473933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 msgr2=0x56027862ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x56027862ca20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7feec4009bf0 tx=0x7feec40117b0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 -- 172.21.15.78:0/1237473933 shutdown_connections 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x560278633920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x56027862ca20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 --2- 172.21.15.78:0/1237473933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027828b410 0x56027862c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 -- 172.21.15.78:0/1237473933 >> 172.21.15.78:0/1237473933 conn(0x56027827f560 msgr2=0x560278627e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 -- 172.21.15.78:0/1237473933 shutdown_connections 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 -- 172.21.15.78:0/1237473933 wait complete. 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 -- start start 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.835+0000 7feee1c10740 1 Processor -- start 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feee1c10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x560278636240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feee1c10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278636780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feee1c10740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027828b410 0x56027863d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feee1c10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560278634370 con 0x560278262550 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feee1c10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56027852b5d0 con 0x56027826a730 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feee1c10740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5602786344f0 con 0x56027828b410 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feee1c10740 10 client.0 _ll_register_callbacks cb 0x560278519080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x560278636240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x560278636240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35362/0 (socket says 172.21.15.78:35362) 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 -- 172.21.15.78:0/2307242155 learned_addr learned my addr 172.21.15.78:0/2307242155 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed9229640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027828b410 0x56027863d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed3fff640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278636780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 -- 172.21.15.78:0/2307242155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027828b410 msgr2=0x56027863d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027828b410 0x56027863d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 -- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 msgr2=0x560278636780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278636780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed8a28640 1 -- 172.21.15.78:0/2307242155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602783cbf70 con 0x560278262550 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed9229640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027828b410 0x56027863d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.836+0000 7feed3fff640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278636780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feed8a28640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x560278636240 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec4003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acab5a8118daea98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feeb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feeb3fff640 1 -- 172.21.15.78:0/2307242155 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec401f040 con 0x560278262550 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feeb3fff640 1 -- 172.21.15.78:0/2307242155 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7feec4004380 con 0x560278262550 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec40041d0 con 0x560278262550 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feeb3fff640 1 -- 172.21.15.78:0/2307242155 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feec4005050 con 0x560278262550 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 10 client.16830 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560278263170 con 0x560278262550 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feeb3fff640 1 -- 172.21.15.78:0/2307242155 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec40041d0 con 0x560278262550 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feeb3fff640 1 -- 172.21.15.78:0/2307242155 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7feec4012450 con 0x560278262550 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 10 client.16830 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 20 client.16830 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027824a1f0 0x560278643840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 4 client.16830 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.837+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56027859f710 con 0x56027824a1f0 2024-04-25T14:04:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.838+0000 7feed3fff640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027824a1f0 0x560278643840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.838+0000 7feeb3fff640 10 client.16830 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.838+0000 7feed3fff640 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027824a1f0 0x560278643840 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437738a3027313eb server_cookie=9445781be88d6fba in_seq=0 out_seq=0 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feeb3fff640 1 -- 172.21.15.78:0/2307242155 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56027859f710 con 0x56027824a1f0 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feeb3fff640 10 client.16830 handle_command_reply: tid=0 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 client.16830 shutdown 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 msgr2=0x560278636240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x560278636240 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7feec4000c00 tx=0x7feec4003bb0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 shutdown_connections 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027824a1f0 0x560278643840 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027828b410 0x56027863d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278636780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feeb3fff640 0 client.16830 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 --2- 172.21.15.78:0/2307242155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560278262550 0x560278636240 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 >> 172.21.15.78:0/2307242155 conn(0x56027827f560 msgr2=0x560278626ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 shutdown_connections 2024-04-25T14:04:08.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 1 -- 172.21.15.78:0/2307242155 wait complete. 2024-04-25T14:04:08.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.839+0000 7feee1c10740 20 client.16830 trim_cache size 0 max 16384 2024-04-25T14:04:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 Processor -- start 2024-04-25T14:04:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 -- start start 2024-04-25T14:04:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560278262550 0x560278636520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278647ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x560278648530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feed41adb40 con 0x56027828b410 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feed41ab410 con 0x56027826a730 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feee1c10740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feed41a9260 con 0x560278262550 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x560278648530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x560278648530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35364/0 (socket says 172.21.15.78:35364) 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 -- 172.21.15.78:0/1639918401 learned_addr learned my addr 172.21.15.78:0/1639918401 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed8a28640 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560278262550 0x560278636520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed3fff640 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278647ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 -- 172.21.15.78:0/1639918401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560278262550 msgr2=0x560278636520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560278262550 0x560278636520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 -- 172.21.15.78:0/1639918401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 msgr2=0x560278647ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278647ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.857+0000 7feed9229640 1 -- 172.21.15.78:0/1639918401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560278633a80 con 0x56027828b410 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.858+0000 7feed9229640 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x560278648530 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7feecc031520 tx=0x7feecc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5caf580e84ec1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.858+0000 7feeba7fc640 1 -- 172.21.15.78:0/1639918401 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc004040 con 0x56027828b410 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.858+0000 7feeba7fc640 1 -- 172.21.15.78:0/1639918401 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc036030 con 0x56027828b410 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.858+0000 7feeba7fc640 1 -- 172.21.15.78:0/1639918401 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc02fc00 con 0x56027828b410 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- 172.21.15.78:0/1639918401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 msgr2=0x560278648530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x560278648530 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7feecc031520 tx=0x7feecc005fd0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- 172.21.15.78:0/1639918401 shutdown_connections 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x560278648530 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56027826a730 0x560278647ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 --2- 172.21.15.78:0/1639918401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560278262550 0x560278636520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- 172.21.15.78:0/1639918401 >> 172.21.15.78:0/1639918401 conn(0x56027827f560 msgr2=0x560278635e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- 172.21.15.78:0/1639918401 shutdown_connections 2024-04-25T14:04:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- 172.21.15.78:0/1639918401 wait complete. 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- start start 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 Processor -- start 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560278262550 0x56027863aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x56027863afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x56027863bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feed41a98d0 con 0x56027828b410 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feed41aad40 con 0x560278262550 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feed41a9fa0 con 0x56027826a730 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.859+0000 7feee1c10740 10 client.0 _ll_register_callbacks cb 0x7feec4003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x56027863bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x56027863bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35378/0 (socket says 172.21.15.78:35378) 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 -- 172.21.15.78:0/2636115143 learned_addr learned my addr 172.21.15.78:0/2636115143 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed8a28640 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560278262550 0x56027863aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed3fff640 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x56027863afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 -- 172.21.15.78:0/2636115143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 msgr2=0x56027863afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x56027863afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 -- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560278262550 msgr2=0x56027863aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560278262550 0x56027863aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 -- 172.21.15.78:0/2636115143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56027863c480 con 0x56027828b410 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feed9229640 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x56027863bec0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7feecc02f6a0 tx=0x7feecc02f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908f33d610edddaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feeb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feeb3fff640 1 -- 172.21.15.78:0/2636115143 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc02fd40 con 0x56027828b410 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feeb3fff640 1 -- 172.21.15.78:0/2636115143 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7feecc004040 con 0x56027828b410 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc036030 con 0x56027828b410 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feeb3fff640 1 -- 172.21.15.78:0/2636115143 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feecc004240 con 0x56027828b410 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feee1c10740 10 client.16836 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5602786ac140 con 0x56027828b410 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.860+0000 7feeb3fff640 1 -- 172.21.15.78:0/2636115143 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc036030 con 0x56027828b410 2024-04-25T14:04:08.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feeb3fff640 1 -- 172.21.15.78:0/2636115143 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7feecc049040 con 0x56027828b410 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feee1c10740 10 client.16836 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feee1c10740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feee1c10740 20 client.16836 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feee1c10740 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027821f630 0x5602786aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feee1c10740 4 client.16836 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560278599580 con 0x56027821f630 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feed8a28640 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027821f630 0x5602786aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feeb3fff640 10 client.16836 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:08.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.861+0000 7feed8a28640 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027821f630 0x5602786aeac0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4222cae96be52cc9 server_cookie=3a255a54ec065a83 in_seq=0 out_seq=0 2024-04-25T14:04:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.862+0000 7feeb3fff640 1 -- 172.21.15.78:0/2636115143 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560278599580 con 0x56027821f630 2024-04-25T14:04:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.862+0000 7feeb3fff640 10 client.16836 handle_command_reply: tid=0 2024-04-25T14:04:08.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.862+0000 7feee1c10740 1 client.16836 shutdown 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 msgr2=0x56027863bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x56027863bec0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7feecc02f6a0 tx=0x7feecc02f940 comp rx=0 tx=0).stop 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 shutdown_connections 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56027828b410 0x56027863bec0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56027821f630 0x5602786aeac0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56027826a730 0x56027863afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 --2- 172.21.15.78:0/2636115143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560278262550 0x56027863aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feeb3fff640 0 client.16836 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 >> 172.21.15.78:0/2636115143 conn(0x56027827f560 msgr2=0x56027862ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 shutdown_connections 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 1 -- 172.21.15.78:0/2636115143 wait complete. 2024-04-25T14:04:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feee1c10740 20 client.16836 trim_cache size 0 max 16384 2024-04-25T14:04:08.869 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:04:08.869 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:04:08.869 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "d2bbc916-abd5-4998-9f69-d283d3a493b1", 2024-04-25T14:04:08.869 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:04:08.869 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feebc0afaf0 msgr2=0x7feebc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feebc0afaf0 0x7feebc0b2000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56027862d670 tx=0x7feecc031a20 comp rx=0 tx=0).stop 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 msgr2=0x7feed419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed419a710 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feec800c920 tx=0x7feec800cde0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 shutdown_connections 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feed41050b0 0x7feed41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7feebc0afaf0 0x7feebc0b2000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4102260 0x7feed419a710 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 --2- 172.21.15.78:0/618734173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feed4101760 0x7feed419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.863+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 >> 172.21.15.78:0/618734173 conn(0x7feed40fd380 msgr2=0x7feed4107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.864+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 shutdown_connections 2024-04-25T14:04:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:08.864+0000 7feedac8c640 1 -- 172.21.15.78:0/618734173 wait complete. 2024-04-25T14:04:08.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:08.874 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:09.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 -- 172.21.15.78:0/433320100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c103960 msgr2=0x7fe32c103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 --2- 172.21.15.78:0/433320100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c103960 0x7fe32c103df0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009900 tx=0x7fe31c02a410 comp rx=0 tx=0).stop 2024-04-25T14:04:09.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 -- 172.21.15.78:0/433320100 shutdown_connections 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 --2- 172.21.15.78:0/433320100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c104480 0x7fe32c10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 --2- 172.21.15.78:0/433320100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c103960 0x7fe32c103df0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 --2- 172.21.15.78:0/433320100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32c102f80 0x7fe32c103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 -- 172.21.15.78:0/433320100 >> 172.21.15.78:0/433320100 conn(0x7fe32c0fd380 msgr2=0x7fe32c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 -- 172.21.15.78:0/433320100 shutdown_connections 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.025+0000 7fe33389f640 1 -- 172.21.15.78:0/433320100 wait complete. 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 Processor -- start 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 -- start start 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c102f80 0x7fe32c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32c103960 0x7fe32c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c104480 0x7fe32c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c10e550 con 0x7fe32c104480 2024-04-25T14:04:09.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c10e3d0 con 0x7fe32c102f80 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe33389f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c10e6d0 con 0x7fe32c103960 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c104480 0x7fe32c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c104480 0x7fe32c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35390/0 (socket says 172.21.15.78:35390) 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 -- 172.21.15.78:0/2663850577 learned_addr learned my addr 172.21.15.78:0/2663850577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe330e3a640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32c103960 0x7fe32c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 -- 172.21.15.78:0/2663850577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32c103960 msgr2=0x7fe32c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32c103960 0x7fe32c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 -- 172.21.15.78:0/2663850577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c102f80 msgr2=0x7fe32c199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c102f80 0x7fe32c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 -- 172.21.15.78:0/2663850577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c1a0c90 con 0x7fe32c104480 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe330e3a640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32c103960 0x7fe32c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe331e3c640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c104480 0x7fe32c1a0620 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe32400cad0 tx=0x7fe32400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.026+0000 7fe3227fc640 1 -- 172.21.15.78:0/2663850577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32400ae20 con 0x7fe32c104480 2024-04-25T14:04:09.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.027+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c1a0f80 con 0x7fe32c104480 2024-04-25T14:04:09.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.027+0000 7fe3227fc640 1 -- 172.21.15.78:0/2663850577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe324005020 con 0x7fe32c104480 2024-04-25T14:04:09.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.027+0000 7fe3227fc640 1 -- 172.21.15.78:0/2663850577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32400f6b0 con 0x7fe32c104480 2024-04-25T14:04:09.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.027+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1a13e0 con 0x7fe32c104480 2024-04-25T14:04:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.029+0000 7fe3227fc640 1 -- 172.21.15.78:0/2663850577 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe3240051d0 con 0x7fe32c104480 2024-04-25T14:04:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.029+0000 7fe33a82e740 1 Processor -- start 2024-04-25T14:04:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.029+0000 7fe33a82e740 1 -- start start 2024-04-25T14:04:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcd94140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd92630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 0x55bfdcd92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe33a82e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bfdccf7d80 con 0x55bfdc9f1390 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe33a82e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bfdccb18b0 con 0x55bfdc9c84d0 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe33a82e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bfdccf3d40 con 0x55bfdc9d06b0 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcd94140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcd94140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35398/0 (socket says 172.21.15.78:35398) 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 -- 172.21.15.78:0/525382542 learned_addr learned my addr 172.21.15.78:0/525382542 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe3227fc640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe3180afaf0 0x7fe3180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe33163b640 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd92630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 -- 172.21.15.78:0/525382542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 msgr2=0x55bfdcd92d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 0x55bfdcd92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 -- 172.21.15.78:0/525382542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 msgr2=0x55bfdcd92630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd92630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 -- 172.21.15.78:0/525382542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfdcc82a40 con 0x55bfdc9f1390 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe331e3c640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe3180afaf0 0x7fe3180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe3227fc640 1 -- 172.21.15.78:0/2663850577 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe3240ded10 con 0x7fe32c104480 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe330e3a640 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcd94140 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005940 tx=0x7fe31c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0f16d2330ab1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe3177fe640 1 -- 172.21.15.78:0/525382542 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c040070 con 0x55bfdc9f1390 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe3177fe640 1 -- 172.21.15.78:0/525382542 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c0317a0 con 0x55bfdc9f1390 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.030+0000 7fe3177fe640 1 -- 172.21.15.78:0/525382542 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c035050 con 0x55bfdc9f1390 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 -- 172.21.15.78:0/525382542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 msgr2=0x55bfdcd94140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe331e3c640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe3180afaf0 0x7fe3180b2000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe3240a3260 tx=0x7fe3240d5930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcd94140 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005940 tx=0x7fe31c032c60 comp rx=0 tx=0).stop 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 -- 172.21.15.78:0/525382542 shutdown_connections 2024-04-25T14:04:09.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 0x55bfdcd92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd92630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 --2- 172.21.15.78:0/525382542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcd94140 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 -- 172.21.15.78:0/525382542 >> 172.21.15.78:0/525382542 conn(0x55bfdc9e54e0 msgr2=0x55bfdcd8e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 -- 172.21.15.78:0/525382542 shutdown_connections 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.031+0000 7fe33a82e740 1 -- 172.21.15.78:0/525382542 wait complete. 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 -- start start 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 Processor -- start 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bfdcd9eda0 con 0x55bfdc9f1390 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bfdcc7ed10 con 0x55bfdc9d06b0 2024-04-25T14:04:09.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bfdcd9ef20 con 0x55bfdc9c84d0 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33a82e740 10 client.0 _ll_register_callbacks cb 0x55bfdccab210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35414/0 (socket says 172.21.15.78:35414) 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe330e3a640 1 -- 172.21.15.78:0/1410337579 learned_addr learned my addr 172.21.15.78:0/1410337579 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.032+0000 7fe33163b640 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe330e3a640 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe330e3a640 1 -- 172.21.15.78:0/1410337579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9c84d0 msgr2=0x55bfdcda0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe330e3a640 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe330e3a640 1 -- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 msgr2=0x55bfdcda1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe330e3a640 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe330e3a640 1 -- 172.21.15.78:0/1410337579 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfdcad37d0 con 0x55bfdc9f1390 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe330e3a640 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda81c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005ab0 tx=0x7fe31c032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f879a16e10ce2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe314ff9640 1 -- 172.21.15.78:0/1410337579 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c035030 con 0x55bfdc9f1390 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe314ff9640 1 -- 172.21.15.78:0/1410337579 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe31c0046c0 con 0x55bfdc9f1390 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31c0317a0 con 0x55bfdc9f1390 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe314ff9640 1 -- 172.21.15.78:0/1410337579 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe31c040040 con 0x55bfdc9f1390 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe33a82e740 10 client.16848 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bfdca093f0 con 0x55bfdc9f1390 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe314ff9640 1 -- 172.21.15.78:0/1410337579 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c0317a0 con 0x55bfdc9f1390 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe314ff9640 1 -- 172.21.15.78:0/1410337579 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe31c003be0 con 0x55bfdc9f1390 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe33a82e740 10 client.16848 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe33a82e740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.033+0000 7fe33a82e740 20 client.16848 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.034+0000 7fe33a82e740 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9aff60 0x55bfdcdae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.034+0000 7fe33a82e740 4 client.16848 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.034+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfdcd05990 con 0x55bfdc9aff60 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.034+0000 7fe33163b640 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9aff60 0x55bfdcdae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.034+0000 7fe314ff9640 10 client.16848 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.034+0000 7fe33163b640 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9aff60 0x55bfdcdae260 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9aaefe22453c39 server_cookie=a1c4ce8df898d086 in_seq=0 out_seq=0 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe314ff9640 1 -- 172.21.15.78:0/1410337579 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfdcd05990 con 0x55bfdc9aff60 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe314ff9640 10 client.16848 handle_command_reply: tid=0 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 client.16848 shutdown 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 msgr2=0x55bfdcda81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda81c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005ab0 tx=0x7fe31c032f10 comp rx=0 tx=0).stop 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 shutdown_connections 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda81c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9aff60 0x55bfdcdae260 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 --2- 172.21.15.78:0/1410337579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe314ff9640 0 client.16848 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 >> 172.21.15.78:0/1410337579 conn(0x55bfdc9e54e0 msgr2=0x55bfdcd8d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 shutdown_connections 2024-04-25T14:04:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 1 -- 172.21.15.78:0/1410337579 wait complete. 2024-04-25T14:04:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.035+0000 7fe33a82e740 20 client.16848 trim_cache size 0 max 16384 2024-04-25T14:04:09.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 Processor -- start 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 -- start start 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd9bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1ab8d0 con 0x55bfdc9f1390 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1a91a0 con 0x55bfdc9c84d0 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33a82e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1a6ff0 con 0x55bfdc9d06b0 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35422/0 (socket says 172.21.15.78:35422) 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe330e3a640 1 -- 172.21.15.78:0/537786170 learned_addr learned my addr 172.21.15.78:0/537786170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe330e3a640 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd9bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.053+0000 7fe33163b640 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe330e3a640 1 -- 172.21.15.78:0/537786170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 msgr2=0x55bfdcda80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe330e3a640 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe330e3a640 1 -- 172.21.15.78:0/537786170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 msgr2=0x55bfdcd9bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe330e3a640 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd9bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe330e3a640 1 -- 172.21.15.78:0/537786170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfdc9c90f0 con 0x55bfdc9f1390 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe330e3a640 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda8620 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x55bfdcd898b0 tx=0x7fe31c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98066495378ffcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe316ffd640 1 -- 172.21.15.78:0/537786170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c035070 con 0x55bfdc9f1390 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe316ffd640 1 -- 172.21.15.78:0/537786170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c005dc0 con 0x55bfdc9f1390 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.054+0000 7fe316ffd640 1 -- 172.21.15.78:0/537786170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c031c90 con 0x55bfdc9f1390 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 -- 172.21.15.78:0/537786170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 msgr2=0x55bfdcda8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda8620 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x55bfdcd898b0 tx=0x7fe31c005250 comp rx=0 tx=0).stop 2024-04-25T14:04:09.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 -- 172.21.15.78:0/537786170 shutdown_connections 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9f1390 0x55bfdcda8620 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 --2- 172.21.15.78:0/537786170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9c84d0 0x55bfdcd9bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 -- 172.21.15.78:0/537786170 >> 172.21.15.78:0/537786170 conn(0x55bfdc9e54e0 msgr2=0x55bfdcd8dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 -- 172.21.15.78:0/537786170 shutdown_connections 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 -- 172.21.15.78:0/537786170 wait complete. 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 -- start start 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 Processor -- start 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.055+0000 7fe33a82e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9f1390 0x55bfdce144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33a82e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1a7660 con 0x55bfdc9c84d0 2024-04-25T14:04:09.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33a82e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1a8ad0 con 0x55bfdc9d06b0 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33a82e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1a7d30 con 0x55bfdc9f1390 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33a82e740 10 client.0 _ll_register_callbacks cb 0x7fe31c032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35430/0 (socket says 172.21.15.78:35430) 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 -- 172.21.15.78:0/393842305 learned_addr learned my addr 172.21.15.78:0/393842305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33163b640 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9f1390 0x55bfdce144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 -- 172.21.15.78:0/393842305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9f1390 msgr2=0x55bfdce144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9f1390 0x55bfdce144d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 -- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 msgr2=0x55bfdcda5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 -- 172.21.15.78:0/393842305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfdccb84b0 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe330e3a640 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda5740 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031870 tx=0x7fe31c0315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43eaa690b5116388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe314ff9640 1 -- 172.21.15.78:0/393842305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c035030 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe314ff9640 1 -- 172.21.15.78:0/393842305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe31c004350 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31c005dc0 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe314ff9640 1 -- 172.21.15.78:0/393842305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe31c045a60 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33a82e740 10 client.16854 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.056+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bfdce157f0 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe314ff9640 1 -- 172.21.15.78:0/393842305 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c005dc0 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe314ff9640 1 -- 172.21.15.78:0/393842305 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe31c0039b0 con 0x55bfdc9c84d0 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33a82e740 10 client.16854 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:09.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33a82e740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33a82e740 20 client.16854 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33a82e740 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9853a0 0x55bfdce18040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33a82e740 4 client.16854 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bfdccff800 con 0x55bfdc9853a0 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33163b640 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9853a0 0x55bfdce18040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe314ff9640 10 client.16854 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.057+0000 7fe33163b640 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9853a0 0x55bfdce18040 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6399c8427bef9922 server_cookie=5ba3cb60ce6cd1a3 in_seq=0 out_seq=0 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe314ff9640 1 -- 172.21.15.78:0/393842305 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bfdccff800 con 0x55bfdc9853a0 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe314ff9640 10 client.16854 handle_command_reply: tid=0 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 client.16854 shutdown 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9c84d0 msgr2=0x55bfdcda5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda5740 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031870 tx=0x7fe31c0315a0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 shutdown_connections 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bfdc9853a0 0x55bfdce18040 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bfdc9f1390 0x55bfdce144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe314ff9640 0 client.16854 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bfdc9d06b0 0x55bfdcda5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 --2- 172.21.15.78:0/393842305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfdc9c84d0 0x55bfdcda5740 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 >> 172.21.15.78:0/393842305 conn(0x55bfdc9e54e0 msgr2=0x55bfdcdb2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 shutdown_connections 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 1 -- 172.21.15.78:0/393842305 wait complete. 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.058+0000 7fe33a82e740 20 client.16854 trim_cache size 0 max 16384 2024-04-25T14:04:09.065 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: "d2bbc916-abd5-4998-9f69-d283d3a493b1": { 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "d2bbc916-abd5-4998-9f69-d283d3a493b1", 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:04:09.066 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:04:09.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe3180afaf0 msgr2=0x7fe3180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe3180afaf0 0x7fe3180b2000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe3240a3260 tx=0x7fe3240d5930 comp rx=0 tx=0).stop 2024-04-25T14:04:09.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c104480 msgr2=0x7fe32c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:09.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c104480 0x7fe32c1a0620 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe32400cad0 tx=0x7fe32400cf90 comp rx=0 tx=0).stop 2024-04-25T14:04:09.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 shutdown_connections 2024-04-25T14:04:09.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe3180afaf0 0x7fe3180b2000 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe32c104480 0x7fe32c1a0620 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe32c103960 0x7fe32c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 --2- 172.21.15.78:0/2663850577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe32c102f80 0x7fe32c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:09.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 >> 172.21.15.78:0/2663850577 conn(0x7fe32c0fd380 msgr2=0x7fe32c104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:09.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 shutdown_connections 2024-04-25T14:04:09.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:09.059+0000 7fe33389f640 1 -- 172.21.15.78:0/2663850577 wait complete. 2024-04-25T14:04:09.070 INFO:tasks.cephfs.filesystem:scrub status for tag:d2bbc916-abd5-4998-9f69-d283d3a493b1 - {'path': '/', 'tag': 'd2bbc916-abd5-4998-9f69-d283d3a493b1', 'options': 'recursive,force'} 2024-04-25T14:04:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:09 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:04:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:09 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:09 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:04:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:09 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:09.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:09 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:04:09.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:09 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:09 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:04:09.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:09 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:09.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:09 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:04:09.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:09 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:09.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:09 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:04:09.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:09 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:10 smithi129 ceph-mon[32569]: pgmap v488: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:10 smithi078 ceph-mon[29163]: pgmap v488: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:10.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:10 smithi112 ceph-mon[32617]: pgmap v488: 97 pgs: 97 active+clean; 3.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:11.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:11 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:04:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:11 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:04:11.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:11 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:04:12.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:12 smithi129 ceph-mon[32569]: pgmap v489: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:12 smithi078 ceph-mon[29163]: pgmap v489: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:12.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:12 smithi112 ceph-mon[32617]: pgmap v489: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:14 smithi129 ceph-mon[32569]: pgmap v490: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:14 smithi078 ceph-mon[29163]: pgmap v490: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:14 smithi112 ceph-mon[32617]: pgmap v490: 97 pgs: 97 active+clean; 3.2 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:16.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:16 smithi129 ceph-mon[32569]: pgmap v491: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:04:16.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:16 smithi078 ceph-mon[29163]: pgmap v491: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:04:16.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:16 smithi112 ceph-mon[32617]: pgmap v491: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:04:18.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:18 smithi129 ceph-mon[32569]: pgmap v492: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:18.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:18 smithi078 ceph-mon[29163]: pgmap v492: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:18.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:18 smithi112 ceph-mon[32617]: pgmap v492: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 -- 172.21.15.78:0/3051481196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 msgr2=0x7f11ac104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 --2- 172.21.15.78:0/3051481196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac104050 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f119c009900 tx=0x7f119c02a410 comp rx=0 tx=0).stop 2024-04-25T14:04:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 -- 172.21.15.78:0/3051481196 shutdown_connections 2024-04-25T14:04:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 --2- 172.21.15.78:0/3051481196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11ac108940 0x7f11ac10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 --2- 172.21.15.78:0/3051481196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac104050 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 --2- 172.21.15.78:0/3051481196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac102a00 0x7f11ac103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 -- 172.21.15.78:0/3051481196 >> 172.21.15.78:0/3051481196 conn(0x7f11ac0fdbd0 msgr2=0x7f11ac1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 -- 172.21.15.78:0/3051481196 shutdown_connections 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.015+0000 7f11b3296640 1 -- 172.21.15.78:0/3051481196 wait complete. 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 Processor -- start 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 -- start start 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11ac102a00 0x7f11ac075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac108940 0x7f11ac1a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10e500 con 0x7f11ac103c40 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10e380 con 0x7f11ac108940 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b3296640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac10e680 con 0x7f11ac102a00 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac1a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53404/0 (socket says 172.21.15.78:53404) 2024-04-25T14:04:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 -- 172.21.15.78:0/2022039479 learned_addr learned my addr 172.21.15.78:0/2022039479 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b1032640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11ac102a00 0x7f11ac075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b1833640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac108940 0x7f11ac1a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 -- 172.21.15.78:0/2022039479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11ac102a00 msgr2=0x7f11ac075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11ac102a00 0x7f11ac075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 -- 172.21.15.78:0/2022039479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac108940 msgr2=0x7f11ac1a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac108940 0x7f11ac1a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.016+0000 7f11b0831640 1 -- 172.21.15.78:0/2022039479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ac1a3f70 con 0x7f11ac103c40 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.017+0000 7f11b0831640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac1a13e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f119c006010 tx=0x7f119c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.017+0000 7f11a27fc640 1 -- 172.21.15.78:0/2022039479 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c03f030 con 0x7f11ac103c40 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.017+0000 7f11a27fc640 1 -- 172.21.15.78:0/2022039479 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f119c003cb0 con 0x7f11ac103c40 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.017+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ac1a4200 con 0x7f11ac103c40 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.017+0000 7f11a27fc640 1 -- 172.21.15.78:0/2022039479 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035070 con 0x7f11ac103c40 2024-04-25T14:04:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.017+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ac1a46f0 con 0x7f11ac103c40 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.019+0000 7f11ba229740 1 Processor -- start 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.019+0000 7f11ba229740 1 -- start start 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11a27fc640 1 -- 172.21.15.78:0/2022039479 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f119c042030 con 0x7f11ac103c40 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11ba229740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11ba229740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11ba229740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acafdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11ba229740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c58ac0e460 con 0x55c58a8df2f0 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11ba229740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c58abc8590 con 0x55c58a8e74d0 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11ba229740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c58ac0a420 con 0x55c58a9081b0 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53414/0 (socket says 172.21.15.78:53414) 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 -- 172.21.15.78:0/3214631070 learned_addr learned my addr 172.21.15.78:0/3214631070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1833640 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b0831640 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acafdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 -- 172.21.15.78:0/3214631070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 msgr2=0x55c58aca8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 -- 172.21.15.78:0/3214631070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 msgr2=0x55c58acafdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acafdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 -- 172.21.15.78:0/3214631070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c58abd50d0 con 0x55c58a8df2f0 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1032640 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca8ef0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11a800abc0 tx=0x7f11a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df76192c070cde2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11a27fc640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11980afae0 0x7f11980b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11977fe640 1 -- 172.21.15.78:0/3214631070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8004220 con 0x55c58a8df2f0 2024-04-25T14:04:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11b1833640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11980afae0 0x7f11980b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.020+0000 7f11977fe640 1 -- 172.21.15.78:0/3214631070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a80043c0 con 0x55c58a8df2f0 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11977fe640 1 -- 172.21.15.78:0/3214631070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a80187b0 con 0x55c58a8df2f0 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11a27fc640 1 -- 172.21.15.78:0/2022039479 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f119c0ff290 con 0x7f11ac103c40 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 -- 172.21.15.78:0/3214631070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 msgr2=0x55c58aca8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11b1833640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11980afae0 0x7f11980b1ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c58aca9b40 tx=0x7f11a40097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca8ef0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f11a800abc0 tx=0x7f11a800f790 comp rx=0 tx=0).stop 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 -- 172.21.15.78:0/3214631070 shutdown_connections 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acafdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca8ef0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 --2- 172.21.15.78:0/3214631070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 -- 172.21.15.78:0/3214631070 >> 172.21.15.78:0/3214631070 conn(0x55c58a8fc300 msgr2=0x55c58aca4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 -- 172.21.15.78:0/3214631070 shutdown_connections 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.021+0000 7f11ba229740 1 -- 172.21.15.78:0/3214631070 wait complete. 2024-04-25T14:04:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 -- start start 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 Processor -- start 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a8e74d0 0x55c58acb2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a9081b0 0x55c58acb9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c58acb0840 con 0x55c58a8df2f0 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c58ab95b60 con 0x55c58a9081b0 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c58acb09c0 con 0x55c58a8e74d0 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11ba229740 10 client.0 _ll_register_callbacks cb 0x55c58abba2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53430/0 (socket says 172.21.15.78:53430) 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.022+0000 7f11b1032640 1 -- 172.21.15.78:0/2306087917 learned_addr learned my addr 172.21.15.78:0/2306087917 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b0831640 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a8e74d0 0x55c58acb2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b1833640 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a9081b0 0x55c58acb9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b1032640 1 -- 172.21.15.78:0/2306087917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a8e74d0 msgr2=0x55c58acb2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b1032640 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a8e74d0 0x55c58acb2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b1032640 1 -- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a9081b0 msgr2=0x55c58acb9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b1032640 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a9081b0 0x55c58acb9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b1032640 1 -- 172.21.15.78:0/2306087917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c58a920210 con 0x55c58a8df2f0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11b1032640 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb25f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f11a800b000 tx=0x7f11a8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982a544ea5633377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f1194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f1194ff9640 1 -- 172.21.15.78:0/2306087917 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8027020 con 0x55c58a8df2f0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a80043c0 con 0x55c58a8df2f0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f1194ff9640 1 -- 172.21.15.78:0/2306087917 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f11a8003980 con 0x55c58a8df2f0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f1194ff9640 1 -- 172.21.15.78:0/2306087917 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f11a8032050 con 0x55c58a8df2f0 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11ba229740 10 client.16866 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.023+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c58a9ec620 con 0x55c58a8df2f0 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f1194ff9640 1 -- 172.21.15.78:0/2306087917 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a80043c0 con 0x55c58a8df2f0 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f1194ff9640 1 -- 172.21.15.78:0/2306087917 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f11a8004b80 con 0x55c58a8df2f0 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11ba229740 10 client.16866 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11ba229740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11ba229740 20 client.16866 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11ba229740 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a8c6d80 0x55c58acc0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11ba229740 4 client.16866 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c58ac1c070 con 0x55c58a8c6d80 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11b0831640 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a8c6d80 0x55c58acc0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f1194ff9640 10 client.16866 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.024+0000 7f11b0831640 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a8c6d80 0x55c58acc0050 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd411e624799429 server_cookie=8381435da8d8a3f6 in_seq=0 out_seq=0 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f1194ff9640 1 -- 172.21.15.78:0/2306087917 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c58ac1c070 con 0x55c58a8c6d80 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f1194ff9640 10 client.16866 handle_command_reply: tid=0 2024-04-25T14:04:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 client.16866 shutdown 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 msgr2=0x55c58acb25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb25f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f11a800b000 tx=0x7f11a8003d80 comp rx=0 tx=0).stop 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 shutdown_connections 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a9081b0 0x55c58acb9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a8c6d80 0x55c58acc0050 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a8e74d0 0x55c58acb2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 --2- 172.21.15.78:0/2306087917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb25f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f1194ff9640 0 client.16866 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.025+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 >> 172.21.15.78:0/2306087917 conn(0x55c58a8fc300 msgr2=0x55c58aca7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.026+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 shutdown_connections 2024-04-25T14:04:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.026+0000 7f11ba229740 1 -- 172.21.15.78:0/2306087917 wait complete. 2024-04-25T14:04:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.026+0000 7f11ba229740 20 client.16866 trim_cache size 0 max 16384 2024-04-25T14:04:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.040+0000 7f11ba229740 1 Processor -- start 2024-04-25T14:04:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.040+0000 7f11ba229740 1 -- start start 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11ba229740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11ba229740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11ba229740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11ba229740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1afd00 con 0x55c58a8df2f0 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11ba229740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1ad5d0 con 0x55c58a8e74d0 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11ba229740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1ab420 con 0x55c58a9081b0 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53438/0 (socket says 172.21.15.78:53438) 2024-04-25T14:04:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 -- 172.21.15.78:0/50782403 learned_addr learned my addr 172.21.15.78:0/50782403 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1833640 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b0831640 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 -- 172.21.15.78:0/50782403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 msgr2=0x55c58aca7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 -- 172.21.15.78:0/50782403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 msgr2=0x55c58acb99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 -- 172.21.15.78:0/50782403 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c58acb01b0 con 0x55c58a8df2f0 2024-04-25T14:04:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1833640 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b0831640 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f11b1032640 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca4320 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f11a8018640 tx=0x7f11a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2089f1bc01d83b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f1196ffd640 1 -- 172.21.15.78:0/50782403 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a803b070 con 0x55c58a8df2f0 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f1196ffd640 1 -- 172.21.15.78:0/50782403 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a8003980 con 0x55c58a8df2f0 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.041+0000 7f1196ffd640 1 -- 172.21.15.78:0/50782403 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8027070 con 0x55c58a8df2f0 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 -- 172.21.15.78:0/50782403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 msgr2=0x55c58aca4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca4320 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f11a8018640 tx=0x7f11a800f790 comp rx=0 tx=0).stop 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 -- 172.21.15.78:0/50782403 shutdown_connections 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58aca7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 --2- 172.21.15.78:0/50782403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58aca4320 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 -- 172.21.15.78:0/50782403 >> 172.21.15.78:0/50782403 conn(0x55c58a8fc300 msgr2=0x55c58aca3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 -- 172.21.15.78:0/50782403 shutdown_connections 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.042+0000 7f11ba229740 1 -- 172.21.15.78:0/50782403 wait complete. 2024-04-25T14:04:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 -- start start 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 Processor -- start 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58acb81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1aba90 con 0x55c58a8df2f0 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1acf00 con 0x55c58a8e74d0 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11ac1ac160 con 0x55c58a9081b0 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11ba229740 10 client.0 _ll_register_callbacks cb 0x7f11a4009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53452/0 (socket says 172.21.15.78:53452) 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 -- 172.21.15.78:0/1008119839 learned_addr learned my addr 172.21.15.78:0/1008119839 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b0831640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1833640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58acb81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 -- 172.21.15.78:0/1008119839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 msgr2=0x55c58acb81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58acb81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 -- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 msgr2=0x55c58acb7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b1032640 1 -- 172.21.15.78:0/1008119839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c58acb9ef0 con 0x55c58a8df2f0 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.043+0000 7f11b0831640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11b1032640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb6dd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000c00 tx=0x7f11a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9a6ca74a3e8481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f1194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f1194ff9640 1 -- 172.21.15.78:0/1008119839 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a8027070 con 0x55c58a8df2f0 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f1194ff9640 1 -- 172.21.15.78:0/1008119839 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f11a8018b10 con 0x55c58a8df2f0 2024-04-25T14:04:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a8003980 con 0x55c58a8df2f0 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f1194ff9640 1 -- 172.21.15.78:0/1008119839 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f11a803b040 con 0x55c58a8df2f0 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 10 client.16872 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c58acad4a0 con 0x55c58a8df2f0 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f1194ff9640 1 -- 172.21.15.78:0/1008119839 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a8003980 con 0x55c58a8df2f0 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f1194ff9640 1 -- 172.21.15.78:0/1008119839 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f11a8004fa0 con 0x55c58a8df2f0 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 10 client.16872 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 20 client.16872 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a89c1c0 0x55c58ad2a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 4 client.16872 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.044+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c58ac15ee0 con 0x55c58a89c1c0 2024-04-25T14:04:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.045+0000 7f11b0831640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a89c1c0 0x55c58ad2a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.045+0000 7f11b0831640 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a89c1c0 0x55c58ad2a390 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc348c954b5883f server_cookie=3f8bf023fa569ae3 in_seq=0 out_seq=0 2024-04-25T14:04:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.045+0000 7f1194ff9640 10 client.16872 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f1194ff9640 1 -- 172.21.15.78:0/1008119839 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c58ac15ee0 con 0x55c58a89c1c0 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f1194ff9640 10 client.16872 handle_command_reply: tid=0 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 client.16872 shutdown 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 msgr2=0x55c58acb6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb6dd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000c00 tx=0x7f11a800a620 comp rx=0 tx=0).stop 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 shutdown_connections 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c58a89c1c0 0x55c58ad2a390 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c58a9081b0 0x55c58acb81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f1194ff9640 0 client.16872 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c58a8e74d0 0x55c58acb7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 --2- 172.21.15.78:0/1008119839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c58a8df2f0 0x55c58acb6dd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 >> 172.21.15.78:0/1008119839 conn(0x55c58a8fc300 msgr2=0x55c58acb2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 shutdown_connections 2024-04-25T14:04:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 1 -- 172.21.15.78:0/1008119839 wait complete. 2024-04-25T14:04:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.059+0000 7f11ba229740 20 client.16872 trim_cache size 0 max 16384 2024-04-25T14:04:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:04:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:04:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:04:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:04:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11980afae0 msgr2=0x7f11980b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11980afae0 0x7f11980b1ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55c58aca9b40 tx=0x7f11a40097e0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 msgr2=0x7f11ac1a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac1a13e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f119c006010 tx=0x7f119c004340 comp rx=0 tx=0).stop 2024-04-25T14:04:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 shutdown_connections 2024-04-25T14:04:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11ac108940 0x7f11ac1a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11980afae0 0x7f11980b1ff0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11ac103c40 0x7f11ac1a13e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 --2- 172.21.15.78:0/2022039479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11ac102a00 0x7f11ac075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 >> 172.21.15.78:0/2022039479 conn(0x7f11ac0fdbd0 msgr2=0x7f11ac104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 shutdown_connections 2024-04-25T14:04:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:20.060+0000 7f11b3296640 1 -- 172.21.15.78:0/2022039479 wait complete. 2024-04-25T14:04:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:20 smithi129 ceph-mon[32569]: pgmap v493: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:20.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:20 smithi078 ceph-mon[29163]: pgmap v493: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:20.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:20 smithi112 ceph-mon[32617]: pgmap v493: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:22.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:22 smithi112 ceph-mon[32617]: pgmap v494: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:04:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:22 smithi129 ceph-mon[32569]: pgmap v494: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:04:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:22 smithi078 ceph-mon[29163]: pgmap v494: 97 pgs: 97 active+clean; 3.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:04:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:24 smithi129 ceph-mon[32569]: pgmap v495: 97 pgs: 97 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:24 smithi078 ceph-mon[29163]: pgmap v495: 97 pgs: 97 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:24.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:24 smithi112 ceph-mon[32617]: pgmap v495: 97 pgs: 97 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:26.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:26 smithi129 ceph-mon[32569]: pgmap v496: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:04:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:26 smithi078 ceph-mon[29163]: pgmap v496: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:04:26.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:26 smithi112 ceph-mon[32617]: pgmap v496: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:04:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:28 smithi129 ceph-mon[32569]: pgmap v497: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:28.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:28 smithi078 ceph-mon[29163]: pgmap v497: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:28.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:28 smithi112 ceph-mon[32617]: pgmap v497: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:30 smithi129 ceph-mon[32569]: pgmap v498: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:30 smithi078 ceph-mon[29163]: pgmap v498: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:30.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:30 smithi112 ceph-mon[32617]: pgmap v498: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:04:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:32 smithi129 ceph-mon[32569]: pgmap v499: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:32 smithi078 ceph-mon[29163]: pgmap v499: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:32.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:32 smithi112 ceph-mon[32617]: pgmap v499: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 -- 172.21.15.78:0/3375139448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 msgr2=0x7f616810bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 --2- 172.21.15.78:0/3375139448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f616810bcf0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6160009ca0 tx=0x7f616002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 -- 172.21.15.78:0/3375139448 shutdown_connections 2024-04-25T14:04:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 --2- 172.21.15.78:0/3375139448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f616810bcf0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 --2- 172.21.15.78:0/3375139448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6168106f70 0x7f6168109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 --2- 172.21.15.78:0/3375139448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6168104610 0x7f6168106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 -- 172.21.15.78:0/3375139448 >> 172.21.15.78:0/3375139448 conn(0x7f61680fdc10 msgr2=0x7f61681000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 -- 172.21.15.78:0/3375139448 shutdown_connections 2024-04-25T14:04:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.165+0000 7f616e71f640 1 -- 172.21.15.78:0/3375139448 wait complete. 2024-04-25T14:04:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 Processor -- start 2024-04-25T14:04:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 -- start start 2024-04-25T14:04:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6168104610 0x7f616819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6168106f70 0x7f616819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f61681a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f616810f650 con 0x7f61681098d0 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f616810f4d0 con 0x7f6168106f70 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616e71f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f616810f7d0 con 0x7f6168104610 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616ccbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f61681a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616ccbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f61681a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47590/0 (socket says 172.21.15.78:47590) 2024-04-25T14:04:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f616ccbc640 1 -- 172.21.15.78:0/3849129578 learned_addr learned my addr 172.21.15.78:0/3849129578 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f61677fe640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6168106f70 0x7f616819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.166+0000 7f6167fff640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6168104610 0x7f616819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616ccbc640 1 -- 172.21.15.78:0/3849129578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6168104610 msgr2=0x7f616819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616ccbc640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6168104610 0x7f616819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616ccbc640 1 -- 172.21.15.78:0/3849129578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6168106f70 msgr2=0x7f616819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616ccbc640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6168106f70 0x7f616819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616ccbc640 1 -- 172.21.15.78:0/3849129578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681a1d60 con 0x7f61681098d0 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f61677fe640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6168106f70 0x7f616819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616ccbc640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f61681a16f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f61600317c0 tx=0x7f61600317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f61657fa640 1 -- 172.21.15.78:0/3849129578 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160003b70 con 0x7f61681098d0 2024-04-25T14:04:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f61657fa640 1 -- 172.21.15.78:0/3849129578 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6160003d10 con 0x7f61681098d0 2024-04-25T14:04:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f61657fa640 1 -- 172.21.15.78:0/3849129578 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616002fbd0 con 0x7f61681098d0 2024-04-25T14:04:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681a1ff0 con 0x7f61681098d0 2024-04-25T14:04:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.167+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681a2450 con 0x7f61681098d0 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 Processor -- start 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61657fa640 1 -- 172.21.15.78:0/3849129578 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f616002fd70 con 0x7f61681098d0 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 -- start start 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d47792970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d47792ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d47799df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d476f8460 con 0x562d473d14d0 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d476b2590 con 0x562d473c92f0 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61755d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562d476f4420 con 0x562d473f21b0 2024-04-25T14:04:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d47799df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d47799df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47606/0 (socket says 172.21.15.78:47606) 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61677fe640 1 -- 172.21.15.78:0/3331926569 learned_addr learned my addr 172.21.15.78:0/3331926569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61677fe640 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d47792970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f6167fff640 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d47792ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.170+0000 7f61657fa640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f61500afae0 0x7f61500b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f61677fe640 1 -- 172.21.15.78:0/3331926569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 msgr2=0x562d47792970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f61677fe640 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d47792970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f61677fe640 1 -- 172.21.15.78:0/3331926569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 msgr2=0x562d47792ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f61677fe640 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d47792ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f61677fe640 1 -- 172.21.15.78:0/3331926569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d476bf0d0 con 0x562d473d14d0 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f616ccbc640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f61500afae0 0x7f61500b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f61677fe640 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d47799df0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f615c0149e0 tx=0x7f615c00b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6949d39db6a27845 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f61657fa640 1 -- 172.21.15.78:0/3849129578 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6160100340 con 0x7f61681098d0 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f614e7fc640 1 -- 172.21.15.78:0/3331926569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c0142d0 con 0x562d473d14d0 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f614e7fc640 1 -- 172.21.15.78:0/3331926569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f615c005020 con 0x562d473d14d0 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.171+0000 7f614e7fc640 1 -- 172.21.15.78:0/3331926569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c0044e0 con 0x562d473d14d0 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 -- 172.21.15.78:0/3331926569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 msgr2=0x562d47799df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f616ccbc640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f61500afae0 0x7f61500b1ff0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61600fe1a0 tx=0x7f61600f6f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d47799df0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f615c0149e0 tx=0x7f615c00b520 comp rx=0 tx=0).stop 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 -- 172.21.15.78:0/3331926569 shutdown_connections 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d47799df0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d47792ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 --2- 172.21.15.78:0/3331926569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d47792970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 -- 172.21.15.78:0/3331926569 >> 172.21.15.78:0/3331926569 conn(0x562d473e6300 msgr2=0x562d4778e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 -- 172.21.15.78:0/3331926569 shutdown_connections 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 -- 172.21.15.78:0/3331926569 wait complete. 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.172+0000 7f61755d4740 1 -- start start 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 1 Processor -- start 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d477a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562d4779ebf0 con 0x562d473d14d0 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562d4767fb60 con 0x562d473c92f0 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562d4779ed70 con 0x562d473f21b0 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61755d4740 10 client.0 _ll_register_callbacks cb 0x562d476a42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47622/0 (socket says 172.21.15.78:47622) 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 -- 172.21.15.78:0/3081271538 learned_addr learned my addr 172.21.15.78:0/3081271538 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61677fe640 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61677fe640 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d477a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 -- 172.21.15.78:0/3081271538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 msgr2=0x562d477a7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d477a7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 -- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 msgr2=0x562d477a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 -- 172.21.15.78:0/3081271538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4740a210 con 0x562d473d14d0 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f6167fff640 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a0f80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6158013710 tx=0x7f6158013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85ff1f3648e5533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.173+0000 7f61477fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61477fe640 1 -- 172.21.15.78:0/3081271538 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158004460 con 0x562d473d14d0 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61477fe640 1 -- 172.21.15.78:0/3081271538 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f61580119c0 con 0x562d473d14d0 2024-04-25T14:04:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f615c005020 con 0x562d473d14d0 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61477fe640 1 -- 172.21.15.78:0/3081271538 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6158011bc0 con 0x562d473d14d0 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 10 client.16884 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562d474d6620 con 0x562d473d14d0 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61477fe640 1 -- 172.21.15.78:0/3081271538 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f615c005020 con 0x562d473d14d0 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61477fe640 1 -- 172.21.15.78:0/3081271538 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6158025050 con 0x562d473d14d0 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 10 client.16884 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 20 client.16884 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473b0d80 0x562d477ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 4 client.16884 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.174+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d47706070 con 0x562d473b0d80 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.175+0000 7f61677fe640 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473b0d80 0x562d477ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.175+0000 7f61477fe640 10 client.16884 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.175+0000 7f61677fe640 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473b0d80 0x562d477ae830 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54a993573fa674c server_cookie=9d6e8f5515509c3f in_seq=0 out_seq=0 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61477fe640 1 -- 172.21.15.78:0/3081271538 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d47706070 con 0x562d473b0d80 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61477fe640 10 client.16884 handle_command_reply: tid=0 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 client.16884 shutdown 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 msgr2=0x562d477a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a0f80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6158013710 tx=0x7f6158013bd0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 shutdown_connections 2024-04-25T14:04:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473b0d80 0x562d477ae830 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473f21b0 0x562d477a7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a0f80 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 --2- 172.21.15.78:0/3081271538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61477fe640 0 client.16884 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 >> 172.21.15.78:0/3081271538 conn(0x562d473e6300 msgr2=0x562d4778d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 shutdown_connections 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 1 -- 172.21.15.78:0/3081271538 wait complete. 2024-04-25T14:04:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.176+0000 7f61755d4740 20 client.16884 trim_cache size 0 max 16384 2024-04-25T14:04:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 Processor -- start 2024-04-25T14:04:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 -- start start 2024-04-25T14:04:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473c92f0 0x562d4778dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473f21b0 0x562d4778e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61681ab980 con 0x562d473d14d0 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61681a9250 con 0x562d473f21b0 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61755d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61681a70a0 con 0x562d473c92f0 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f6167fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f6167fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47630/0 (socket says 172.21.15.78:47630) 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f6167fff640 1 -- 172.21.15.78:0/3666277789 learned_addr learned my addr 172.21.15.78:0/3666277789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.191+0000 7f61677fe640 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473c92f0 0x562d4778dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f6167fff640 1 -- 172.21.15.78:0/3666277789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473c92f0 msgr2=0x562d4778dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f6167fff640 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473c92f0 0x562d4778dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f61677fe640 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473f21b0 0x562d4778e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f6167fff640 1 -- 172.21.15.78:0/3666277789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473f21b0 msgr2=0x562d4778e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f6167fff640 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473f21b0 0x562d4778e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f6167fff640 1 -- 172.21.15.78:0/3666277789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d47797530 con 0x562d473d14d0 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f61677fe640 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473f21b0 0x562d4778e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f6167fff640 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a7e00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6158001eb0 tx=0x7f6158011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52da8e61ca441b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f614dffb640 1 -- 172.21.15.78:0/3666277789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158000e60 con 0x562d473d14d0 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f614dffb640 1 -- 172.21.15.78:0/3666277789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f615802b070 con 0x562d473d14d0 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.192+0000 7f614dffb640 1 -- 172.21.15.78:0/3666277789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158003c00 con 0x562d473d14d0 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 -- 172.21.15.78:0/3666277789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 msgr2=0x562d477a7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a7e00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6158001eb0 tx=0x7f6158011b10 comp rx=0 tx=0).stop 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 -- 172.21.15.78:0/3666277789 shutdown_connections 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473f21b0 0x562d4778e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473d14d0 0x562d477a7e00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 --2- 172.21.15.78:0/3666277789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473c92f0 0x562d4778dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 -- 172.21.15.78:0/3666277789 >> 172.21.15.78:0/3666277789 conn(0x562d473e6300 msgr2=0x562d477928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 -- 172.21.15.78:0/3666277789 shutdown_connections 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 -- 172.21.15.78:0/3666277789 wait complete. 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 -- start start 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.193+0000 7f61755d4740 1 Processor -- start 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473d14d0 0x562d477a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473f21b0 0x562d477a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61681a7710 con 0x562d473f21b0 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61681a8b80 con 0x562d473c92f0 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61681a7de0 con 0x562d473d14d0 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 10 client.0 _ll_register_callbacks cb 0x7f6158009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473f21b0 0x562d477a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473f21b0 0x562d477a7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47640/0 (socket says 172.21.15.78:47640) 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 -- 172.21.15.78:0/1079386288 learned_addr learned my addr 172.21.15.78:0/1079386288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f6167fff640 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473d14d0 0x562d477a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 -- 172.21.15.78:0/1079386288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473d14d0 msgr2=0x562d477a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473d14d0 0x562d477a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 -- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 msgr2=0x562d477a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 -- 172.21.15.78:0/1079386288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d477a8340 con 0x562d473f21b0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61677fe640 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473f21b0 0x562d477a7100 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f615c00bef0 tx=0x7f6158009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed683da2f7b4df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61477fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61477fe640 1 -- 172.21.15.78:0/1079386288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c02a040 con 0x562d473f21b0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61477fe640 1 -- 172.21.15.78:0/1079386288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f615c00f730 con 0x562d473f21b0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f615802b070 con 0x562d473f21b0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.194+0000 7f61477fe640 1 -- 172.21.15.78:0/1079386288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f615c00f930 con 0x562d473f21b0 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 10 client.16890 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562d47816f90 con 0x562d473f21b0 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61477fe640 1 -- 172.21.15.78:0/1079386288 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f615802b070 con 0x562d473f21b0 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61477fe640 1 -- 172.21.15.78:0/1079386288 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f615c02d920 con 0x562d473f21b0 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 10 client.16890 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 20 client.16890 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473861c0 0x562d47819730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 4 client.16890 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d476ffee0 con 0x562d473861c0 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.195+0000 7f6167fff640 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473861c0 0x562d47819730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.196+0000 7f61477fe640 10 client.16890 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.196+0000 7f6167fff640 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473861c0 0x562d47819730 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9737c5c6c056672e server_cookie=62864c5db3e798d5 in_seq=0 out_seq=0 2024-04-25T14:04:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61477fe640 1 -- 172.21.15.78:0/1079386288 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d476ffee0 con 0x562d473861c0 2024-04-25T14:04:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61477fe640 10 client.16890 handle_command_reply: tid=0 2024-04-25T14:04:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 client.16890 shutdown 2024-04-25T14:04:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473f21b0 msgr2=0x562d477a7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473f21b0 0x562d477a7100 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f615c00bef0 tx=0x7f6158009cb0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 shutdown_connections 2024-04-25T14:04:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562d473f21b0 0x562d477a7100 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562d473861c0 0x562d47819730 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d473d14d0 0x562d477a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 --2- 172.21.15.78:0/1079386288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562d473c92f0 0x562d477a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61477fe640 0 client.16890 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 >> 172.21.15.78:0/1079386288 conn(0x562d473e6300 msgr2=0x562d477928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 shutdown_connections 2024-04-25T14:04:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 1 -- 172.21.15.78:0/1079386288 wait complete. 2024-04-25T14:04:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.212+0000 7f61755d4740 20 client.16890 trim_cache size 0 max 16384 2024-04-25T14:04:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:04:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:04:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:04:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:04:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f61500afae0 msgr2=0x7f61500b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f61500afae0 0x7f61500b1ff0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61600fe1a0 tx=0x7f61600f6f60 comp rx=0 tx=0).stop 2024-04-25T14:04:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 msgr2=0x7f61681a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f61681a16f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f61600317c0 tx=0x7f61600317f0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 shutdown_connections 2024-04-25T14:04:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f61500afae0 0x7f61500b1ff0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61681098d0 0x7f61681a16f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6168106f70 0x7f616819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 --2- 172.21.15.78:0/3849129578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6168104610 0x7f616819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 >> 172.21.15.78:0/3849129578 conn(0x7f61680fdc10 msgr2=0x7f6168105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 shutdown_connections 2024-04-25T14:04:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:33.213+0000 7f616e71f640 1 -- 172.21.15.78:0/3849129578 wait complete. 2024-04-25T14:04:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:34 smithi129 ceph-mon[32569]: pgmap v500: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:34 smithi078 ceph-mon[29163]: pgmap v500: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:34 smithi112 ceph-mon[32617]: pgmap v500: 97 pgs: 97 active+clean; 3.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:04:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:36 smithi129 ceph-mon[32569]: pgmap v501: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:36.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:36 smithi078 ceph-mon[29163]: pgmap v501: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:36.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:36 smithi112 ceph-mon[32617]: pgmap v501: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:04:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:38 smithi129 ceph-mon[32569]: pgmap v502: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:38 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:38 smithi078 ceph-mon[29163]: pgmap v502: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:38 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:38 smithi112 ceph-mon[32617]: pgmap v502: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:38 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:04:39.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:39.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:39.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.182+0000 7fdc55714640 1 -- 172.21.15.78:0/520182887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 msgr2=0x7fdc50104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.182+0000 7fdc55714640 1 --2- 172.21.15.78:0/520182887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc50104030 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009900 tx=0x7fdc3802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.182+0000 7fdc55714640 1 -- 172.21.15.78:0/520182887 shutdown_connections 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.182+0000 7fdc55714640 1 --2- 172.21.15.78:0/520182887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc50108920 0x7fdc5010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.182+0000 7fdc55714640 1 --2- 172.21.15.78:0/520182887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc50104030 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.182+0000 7fdc55714640 1 --2- 172.21.15.78:0/520182887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc501029e0 0x7fdc501036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.182+0000 7fdc55714640 1 -- 172.21.15.78:0/520182887 >> 172.21.15.78:0/520182887 conn(0x7fdc500fdbd0 msgr2=0x7fdc50100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 -- 172.21.15.78:0/520182887 shutdown_connections 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 -- 172.21.15.78:0/520182887 wait complete. 2024-04-25T14:04:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 Processor -- start 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 -- start start 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc501029e0 0x7fdc5010d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc5010b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108920 0x7fdc5010be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5010f610 con 0x7fdc50103c20 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5010f490 con 0x7fdc50108920 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc55714640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5010f790 con 0x7fdc501029e0 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc5010b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc5010b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47642/0 (socket says 172.21.15.78:47642) 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.183+0000 7fdc47fff640 1 -- 172.21.15.78:0/705242706 learned_addr learned my addr 172.21.15.78:0/705242706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc4effd640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc501029e0 0x7fdc5010d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108920 0x7fdc5010be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc47fff640 1 -- 172.21.15.78:0/705242706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc501029e0 msgr2=0x7fdc5010d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc47fff640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc501029e0 0x7fdc5010d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc47fff640 1 -- 172.21.15.78:0/705242706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108920 msgr2=0x7fdc5010be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc47fff640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108920 0x7fdc5010be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc47fff640 1 -- 172.21.15.78:0/705242706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5010c530 con 0x7fdc50103c20 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc47fff640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc5010b950 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc380043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc4cff9640 1 -- 172.21.15.78:0/705242706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3803f030 con 0x7fdc50103c20 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc4cff9640 1 -- 172.21.15.78:0/705242706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc38003cb0 con 0x7fdc50103c20 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc501a45a0 con 0x7fdc50103c20 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc4cff9640 1 -- 172.21.15.78:0/705242706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38035070 con 0x7fdc50103c20 2024-04-25T14:04:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.184+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc501a4950 con 0x7fdc50103c20 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 Processor -- start 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 -- start start 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc4cff9640 1 -- 172.21.15.78:0/705242706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdc38042030 con 0x7fdc50103c20 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9551d80 con 0x55b0a922a6b0 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b0a950b8b0 con 0x55b0a92224d0 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b0a954dd40 con 0x55b0a924b390 2024-04-25T14:04:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95edc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47646/0 (socket says 172.21.15.78:47646) 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.187+0000 7fdc47fff640 1 -- 172.21.15.78:0/3361523908 learned_addr learned my addr 172.21.15.78:0/3361523908 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc4effd640 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc47fff640 1 -- 172.21.15.78:0/3361523908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 msgr2=0x55b0a95eca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc47fff640 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95eca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc47fff640 1 -- 172.21.15.78:0/3361523908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 msgr2=0x55b0a95ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc47fff640 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc47fff640 1 -- 172.21.15.78:0/3361523908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a94dca40 con 0x55b0a922a6b0 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc4effd640 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc47fff640 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95edc70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdc381074e0 tx=0x7fdc381075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f3606998b345eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc4cff9640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc3c0afaf0 0x7fdc3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc457fa640 1 -- 172.21.15.78:0/3361523908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc380c6fe0 con 0x55b0a922a6b0 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc3c0afaf0 0x7fdc3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc457fa640 1 -- 172.21.15.78:0/3361523908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc380c3380 con 0x55b0a922a6b0 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc457fa640 1 -- 172.21.15.78:0/3361523908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc380c71c0 con 0x55b0a922a6b0 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.188+0000 7fdc4cff9640 1 -- 172.21.15.78:0/705242706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdc380ff2f0 con 0x7fdc50103c20 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3361523908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 msgr2=0x55b0a95edc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc3c0afaf0 0x7fdc3c0b2000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55b0a95edab0 tx=0x7fdc48007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95edc70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdc381074e0 tx=0x7fdc381075d0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3361523908 shutdown_connections 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95edc70 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3361523908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3361523908 >> 172.21.15.78:0/3361523908 conn(0x55b0a923f4e0 msgr2=0x55b0a95e8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3361523908 shutdown_connections 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.189+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3361523908 wait complete. 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 -- start start 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 Processor -- start 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95fda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b0a95ee540 con 0x55b0a922a6b0 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b0a94d8d10 con 0x55b0a92224d0 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b0a95ee6c0 con 0x55b0a924b390 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc5c5c6740 10 client.0 _ll_register_callbacks cb 0x55b0a9505210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47652/0 (socket says 172.21.15.78:47652) 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 -- 172.21.15.78:0/3399776231 learned_addr learned my addr 172.21.15.78:0/3399776231 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc4effd640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 -- 172.21.15.78:0/3399776231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 msgr2=0x55b0a95fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 -- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 msgr2=0x55b0a95f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 -- 172.21.15.78:0/3399776231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a932d7d0 con 0x55b0a922a6b0 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc4effd640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc47fff640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95f6ae0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdc380c7570 tx=0x7fdc380bb550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca46a0f21cc9bbed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.190+0000 7fdc367fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc367fc640 1 -- 172.21.15.78:0/3399776231 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38049020 con 0x55b0a922a6b0 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc367fc640 1 -- 172.21.15.78:0/3399776231 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdc380924e0 con 0x55b0a922a6b0 2024-04-25T14:04:39.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc380c3380 con 0x55b0a922a6b0 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc367fc640 1 -- 172.21.15.78:0/3399776231 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdc380c73b0 con 0x55b0a922a6b0 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 10 client.16902 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b0a92633f0 con 0x55b0a922a6b0 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc367fc640 1 -- 172.21.15.78:0/3399776231 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc380c3380 con 0x55b0a922a6b0 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc367fc640 1 -- 172.21.15.78:0/3399776231 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdc3803bce0 con 0x55b0a922a6b0 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 10 client.16902 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 20 client.16902 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a9209f60 0x55b0a9603ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 4 client.16902 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0a955f990 con 0x55b0a9209f60 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.191+0000 7fdc4effd640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a9209f60 0x55b0a9603ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.192+0000 7fdc367fc640 10 client.16902 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:39.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.192+0000 7fdc4effd640 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a9209f60 0x55b0a9603ce0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffadb46dcbb89a3 server_cookie=623957b23c751c73 in_seq=0 out_seq=0 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.192+0000 7fdc367fc640 1 -- 172.21.15.78:0/3399776231 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b0a955f990 con 0x55b0a9209f60 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.192+0000 7fdc367fc640 10 client.16902 handle_command_reply: tid=0 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 client.16902 shutdown 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 msgr2=0x55b0a95f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95f6ae0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdc380c7570 tx=0x7fdc380bb550 comp rx=0 tx=0).stop 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 shutdown_connections 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a9209f60 0x55b0a9603ce0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a924b390 0x55b0a95fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95f6ae0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc367fc640 0 client.16902 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/3399776231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a92224d0 0x55b0a95f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 >> 172.21.15.78:0/3399776231 conn(0x55b0a923f4e0 msgr2=0x55b0a95e73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 shutdown_connections 2024-04-25T14:04:39.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/3399776231 wait complete. 2024-04-25T14:04:39.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.193+0000 7fdc5c5c6740 20 client.16902 trim_cache size 0 max 16384 2024-04-25T14:04:39.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.210+0000 7fdc5c5c6740 1 Processor -- start 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc5c5c6740 1 -- start start 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a92224d0 0x55b0a95f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a95ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501afc90 con 0x55b0a922a6b0 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ad560 con 0x55b0a924b390 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ab3b0 con 0x55b0a92224d0 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47660/0 (socket says 172.21.15.78:47660) 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 -- 172.21.15.78:0/2419212190 learned_addr learned my addr 172.21.15.78:0/2419212190 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc4effd640 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a92224d0 0x55b0a95f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a95ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 -- 172.21.15.78:0/2419212190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a92224d0 msgr2=0x55b0a95f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a92224d0 0x55b0a95f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 -- 172.21.15.78:0/2419212190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 msgr2=0x55b0a95ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a95ebcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.211+0000 7fdc47fff640 1 -- 172.21.15.78:0/2419212190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a92230f0 con 0x55b0a922a6b0 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.212+0000 7fdc47fff640 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95eb780 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdc380c4ae0 tx=0x7fdc380d00b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22545f39191e57fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.212+0000 7fdc44ff9640 1 -- 172.21.15.78:0/2419212190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38049070 con 0x55b0a922a6b0 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.212+0000 7fdc44ff9640 1 -- 172.21.15.78:0/2419212190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc38102ba0 con 0x55b0a922a6b0 2024-04-25T14:04:39.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.212+0000 7fdc44ff9640 1 -- 172.21.15.78:0/2419212190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc380c7640 con 0x55b0a922a6b0 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/2419212190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 msgr2=0x55b0a95eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95eb780 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdc380c4ae0 tx=0x7fdc380d00b0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/2419212190 shutdown_connections 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a95ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a922a6b0 0x55b0a95eb780 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/2419212190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a92224d0 0x55b0a95f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/2419212190 >> 172.21.15.78:0/2419212190 conn(0x55b0a923f4e0 msgr2=0x55b0a95eb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/2419212190 shutdown_connections 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/2419212190 wait complete. 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- start start 2024-04-25T14:04:39.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 Processor -- start 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a92224d0 0x55b0a95fb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a922a6b0 0x55b0a95fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a966a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501aba20 con 0x55b0a92224d0 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ace90 con 0x55b0a924b390 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501ac0f0 con 0x55b0a922a6b0 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc5c5c6740 10 client.0 _ll_register_callbacks cb 0x7fdc38107360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc4effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a92224d0 0x55b0a95fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc4effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a92224d0 0x55b0a95fb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47664/0 (socket says 172.21.15.78:47664) 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.213+0000 7fdc4effd640 1 -- 172.21.15.78:0/1513175897 learned_addr learned my addr 172.21.15.78:0/1513175897 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc47fff640 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a922a6b0 0x55b0a95fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc4f7fe640 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a966a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc4effd640 1 -- 172.21.15.78:0/1513175897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a922a6b0 msgr2=0x55b0a95fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc4effd640 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a922a6b0 0x55b0a95fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc4effd640 1 -- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 msgr2=0x55b0a966a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc4effd640 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a966a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc4effd640 1 -- 172.21.15.78:0/1513175897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a95124b0 con 0x55b0a92224d0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc4effd640 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a92224d0 0x55b0a95fb140 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000aaf0 tx=0x7fdc4001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e984721046aaea4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc367fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc367fc640 1 -- 172.21.15.78:0/1513175897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000fdc0 con 0x55b0a92224d0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc367fc640 1 -- 172.21.15.78:0/1513175897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fdc40025470 con 0x55b0a92224d0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.214+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc38102ba0 con 0x55b0a92224d0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc367fc640 1 -- 172.21.15.78:0/1513175897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdc40025670 con 0x55b0a92224d0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 10 client.16908 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b0a966b640 con 0x55b0a92224d0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc367fc640 1 -- 172.21.15.78:0/1513175897 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc38102ba0 con 0x55b0a92224d0 2024-04-25T14:04:39.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc367fc640 1 -- 172.21.15.78:0/1513175897 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdc40030990 con 0x55b0a92224d0 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 10 client.16908 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 20 client.16908 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a91df3a0 0x55b0a966e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 4 client.16908 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0a9559800 con 0x55b0a91df3a0 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.215+0000 7fdc47fff640 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a91df3a0 0x55b0a966e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc367fc640 10 client.16908 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc47fff640 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a91df3a0 0x55b0a966e060 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e63a89e4855842 server_cookie=b5c81261ed403dfb in_seq=0 out_seq=0 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc367fc640 1 -- 172.21.15.78:0/1513175897 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0a9559800 con 0x55b0a91df3a0 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc367fc640 10 client.16908 handle_command_reply: tid=0 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 client.16908 shutdown 2024-04-25T14:04:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a92224d0 msgr2=0x55b0a95fb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a92224d0 0x55b0a95fb140 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000aaf0 tx=0x7fdc4001cab0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 shutdown_connections 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0a924b390 0x55b0a966a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b0a91df3a0 0x55b0a966e060 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b0a922a6b0 0x55b0a95fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 --2- 172.21.15.78:0/1513175897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0a92224d0 0x55b0a95fb140 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc367fc640 0 client.16908 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 >> 172.21.15.78:0/1513175897 conn(0x55b0a923f4e0 msgr2=0x55b0a95ebaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 shutdown_connections 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 1 -- 172.21.15.78:0/1513175897 wait complete. 2024-04-25T14:04:39.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.216+0000 7fdc5c5c6740 20 client.16908 trim_cache size 0 max 16384 2024-04-25T14:04:39.223 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:04:39.223 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:04:39.223 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:04:39.223 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc3c0afaf0 msgr2=0x7fdc3c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc3c0afaf0 0x7fdc3c0b2000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55b0a95edab0 tx=0x7fdc48007520 comp rx=0 tx=0).stop 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 msgr2=0x7fdc5010b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc5010b950 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc380043b0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 shutdown_connections 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc50108920 0x7fdc5010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fdc3c0afaf0 0x7fdc3c0b2000 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc50103c20 0x7fdc5010b950 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 --2- 172.21.15.78:0/705242706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc501029e0 0x7fdc5010d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 >> 172.21.15.78:0/705242706 conn(0x7fdc500fdbd0 msgr2=0x7fdc50104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 shutdown_connections 2024-04-25T14:04:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.217+0000 7fdc55714640 1 -- 172.21.15.78:0/705242706 wait complete. 2024-04-25T14:04:39.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:04:39.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.336+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2250592997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c103b00 msgr2=0x7f8a3c104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.336+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2250592997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c103b00 0x7f8a3c104800 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009900 tx=0x7f8a3802a410 comp rx=0 tx=0).stop 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2250592997 shutdown_connections 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2250592997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a3c109a40 0x7f8a3c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2250592997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c104d40 0x7f8a3c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2250592997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c103b00 0x7f8a3c104800 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2250592997 >> 172.21.15.78:0/2250592997 conn(0x7f8a3c0fed10 msgr2=0x7f8a3c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2250592997 shutdown_connections 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2250592997 wait complete. 2024-04-25T14:04:39.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 Processor -- start 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 -- start start 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.337+0000 7f8a43b1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c103b00 0x7f8a3c077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a43b1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c104d40 0x7f8a3c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a43b1f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a3c109a40 0x7f8a3c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a43b1f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c10f5e0 con 0x7f8a3c104d40 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a43b1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c10f460 con 0x7f8a3c103b00 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a43b1f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c10f760 con 0x7f8a3c109a40 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c104d40 0x7f8a3c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c104d40 0x7f8a3c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47680/0 (socket says 172.21.15.78:47680) 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 -- 172.21.15.78:0/2923271752 learned_addr learned my addr 172.21.15.78:0/2923271752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a420bc640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a3c109a40 0x7f8a3c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a418bb640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c103b00 0x7f8a3c077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 -- 172.21.15.78:0/2923271752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a3c109a40 msgr2=0x7f8a3c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a3c109a40 0x7f8a3c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 -- 172.21.15.78:0/2923271752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c103b00 msgr2=0x7f8a3c077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c103b00 0x7f8a3c077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 -- 172.21.15.78:0/2923271752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c076c70 con 0x7f8a3c104d40 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a410ba640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c104d40 0x7f8a3c076000 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00d950 tx=0x7f8a2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a32ffd640 1 -- 172.21.15.78:0/2923271752 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c017030 con 0x7f8a3c104d40 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a32ffd640 1 -- 172.21.15.78:0/2923271752 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c004020 con 0x7f8a3c104d40 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1a4300 con 0x7f8a3c104d40 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a32ffd640 1 -- 172.21.15.78:0/2923271752 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0047a0 con 0x7f8a3c104d40 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.338+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a47c0 con 0x7f8a3c104d40 2024-04-25T14:04:39.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.339+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c108850 con 0x7f8a3c104d40 2024-04-25T14:04:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.342+0000 7f8a32ffd640 1 -- 172.21.15.78:0/2923271752 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8a2c009a30 con 0x7f8a3c104d40 2024-04-25T14:04:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.342+0000 7f8a32ffd640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8a280afaf0 0x7f8a280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.342+0000 7f8a418bb640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8a280afaf0 0x7f8a280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.342+0000 7f8a32ffd640 1 -- 172.21.15.78:0/2923271752 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8a2c0ded80 con 0x7f8a3c104d40 2024-04-25T14:04:39.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.343+0000 7f8a418bb640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8a280afaf0 0x7f8a280b2000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009230 tx=0x7f8a38031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.346+0000 7f8a32ffd640 1 -- 172.21.15.78:0/2923271752 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a2c0a7950 con 0x7f8a3c104d40 2024-04-25T14:04:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:04:39.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.560+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a3c02db50 con 0x7f8a3c104d40 2024-04-25T14:04:39.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.561+0000 7f8a32ffd640 1 -- 172.21.15.78:0/2923271752 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f8a2c0a7250 con 0x7f8a3c104d40 2024-04-25T14:04:39.561 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:04:39.561 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:04:39.561 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:04:39.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8a280afaf0 msgr2=0x7f8a280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8a280afaf0 0x7f8a280b2000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009230 tx=0x7f8a38031400 comp rx=0 tx=0).stop 2024-04-25T14:04:39.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c104d40 msgr2=0x7f8a3c076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c104d40 0x7f8a3c076000 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00d950 tx=0x7f8a2c00de10 comp rx=0 tx=0).stop 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 shutdown_connections 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a3c109a40 0x7f8a3c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8a280afaf0 0x7f8a280b2000 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a3c104d40 0x7f8a3c076000 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 --2- 172.21.15.78:0/2923271752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a3c103b00 0x7f8a3c077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 >> 172.21.15.78:0/2923271752 conn(0x7f8a3c0fed10 msgr2=0x7f8a3c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 shutdown_connections 2024-04-25T14:04:39.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.563+0000 7f8a43b1f640 1 -- 172.21.15.78:0/2923271752 wait complete. 2024-04-25T14:04:39.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:04:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:39.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:04:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 -- 172.21.15.78:0/1539210850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 msgr2=0x7f56b41047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1539210850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b41047f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009900 tx=0x7f56a802a410 comp rx=0 tx=0).stop 2024-04-25T14:04:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 -- 172.21.15.78:0/1539210850 shutdown_connections 2024-04-25T14:04:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1539210850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b4109a30 0x7f56b410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1539210850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b4104d30 0x7f56b4105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1539210850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b41047f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 -- 172.21.15.78:0/1539210850 >> 172.21.15.78:0/1539210850 conn(0x7f56b40fece0 msgr2=0x7f56b4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 -- 172.21.15.78:0/1539210850 shutdown_connections 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.684+0000 7f56bbf40640 1 -- 172.21.15.78:0/1539210850 wait complete. 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 Processor -- start 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 -- start start 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b4104d30 0x7f56b419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b4109a30 0x7f56b41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56b410f5f0 con 0x7f56b4103af0 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56b410f470 con 0x7f56b4109a30 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56bbf40640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56b410f770 con 0x7f56b4104d30 2024-04-25T14:04:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47686/0 (socket says 172.21.15.78:47686) 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 -- 172.21.15.78:0/1565795430 learned_addr learned my addr 172.21.15.78:0/1565795430 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b94db640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b4104d30 0x7f56b419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b4109a30 0x7f56b41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 -- 172.21.15.78:0/1565795430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b4104d30 msgr2=0x7f56b419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b4104d30 0x7f56b419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 -- 172.21.15.78:0/1565795430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b4109a30 msgr2=0x7f56b41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b4109a30 0x7f56b41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b9cdc640 1 -- 172.21.15.78:0/1565795430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b41a1da0 con 0x7f56b4103af0 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56b94db640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b4104d30 0x7f56b419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.685+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b4109a30 0x7f56b41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.686+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b419a2b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006010 tx=0x7f56a8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.686+0000 7f56a6ffd640 1 -- 172.21.15.78:0/1565795430 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a803f030 con 0x7f56b4103af0 2024-04-25T14:04:39.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.686+0000 7f56a6ffd640 1 -- 172.21.15.78:0/1565795430 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a8003aa0 con 0x7f56b4103af0 2024-04-25T14:04:39.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.686+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41a2030 con 0x7f56b4103af0 2024-04-25T14:04:39.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.686+0000 7f56a6ffd640 1 -- 172.21.15.78:0/1565795430 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8035070 con 0x7f56b4103af0 2024-04-25T14:04:39.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.686+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41a2520 con 0x7f56b4103af0 2024-04-25T14:04:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.688+0000 7f56a6ffd640 1 -- 172.21.15.78:0/1565795430 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f56a8042030 con 0x7f56b4103af0 2024-04-25T14:04:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.688+0000 7f56c2df1740 1 Processor -- start 2024-04-25T14:04:39.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56c2df1740 1 -- start start 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 0x55ce9591a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce9591ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce9591ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56c2df1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce9587ed80 con 0x55ce9554f4d0 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56c2df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce95838a20 con 0x55ce955576b0 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56c2df1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ce9587ad40 con 0x55ce95578390 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce9591ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce9591ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47700/0 (socket says 172.21.15.78:47700) 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2016127548 learned_addr learned my addr 172.21.15.78:0/2016127548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56a6ffd640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f56a00afaf0 0x7f56a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b94db640 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce9591ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 0x55ce9591a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f56a00afaf0 0x7f56a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56a6ffd640 1 -- 172.21.15.78:0/1565795430 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f56a80ff270 con 0x7f56b4103af0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2016127548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 msgr2=0x55ce9591a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 0x55ce9591a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2016127548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 msgr2=0x55ce9591ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce9591ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2016127548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce95843c30 con 0x55ce9554f4d0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.689+0000 7f56b94db640 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce9591ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.690+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce9591ad90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f56a80e89f0 tx=0x7f56a8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6975e32bd23f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.690+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f56a00afaf0 0x7f56a00b2000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009270 tx=0x7f56ac005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.690+0000 7f569ffff640 1 -- 172.21.15.78:0/2016127548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a80c7250 con 0x55ce9554f4d0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.690+0000 7f569ffff640 1 -- 172.21.15.78:0/2016127548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a80c7e40 con 0x55ce9554f4d0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.690+0000 7f569ffff640 1 -- 172.21.15.78:0/2016127548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8003c50 con 0x55ce9554f4d0 2024-04-25T14:04:39.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 -- 172.21.15.78:0/2016127548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 msgr2=0x55ce9591ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce9591ad90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f56a80e89f0 tx=0x7f56a8003f90 comp rx=0 tx=0).stop 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 -- 172.21.15.78:0/2016127548 shutdown_connections 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce9591ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce9591ad90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2016127548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 0x55ce9591a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 -- 172.21.15.78:0/2016127548 >> 172.21.15.78:0/2016127548 conn(0x55ce9556c4e0 msgr2=0x55ce95916020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 -- 172.21.15.78:0/2016127548 shutdown_connections 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 -- 172.21.15.78:0/2016127548 wait complete. 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 -- start start 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.691+0000 7f56c2df1740 1 Processor -- start 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce95924210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce95924750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 0x55ce9592b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce9591c2f0 con 0x55ce9554f4d0 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ce95805dc0 con 0x55ce955576b0 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ce9591c470 con 0x55ce95578390 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 10 client.0 _ll_register_callbacks cb 0x55ce95821600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce95924210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce95924210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47716/0 (socket says 172.21.15.78:47716) 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2059845669 learned_addr learned my addr 172.21.15.78:0/2059845669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b94db640 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce95924750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2059845669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 msgr2=0x55ce9592b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 0x55ce9592b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2059845669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 msgr2=0x55ce95924750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce95924750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2059845669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce955903f0 con 0x55ce9554f4d0 2024-04-25T14:04:39.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b94db640 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce95924750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce95924210 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f56a80c49b0 tx=0x7f56a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914a0b61e33cd0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f569d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f569d7fa640 1 -- 172.21.15.78:0/2059845669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8049070 con 0x55ce9554f4d0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f569d7fa640 1 -- 172.21.15.78:0/2059845669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f56a80b0070 con 0x55ce9554f4d0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56a80c7e40 con 0x55ce9554f4d0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f569d7fa640 1 -- 172.21.15.78:0/2059845669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f56a80c7250 con 0x55ce9554f4d0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.692+0000 7f56c2df1740 10 client.16926 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ce9565a7d0 con 0x55ce9554f4d0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f569d7fa640 1 -- 172.21.15.78:0/2059845669 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a80c7e40 con 0x55ce9554f4d0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f569d7fa640 1 -- 172.21.15.78:0/2059845669 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f56a80aebe0 con 0x55ce9554f4d0 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56c2df1740 10 client.16926 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56c2df1740 10 client.16926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56c2df1740 20 client.16926 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:39.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95536f60 0x55ce95931cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56c2df1740 4 client.16926 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce9588c990 con 0x55ce95536f60 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.693+0000 7f56b94db640 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95536f60 0x55ce95931cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f569d7fa640 10 client.16926 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56b94db640 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95536f60 0x55ce95931cf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e96902c634dd9fc server_cookie=d20c66aa902254bb in_seq=0 out_seq=0 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f569d7fa640 1 -- 172.21.15.78:0/2059845669 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce9588c990 con 0x55ce95536f60 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f569d7fa640 10 client.16926 handle_command_reply: tid=0 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 client.16926 shutdown 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 msgr2=0x55ce95924210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce95924210 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f56a80c49b0 tx=0x7f56a8009260 comp rx=0 tx=0).stop 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 shutdown_connections 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95536f60 0x55ce95931cf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce95578390 0x55ce9592b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce95924750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f569d7fa640 0 client.16926 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2059845669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9554f4d0 0x55ce95924210 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.694+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 >> 172.21.15.78:0/2059845669 conn(0x55ce9556c4e0 msgr2=0x55ce95918f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.695+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 shutdown_connections 2024-04-25T14:04:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.695+0000 7f56c2df1740 1 -- 172.21.15.78:0/2059845669 wait complete. 2024-04-25T14:04:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.695+0000 7f56c2df1740 20 client.16926 trim_cache size 0 max 16384 2024-04-25T14:04:39.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 Processor -- start 2024-04-25T14:04:39.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 -- start start 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce95536f60 0x55ce959244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce959364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce959369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41adbb0 con 0x55ce95536f60 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41ab480 con 0x55ce955576b0 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56c2df1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41a92d0 con 0x55ce9554f4d0 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce95536f60 0x55ce959244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56b9cdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce95536f60 0x55ce959244f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47718/0 (socket says 172.21.15.78:47718) 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2083595643 learned_addr learned my addr 172.21.15.78:0/2083595643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.708+0000 7f56b94db640 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce959364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce959369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2083595643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 msgr2=0x55ce959364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce959364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2083595643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 msgr2=0x55ce959369e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce959369e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56b9cdc640 1 -- 172.21.15.78:0/2083595643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce9591fb20 con 0x55ce95536f60 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce959369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce95536f60 0x55ce959244f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f56a80c7200 tx=0x7f56a803b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4b19ac99d9652c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f569f7fe640 1 -- 172.21.15.78:0/2083595643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8049070 con 0x55ce95536f60 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f569f7fe640 1 -- 172.21.15.78:0/2083595643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a8087040 con 0x55ce95536f60 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.709+0000 7f569f7fe640 1 -- 172.21.15.78:0/2083595643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a80cbaf0 con 0x55ce95536f60 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 -- 172.21.15.78:0/2083595643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce95536f60 msgr2=0x55ce959244f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce95536f60 0x55ce959244f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f56a80c7200 tx=0x7f56a803b720 comp rx=0 tx=0).stop 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 -- 172.21.15.78:0/2083595643 shutdown_connections 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce955576b0 0x55ce959369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce959364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 --2- 172.21.15.78:0/2083595643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce95536f60 0x55ce959244f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 -- 172.21.15.78:0/2083595643 >> 172.21.15.78:0/2083595643 conn(0x55ce9556c4e0 msgr2=0x55ce95923de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 -- 172.21.15.78:0/2083595643 shutdown_connections 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 -- 172.21.15.78:0/2083595643 wait complete. 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 -- start start 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 Processor -- start 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.710+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce95536f60 0x55ce9592a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce9592a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce955576b0 0x55ce95998400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41a9940 con 0x55ce955576b0 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41aadb0 con 0x55ce95536f60 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56b41aa010 con 0x55ce9554f4d0 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 10 client.0 _ll_register_callbacks cb 0x7f56b000c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce955576b0 0x55ce95998400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce955576b0 0x55ce95998400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47734/0 (socket says 172.21.15.78:47734) 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 -- 172.21.15.78:0/1807158791 learned_addr learned my addr 172.21.15.78:0/1807158791 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce95536f60 0x55ce9592a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56b94db640 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce9592a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 -- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 msgr2=0x55ce9592a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce9592a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 -- 172.21.15.78:0/1807158791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce95536f60 msgr2=0x55ce9592a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce95536f60 0x55ce9592a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 -- 172.21.15.78:0/1807158791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce959989f0 con 0x55ce955576b0 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56ba4dd640 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce955576b0 0x55ce95998400 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005fd0 tx=0x7f56ac009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcc4f6a09b48fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f569d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f569d7fa640 1 -- 172.21.15.78:0/1807158791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac01fdd0 con 0x55ce955576b0 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f569d7fa640 1 -- 172.21.15.78:0/1807158791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f56ac004b20 con 0x55ce955576b0 2024-04-25T14:04:39.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56a8087040 con 0x55ce955576b0 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f569d7fa640 1 -- 172.21.15.78:0/1807158791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f56ac004d20 con 0x55ce955576b0 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 10 client.16932 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.711+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ce95999d40 con 0x55ce955576b0 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f569d7fa640 1 -- 172.21.15.78:0/1807158791 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a8087040 con 0x55ce955576b0 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f569d7fa640 1 -- 172.21.15.78:0/1807158791 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f56ac032050 con 0x55ce955576b0 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f56c2df1740 10 client.16932 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f56c2df1740 10 client.16932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f56c2df1740 20 client.16932 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f56c2df1740 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95578390 0x55ce9599c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f56c2df1740 4 client.16932 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce95886800 con 0x55ce95578390 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95578390 0x55ce9599c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.712+0000 7f569d7fa640 10 client.16932 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56b9cdc640 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95578390 0x55ce9599c790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44f5c482b29ae28e server_cookie=3d33460dedac2c1f in_seq=0 out_seq=0 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f569d7fa640 1 -- 172.21.15.78:0/1807158791 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce95886800 con 0x55ce95578390 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f569d7fa640 10 client.16932 handle_command_reply: tid=0 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 client.16932 shutdown 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce955576b0 msgr2=0x55ce95998400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce955576b0 0x55ce95998400 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005fd0 tx=0x7f56ac009970 comp rx=0 tx=0).stop 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 shutdown_connections 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ce95578390 0x55ce9599c790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce955576b0 0x55ce95998400 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce9554f4d0 0x55ce9592a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 --2- 172.21.15.78:0/1807158791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ce95536f60 0x55ce9592a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f569d7fa640 0 client.16932 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:04:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 >> 172.21.15.78:0/1807158791 conn(0x55ce9556c4e0 msgr2=0x55ce95918e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 shutdown_connections 2024-04-25T14:04:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 1 -- 172.21.15.78:0/1807158791 wait complete. 2024-04-25T14:04:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.713+0000 7f56c2df1740 20 client.16932 trim_cache size 0 max 16384 2024-04-25T14:04:39.720 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:04:39.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f56a00afaf0 msgr2=0x7f56a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f56a00afaf0 0x7f56a00b2000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009270 tx=0x7f56ac005bd0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 msgr2=0x7f56b419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b419a2b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006010 tx=0x7f56a8031a50 comp rx=0 tx=0).stop 2024-04-25T14:04:39.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 shutdown_connections 2024-04-25T14:04:39.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56b4109a30 0x7f56b41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56b4104d30 0x7f56b419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f56a00afaf0 0x7f56a00b2000 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 --2- 172.21.15.78:0/1565795430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56b4103af0 0x7f56b419a2b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 >> 172.21.15.78:0/1565795430 conn(0x7f56b40fece0 msgr2=0x7f56b4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 shutdown_connections 2024-04-25T14:04:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.714+0000 7f56bbf40640 1 -- 172.21.15.78:0/1565795430 wait complete. 2024-04-25T14:04:39.724 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:04:39.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:04:39.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:04:39.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:04:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 -- 172.21.15.78:0/907990348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758104480 msgr2=0x7f075810abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 --2- 172.21.15.78:0/907990348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758104480 0x7f075810abd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f074802a410 comp rx=0 tx=0).stop 2024-04-25T14:04:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 -- 172.21.15.78:0/907990348 shutdown_connections 2024-04-25T14:04:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 --2- 172.21.15.78:0/907990348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758104480 0x7f075810abd0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 --2- 172.21.15.78:0/907990348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758103960 0x7f0758103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 --2- 172.21.15.78:0/907990348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0758102f80 0x7f0758103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 -- 172.21.15.78:0/907990348 >> 172.21.15.78:0/907990348 conn(0x7f07580fd380 msgr2=0x7f07580ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 -- 172.21.15.78:0/907990348 shutdown_connections 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.834+0000 7f075f75e640 1 -- 172.21.15.78:0/907990348 wait complete. 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 Processor -- start 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 -- start start 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0758102f80 0x7f0758102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758103960 0x7f075819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758104480 0x7f07581a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f075810e550 con 0x7f0758103960 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f075810e3d0 con 0x7f0758102f80 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075f75e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f075810e6d0 con 0x7f0758104480 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758103960 0x7f075819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758103960 0x7f075819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47752/0 (socket says 172.21.15.78:47752) 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 -- 172.21.15.78:0/4020187828 learned_addr learned my addr 172.21.15.78:0/4020187828 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075dcfb640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758104480 0x7f07581a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075d4fa640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0758102f80 0x7f0758102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 -- 172.21.15.78:0/4020187828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758104480 msgr2=0x7f07581a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758104480 0x7f07581a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 -- 172.21.15.78:0/4020187828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0758102f80 msgr2=0x7f0758102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0758102f80 0x7f0758102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075ccf9640 1 -- 172.21.15.78:0/4020187828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07581a1d40 con 0x7f0758103960 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.835+0000 7f075d4fa640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0758102f80 0x7f0758102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.836+0000 7f075ccf9640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758103960 0x7f075819efd0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d770 tx=0x7f074c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.836+0000 7f07567fc640 1 -- 172.21.15.78:0/4020187828 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c017070 con 0x7f0758103960 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.836+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07581a1fd0 con 0x7f0758103960 2024-04-25T14:04:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.836+0000 7f07567fc640 1 -- 172.21.15.78:0/4020187828 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c0054d0 con 0x7f0758103960 2024-04-25T14:04:39.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.836+0000 7f07567fc640 1 -- 172.21.15.78:0/4020187828 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c004c30 con 0x7f0758103960 2024-04-25T14:04:39.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.836+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581a24c0 con 0x7f0758103960 2024-04-25T14:04:39.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.838+0000 7f076660f740 1 Processor -- start 2024-04-25T14:04:39.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.838+0000 7f076660f740 1 -- start start 2024-04-25T14:04:39.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.838+0000 7f07567fc640 1 -- 172.21.15.78:0/4020187828 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f074c010460 con 0x7f0758103960 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.838+0000 7f076660f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca80c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.838+0000 7f076660f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4414d0 0x5616ca80cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.838+0000 7f076660f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca80da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f076660f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5616ca770d80 con 0x5616ca46a390 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f076660f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616ca72aa20 con 0x5616ca4496b0 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f076660f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616ca76cd40 con 0x5616ca4414d0 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca80c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca80c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47760/0 (socket says 172.21.15.78:47760) 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 -- 172.21.15.78:0/3764784503 learned_addr learned my addr 172.21.15.78:0/3764784503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075d4fa640 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4414d0 0x5616ca80cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075ccf9640 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca80da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 -- 172.21.15.78:0/3764784503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4414d0 msgr2=0x5616ca80cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4414d0 0x5616ca80cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 -- 172.21.15.78:0/3764784503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 msgr2=0x5616ca80da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca80da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 -- 172.21.15.78:0/3764784503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ca735c30 con 0x5616ca46a390 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075dcfb640 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca80c810 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f0748031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5378589ab7cc7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f07567fc640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f07400afaf0 0x7f07400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f073f7fe640 1 -- 172.21.15.78:0/3764784503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748035030 con 0x5616ca46a390 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f075d4fa640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f07400afaf0 0x7f07400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f07567fc640 1 -- 172.21.15.78:0/4020187828 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f074c0de6b0 con 0x7f0758103960 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f073f7fe640 1 -- 172.21.15.78:0/3764784503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0748040070 con 0x5616ca46a390 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.839+0000 7f073f7fe640 1 -- 172.21.15.78:0/3764784503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748003af0 con 0x5616ca46a390 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 -- 172.21.15.78:0/3764784503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 msgr2=0x5616ca80c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f075d4fa640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f07400afaf0 0x7f07400b2000 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5616ca80c530 tx=0x7f07500099c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca80c810 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f0748031c40 comp rx=0 tx=0).stop 2024-04-25T14:04:39.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 -- 172.21.15.78:0/3764784503 shutdown_connections 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca80da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4414d0 0x5616ca80cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 --2- 172.21.15.78:0/3764784503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca80c810 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 -- 172.21.15.78:0/3764784503 >> 172.21.15.78:0/3764784503 conn(0x5616ca45e4e0 msgr2=0x5616ca808020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 -- 172.21.15.78:0/3764784503 shutdown_connections 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.840+0000 7f076660f740 1 -- 172.21.15.78:0/3764784503 wait complete. 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 -- start start 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 Processor -- start 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca816b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca46a390 0x5616ca81da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5616ca80e4d0 con 0x5616ca4414d0 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616ca6f7dc0 con 0x5616ca46a390 2024-04-25T14:04:39.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616ca80e650 con 0x5616ca4496b0 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f076660f740 10 client.0 _ll_register_callbacks cb 0x5616ca713600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f075d4fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f075d4fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47776/0 (socket says 172.21.15.78:47776) 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f075d4fa640 1 -- 172.21.15.78:0/1497919156 learned_addr learned my addr 172.21.15.78:0/1497919156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f075dcfb640 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca46a390 0x5616ca81da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.841+0000 7f075ccf9640 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca816b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f075d4fa640 1 -- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 msgr2=0x5616ca816b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f075d4fa640 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca816b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f075d4fa640 1 -- 172.21.15.78:0/1497919156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca46a390 msgr2=0x5616ca81da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f075d4fa640 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca46a390 0x5616ca81da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f075d4fa640 1 -- 172.21.15.78:0/1497919156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ca4823f0 con 0x5616ca4414d0 2024-04-25T14:04:39.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f075d4fa640 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8165e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f07500126d0 tx=0x7f0750004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb76002d923664c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f073cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f073cff9640 1 -- 172.21.15.78:0/1497919156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750023070 con 0x5616ca4414d0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f073cff9640 1 -- 172.21.15.78:0/1497919156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0750011ae0 con 0x5616ca4414d0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0748040070 con 0x5616ca4414d0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f073cff9640 1 -- 172.21.15.78:0/1497919156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f075002f050 con 0x5616ca4414d0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 10 client.16944 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5616ca54c7d0 con 0x5616ca4414d0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f073cff9640 1 -- 172.21.15.78:0/1497919156 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0748040070 con 0x5616ca4414d0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f073cff9640 1 -- 172.21.15.78:0/1497919156 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f075000f040 con 0x5616ca4414d0 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 10 client.16944 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 10 client.16944 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 20 client.16944 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:39.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca428f60 0x5616ca824320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 4 client.16944 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.842+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616ca77e990 con 0x5616ca428f60 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.843+0000 7f075ccf9640 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca428f60 0x5616ca824320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.843+0000 7f073cff9640 10 client.16944 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.843+0000 7f075ccf9640 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca428f60 0x5616ca824320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=247deb4efbd01aa server_cookie=f7c9a3de4f280a1f in_seq=0 out_seq=0 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f073cff9640 1 -- 172.21.15.78:0/1497919156 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5616ca77e990 con 0x5616ca428f60 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f073cff9640 10 client.16944 handle_command_reply: tid=0 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 client.16944 shutdown 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 msgr2=0x5616ca8165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8165e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f07500126d0 tx=0x7f0750004ab0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 shutdown_connections 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca46a390 0x5616ca81da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca428f60 0x5616ca824320 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca816b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 --2- 172.21.15.78:0/1497919156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8165e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f073cff9640 0 client.16944 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:39.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 >> 172.21.15.78:0/1497919156 conn(0x5616ca45e4e0 msgr2=0x5616ca80af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 shutdown_connections 2024-04-25T14:04:39.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 1 -- 172.21.15.78:0/1497919156 wait complete. 2024-04-25T14:04:39.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.844+0000 7f076660f740 20 client.16944 trim_cache size 0 max 16384 2024-04-25T14:04:39.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 Processor -- start 2024-04-25T14:04:39.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 -- start start 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca828ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca46a390 0x5616ca829010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07581adbb0 con 0x5616ca4414d0 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07581ab480 con 0x5616ca4496b0 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f076660f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07581a92d0 con 0x5616ca46a390 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8168c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47784/0 (socket says 172.21.15.78:47784) 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 -- 172.21.15.78:0/2629972278 learned_addr learned my addr 172.21.15.78:0/2629972278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075dcfb640 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca46a390 0x5616ca829010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075ccf9640 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca828ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 -- 172.21.15.78:0/2629972278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca46a390 msgr2=0x5616ca829010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca46a390 0x5616ca829010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 -- 172.21.15.78:0/2629972278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 msgr2=0x5616ca828ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca828ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 -- 172.21.15.78:0/2629972278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ca80ce70 con 0x5616ca4414d0 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.858+0000 7f075d4fa640 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8168c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f075001ff40 tx=0x7f0750011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1292950c60a4cfd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f073effd640 1 -- 172.21.15.78:0/2629972278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750037070 con 0x5616ca4414d0 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f073effd640 1 -- 172.21.15.78:0/2629972278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0750011230 con 0x5616ca4414d0 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f073effd640 1 -- 172.21.15.78:0/2629972278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750023040 con 0x5616ca4414d0 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 -- 172.21.15.78:0/2629972278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 msgr2=0x5616ca8168c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8168c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f075001ff40 tx=0x7f0750011040 comp rx=0 tx=0).stop 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 -- 172.21.15.78:0/2629972278 shutdown_connections 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca46a390 0x5616ca829010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4496b0 0x5616ca828ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 --2- 172.21.15.78:0/2629972278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca4414d0 0x5616ca8168c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 -- 172.21.15.78:0/2629972278 >> 172.21.15.78:0/2629972278 conn(0x5616ca45e4e0 msgr2=0x5616ca8161b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 -- 172.21.15.78:0/2629972278 shutdown_connections 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.859+0000 7f076660f740 1 -- 172.21.15.78:0/2629972278 wait complete. 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 -- start start 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 Processor -- start 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4414d0 0x5616ca8876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca887c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca88eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07581a9940 con 0x5616ca46a390 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07581aadb0 con 0x5616ca4414d0 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07581aa010 con 0x5616ca4496b0 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f076660f740 10 client.0 _ll_register_callbacks cb 0x7f0750016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f075dcfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca88eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f075dcfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca88eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47788/0 (socket says 172.21.15.78:47788) 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f075dcfb640 1 -- 172.21.15.78:0/1988875401 learned_addr learned my addr 172.21.15.78:0/1988875401 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f075d4fa640 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4414d0 0x5616ca8876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.860+0000 7f075ccf9640 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca887c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f075dcfb640 1 -- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 msgr2=0x5616ca887c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f075dcfb640 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca887c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f075dcfb640 1 -- 172.21.15.78:0/1988875401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4414d0 msgr2=0x5616ca8876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f075dcfb640 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4414d0 0x5616ca8876f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f075dcfb640 1 -- 172.21.15.78:0/1988875401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ca88f150 con 0x5616ca46a390 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f075dcfb640 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca88eb90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f0748005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3451808f545950f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f073cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f073cff9640 1 -- 172.21.15.78:0/1988875401 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748054060 con 0x5616ca46a390 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f073cff9640 1 -- 172.21.15.78:0/1988875401 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0748045670 con 0x5616ca46a390 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0750011230 con 0x5616ca46a390 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f073cff9640 1 -- 172.21.15.78:0/1988875401 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0748035040 con 0x5616ca46a390 2024-04-25T14:04:39.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f076660f740 10 client.16950 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5616ca890440 con 0x5616ca46a390 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f073cff9640 1 -- 172.21.15.78:0/1988875401 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0750011230 con 0x5616ca46a390 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f073cff9640 1 -- 172.21.15.78:0/1988875401 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f074803c8e0 con 0x5616ca46a390 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f076660f740 10 client.16950 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f076660f740 10 client.16950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.861+0000 7f076660f740 20 client.16950 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.862+0000 7f076660f740 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca3fe3a0 0x5616ca892a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.862+0000 7f076660f740 4 client.16950 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.862+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616ca778800 con 0x5616ca3fe3a0 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.862+0000 7f075d4fa640 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca3fe3a0 0x5616ca892a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.862+0000 7f073cff9640 10 client.16950 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.862+0000 7f075d4fa640 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca3fe3a0 0x5616ca892a80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3ea0fb607b61678 server_cookie=673fe8108c3b386c in_seq=0 out_seq=0 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f073cff9640 1 -- 172.21.15.78:0/1988875401 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616ca778800 con 0x5616ca3fe3a0 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f073cff9640 10 client.16950 handle_command_reply: tid=0 2024-04-25T14:04:39.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 client.16950 shutdown 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 msgr2=0x5616ca88eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca88eb90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f0748005fd0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 shutdown_connections 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616ca46a390 0x5616ca88eb90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x5616ca3fe3a0 0x5616ca892a80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616ca4496b0 0x5616ca887c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 --2- 172.21.15.78:0/1988875401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5616ca4414d0 0x5616ca8876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 >> 172.21.15.78:0/1988875401 conn(0x5616ca45e4e0 msgr2=0x5616ca80ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f073cff9640 0 client.16950 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 shutdown_connections 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 1 -- 172.21.15.78:0/1988875401 wait complete. 2024-04-25T14:04:39.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f076660f740 20 client.16950 trim_cache size 0 max 16384 2024-04-25T14:04:39.870 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:04:39.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f07400afaf0 msgr2=0x7f07400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f07400afaf0 0x7f07400b2000 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5616ca80c530 tx=0x7f07500099c0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758103960 msgr2=0x7f075819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758103960 0x7f075819efd0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d770 tx=0x7f074c00dc30 comp rx=0 tx=0).stop 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 shutdown_connections 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758104480 0x7f07581a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f07400afaf0 0x7f07400b2000 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0758103960 0x7f075819efd0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 --2- 172.21.15.78:0/4020187828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0758102f80 0x7f0758102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.863+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 >> 172.21.15.78:0/4020187828 conn(0x7f07580fd380 msgr2=0x7f0758106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.864+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 shutdown_connections 2024-04-25T14:04:39.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.864+0000 7f075f75e640 1 -- 172.21.15.78:0/4020187828 wait complete. 2024-04-25T14:04:39.874 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:04:39.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 -- 172.21.15.78:0/3230037320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4103ef0 msgr2=0x7f75d41027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 --2- 172.21.15.78:0/3230037320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4103ef0 0x7f75d41027d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009900 tx=0x7f75c402a410 comp rx=0 tx=0).stop 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 -- 172.21.15.78:0/3230037320 shutdown_connections 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 --2- 172.21.15.78:0/3230037320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d4108a20 0x7f75d410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 --2- 172.21.15.78:0/3230037320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d41043d0 0x7f75d4102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 --2- 172.21.15.78:0/3230037320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4103ef0 0x7f75d41027d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 -- 172.21.15.78:0/3230037320 >> 172.21.15.78:0/3230037320 conn(0x7f75d40fd350 msgr2=0x7f75d40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 -- 172.21.15.78:0/3230037320 shutdown_connections 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.994+0000 7f75da051640 1 -- 172.21.15.78:0/3230037320 wait complete. 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 Processor -- start 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 -- start start 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d4103ef0 0x7f75d4102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4108a20 0x7f75d419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d419f020 0x7f75d41a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:39.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75d410e4e0 con 0x7f75d4108a20 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75d410e360 con 0x7f75d4103ef0 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75da051640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75d410e660 con 0x7f75d419f020 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4108a20 0x7f75d419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4108a20 0x7f75d419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47796/0 (socket says 172.21.15.78:47796) 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3417386052 learned_addr learned my addr 172.21.15.78:0/3417386052 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75d37fe640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d4103ef0 0x7f75d4102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.995+0000 7f75d3fff640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d419f020 0x7f75d41a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3417386052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d419f020 msgr2=0x7f75d41a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d419f020 0x7f75d41a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3417386052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d4103ef0 msgr2=0x7f75d4102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d4103ef0 0x7f75d4102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3417386052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d41a1cc0 con 0x7f75d4108a20 2024-04-25T14:04:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d37fe640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d4103ef0 0x7f75d4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4108a20 0x7f75d419eae0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f75c800d950 tx=0x7f75c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d0ff9640 1 -- 172.21.15.78:0/3417386052 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8017070 con 0x7f75d4108a20 2024-04-25T14:04:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d0ff9640 1 -- 172.21.15.78:0/3417386052 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c8009c30 con 0x7f75d4108a20 2024-04-25T14:04:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75d0ff9640 1 -- 172.21.15.78:0/3417386052 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c800f620 con 0x7f75d4108a20 2024-04-25T14:04:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d41a1fb0 con 0x7f75d4108a20 2024-04-25T14:04:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.996+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41a23c0 con 0x7f75d4108a20 2024-04-25T14:04:40.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.998+0000 7f75e0fe8740 1 Processor -- start 2024-04-25T14:04:40.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.998+0000 7f75d0ff9640 1 -- 172.21.15.78:0/3417386052 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f75c8010460 con 0x7f75d4108a20 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.998+0000 7f75e0fe8740 1 -- start start 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfb5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbea4d0 0x55d45dfb5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbf26b0 0x55d45dfb6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d45df19d80 con 0x55d45dc13390 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d45ded3a20 con 0x55d45dbf26b0 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d45df15d40 con 0x55d45dbea4d0 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfb5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfb5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47812/0 (socket says 172.21.15.78:47812) 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 -- 172.21.15.78:0/1838711597 learned_addr learned my addr 172.21.15.78:0/1838711597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbf26b0 0x55d45dfb6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d37fe640 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbea4d0 0x55d45dfb5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 -- 172.21.15.78:0/1838711597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbea4d0 msgr2=0x55d45dfb5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbea4d0 0x55d45dfb5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 -- 172.21.15.78:0/1838711597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbf26b0 msgr2=0x55d45dfb6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbf26b0 0x55d45dfb6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 -- 172.21.15.78:0/1838711597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45dedec30 con 0x55d45dc13390 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbf26b0 0x55d45dfb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d0ff9640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75bc0afaf0 0x7f75bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d3fff640 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfb5810 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009fc0 tx=0x7f75cc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34394bfcea732700 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d37fe640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75bc0afaf0 0x7f75bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75d0ff9640 1 -- 172.21.15.78:0/3417386052 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f75c80de930 con 0x7f75d4108a20 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75b9ffb640 1 -- 172.21.15.78:0/1838711597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003b70 con 0x55d45dc13390 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:39.999+0000 7f75b9ffb640 1 -- 172.21.15.78:0/1838711597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75cc003d10 con 0x55d45dc13390 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75b9ffb640 1 -- 172.21.15.78:0/1838711597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc011970 con 0x55d45dc13390 2024-04-25T14:04:40.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 -- 172.21.15.78:0/1838711597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 msgr2=0x55d45dfb5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfb5810 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009fc0 tx=0x7f75cc010860 comp rx=0 tx=0).stop 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75d37fe640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75bc0afaf0 0x7f75bc0b2000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f75c4000c00 tx=0x7f75c40314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 -- 172.21.15.78:0/1838711597 shutdown_connections 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbf26b0 0x55d45dfb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbea4d0 0x55d45dfb5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/1838711597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfb5810 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 -- 172.21.15.78:0/1838711597 >> 172.21.15.78:0/1838711597 conn(0x55d45dc074e0 msgr2=0x55d45dfb1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 -- 172.21.15.78:0/1838711597 shutdown_connections 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.000+0000 7f75e0fe8740 1 -- 172.21.15.78:0/1838711597 wait complete. 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 -- start start 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 Processor -- start 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfbfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d45dfb7530 con 0x55d45dc13390 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d45dea0dc0 con 0x55d45dbea4d0 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d45dfb76b0 con 0x55d45dbf26b0 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75e0fe8740 10 client.0 _ll_register_callbacks cb 0x55d45debc600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfbfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47818/0 (socket says 172.21.15.78:47818) 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfbfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:44010/0 (socket says 172.21.15.78:44010) 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.001+0000 7f75d3fff640 1 -- 172.21.15.78:0/2018418280 learned_addr learned my addr 172.21.15.78:0/2018418280 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d37fe640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d3fff640 1 -- 172.21.15.78:0/2018418280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 msgr2=0x55d45dfbfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d3fff640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfbfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d3fff640 1 -- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 msgr2=0x55d45dfbf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d3fff640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d3fff640 1 -- 172.21.15.78:0/2018418280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45dc2b3f0 con 0x55d45dc13390 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d37fe640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfbfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75d3fff640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc6ad0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f75cc004c70 tx=0x7f75cc010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dcfefcca5ff0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2018418280 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc004e50 con 0x55d45dc13390 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2018418280 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f75cc004210 con 0x55d45dc13390 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75cc003d10 con 0x55d45dc13390 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2018418280 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f75cc004410 con 0x55d45dc13390 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75e0fe8740 10 client.16962 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:40.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d45dcf57d0 con 0x55d45dc13390 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2018418280 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75cc003d10 con 0x55d45dc13390 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.002+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2018418280 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f75cc005110 con 0x55d45dc13390 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75e0fe8740 10 client.16962 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75e0fe8740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75e0fe8740 20 client.16962 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dbd1f60 0x55d45dfcd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75e0fe8740 4 client.16962 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d45df27990 con 0x55d45dbd1f60 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75d37fe640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dbd1f60 0x55d45dfcd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75d37fe640 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dbd1f60 0x55d45dfcd030 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2368a5b7c90d7529 server_cookie=733907c37d770772 in_seq=0 out_seq=0 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.003+0000 7f75b2ffd640 10 client.16962 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2018418280 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d45df27990 con 0x55d45dbd1f60 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75b2ffd640 10 client.16962 handle_command_reply: tid=0 2024-04-25T14:04:40.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 client.16962 shutdown 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 msgr2=0x55d45dfc6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc6ad0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f75cc004c70 tx=0x7f75cc010fd0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 shutdown_connections 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc6ad0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dbd1f60 0x55d45dfcd030 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfbfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2018418280 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75b2ffd640 0 client.16962 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 >> 172.21.15.78:0/2018418280 conn(0x55d45dc074e0 msgr2=0x55d45dfb3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 shutdown_connections 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2018418280 wait complete. 2024-04-25T14:04:40.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.004+0000 7f75e0fe8740 20 client.16962 trim_cache size 0 max 16384 2024-04-25T14:04:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 Processor -- start 2024-04-25T14:04:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 -- start start 2024-04-25T14:04:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dbf26b0 0x55d45dfc69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dc13390 0x55d45dfbfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41adb90 con 0x55d45dbf26b0 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41ab460 con 0x55d45dbea4d0 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41a92b0 con 0x55d45dc13390 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dbf26b0 0x55d45dfc69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dbf26b0 0x55d45dfc69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47826/0 (socket says 172.21.15.78:47826) 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3257579855 learned_addr learned my addr 172.21.15.78:0/3257579855 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d3fff640 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dc13390 0x55d45dfbfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d37fe640 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3257579855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dc13390 msgr2=0x55d45dfbfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dc13390 0x55d45dfbfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3257579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 msgr2=0x55d45dfbe320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbe320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.018+0000 7f75d2ffd640 1 -- 172.21.15.78:0/3257579855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45dfbadc0 con 0x55d45dbf26b0 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dbf26b0 0x55d45dfc69f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f75c80dc3c0 tx=0x7f75c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9104ef79f1fb0790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75b97fa640 1 -- 172.21.15.78:0/3257579855 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8020070 con 0x55d45dbf26b0 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75b97fa640 1 -- 172.21.15.78:0/3257579855 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c8009de0 con 0x55d45dbf26b0 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75b97fa640 1 -- 172.21.15.78:0/3257579855 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80a2710 con 0x55d45dbf26b0 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75e0fe8740 1 -- 172.21.15.78:0/3257579855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dbf26b0 msgr2=0x55d45dfc69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dbf26b0 0x55d45dfc69f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f75c80dc3c0 tx=0x7f75c8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75e0fe8740 1 -- 172.21.15.78:0/3257579855 shutdown_connections 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dc13390 0x55d45dfbfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dbf26b0 0x55d45dfc69f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/3257579855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfbe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.019+0000 7f75e0fe8740 1 -- 172.21.15.78:0/3257579855 >> 172.21.15.78:0/3257579855 conn(0x55d45dc074e0 msgr2=0x55d45dfbf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 -- 172.21.15.78:0/3257579855 shutdown_connections 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 -- 172.21.15.78:0/3257579855 wait complete. 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 -- start start 2024-04-25T14:04:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 Processor -- start 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfc6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfc5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41a9920 con 0x55d45dc13390 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41aad90 con 0x55d45dbea4d0 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41a9ff0 con 0x55d45dbf26b0 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75e0fe8740 10 client.0 _ll_register_callbacks cb 0x7f75cc009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47830/0 (socket says 172.21.15.78:47830) 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75d3fff640 1 -- 172.21.15.78:0/2442644916 learned_addr learned my addr 172.21.15.78:0/2442644916 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.020+0000 7f75d2ffd640 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfc5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75d37fe640 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfc6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75d3fff640 1 -- 172.21.15.78:0/2442644916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 msgr2=0x55d45dfc5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75d3fff640 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfc5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75d3fff640 1 -- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 msgr2=0x55d45dfc6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75d3fff640 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfc6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75d3fff640 1 -- 172.21.15.78:0/2442644916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45dfc5e70 con 0x55d45dc13390 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75d3fff640 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc56e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55d45dfb1700 tx=0x7f75cc009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01b39297621d664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2442644916 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc034070 con 0x55d45dc13390 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2442644916 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f75cc0057a0 con 0x55d45dc13390 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c8009de0 con 0x55d45dc13390 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2442644916 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f75cc0047c0 con 0x55d45dc13390 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75e0fe8740 10 client.16968 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.021+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d45e034370 con 0x55d45dc13390 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2442644916 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c8009de0 con 0x55d45dc13390 2024-04-25T14:04:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2442644916 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f75cc0116c0 con 0x55d45dc13390 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75e0fe8740 10 client.16968 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75e0fe8740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75e0fe8740 20 client.16968 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dba73a0 0x55d45e036d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75e0fe8740 4 client.16968 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d45df21800 con 0x55d45dba73a0 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75d37fe640 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dba73a0 0x55d45e036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75b2ffd640 10 client.16968 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.022+0000 7f75d37fe640 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dba73a0 0x55d45e036d80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef45eceff28bcd5 server_cookie=622abbc93e73d00a in_seq=0 out_seq=0 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75b2ffd640 1 -- 172.21.15.78:0/2442644916 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d45df21800 con 0x55d45dba73a0 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75b2ffd640 10 client.16968 handle_command_reply: tid=0 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 client.16968 shutdown 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 msgr2=0x55d45dfc56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc56e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55d45dfb1700 tx=0x7f75cc009600 comp rx=0 tx=0).stop 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 shutdown_connections 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d45dc13390 0x55d45dfc56e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d45dba73a0 0x55d45e036d80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45dbf26b0 0x55d45dfc5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 --2- 172.21.15.78:0/2442644916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d45dbea4d0 0x55d45dfc6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75b2ffd640 0 client.16968 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 >> 172.21.15.78:0/2442644916 conn(0x55d45dc074e0 msgr2=0x55d45dfb3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 shutdown_connections 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 1 -- 172.21.15.78:0/2442644916 wait complete. 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75e0fe8740 20 client.16968 trim_cache size 0 max 16384 2024-04-25T14:04:40.029 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:04:40.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75bc0afaf0 msgr2=0x7f75bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75bc0afaf0 0x7f75bc0b2000 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f75c4000c00 tx=0x7f75c40314c0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4108a20 msgr2=0x7f75d419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4108a20 0x7f75d419eae0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f75c800d950 tx=0x7f75c800de10 comp rx=0 tx=0).stop 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 shutdown_connections 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d419f020 0x7f75d41a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f75bc0afaf0 0x7f75bc0b2000 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75d4108a20 0x7f75d419eae0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.023+0000 7f75da051640 1 --2- 172.21.15.78:0/3417386052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75d4103ef0 0x7f75d4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.024+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 >> 172.21.15.78:0/3417386052 conn(0x7f75d40fd350 msgr2=0x7f75d40ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.024+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 shutdown_connections 2024-04-25T14:04:40.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:40.024+0000 7f75da051640 1 -- 172.21.15.78:0/3417386052 wait complete. 2024-04-25T14:04:40.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:40 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2923271752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:40.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:40 smithi129 ceph-mon[32569]: pgmap v503: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:40 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2923271752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:40 smithi078 ceph-mon[29163]: pgmap v503: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:40 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2923271752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:04:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:40 smithi112 ceph-mon[32617]: pgmap v503: 97 pgs: 97 active+clean; 3.5 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:41.035 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:04:41.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:04:41.035 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:04:41.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 -- 172.21.15.78:0/2228208902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4103c40 msgr2=0x7f58e4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 --2- 172.21.15.78:0/2228208902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4103c40 0x7f58e4104050 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009900 tx=0x7f58d802a410 comp rx=0 tx=0).stop 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 -- 172.21.15.78:0/2228208902 shutdown_connections 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 --2- 172.21.15.78:0/2228208902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e4108940 0x7f58e410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 --2- 172.21.15.78:0/2228208902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4103c40 0x7f58e4104050 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 --2- 172.21.15.78:0/2228208902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e4102a00 0x7f58e4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 -- 172.21.15.78:0/2228208902 >> 172.21.15.78:0/2228208902 conn(0x7f58e40fdbf0 msgr2=0x7f58e41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 -- 172.21.15.78:0/2228208902 shutdown_connections 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.167+0000 7f58ea584640 1 -- 172.21.15.78:0/2228208902 wait complete. 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 Processor -- start 2024-04-25T14:04:41.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 -- start start 2024-04-25T14:04:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e4102a00 0x7f58e419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e4103c40 0x7f58e419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4108940 0x7f58e41a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410f630 con 0x7f58e4108940 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410f4b0 con 0x7f58e4103c40 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.168+0000 7f58ea584640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410f7b0 con 0x7f58e4102a00 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4108940 0x7f58e41a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4108940 0x7f58e41a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47852/0 (socket says 172.21.15.78:47852) 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 -- 172.21.15.78:0/2199697493 learned_addr learned my addr 172.21.15.78:0/2199697493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e3fff640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e4102a00 0x7f58e419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e37fe640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e4103c40 0x7f58e419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 -- 172.21.15.78:0/2199697493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e4102a00 msgr2=0x7f58e419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e4102a00 0x7f58e419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 -- 172.21.15.78:0/2199697493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e4103c40 msgr2=0x7f58e419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e4103c40 0x7f58e419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 -- 172.21.15.78:0/2199697493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e41a1d70 con 0x7f58e4108940 2024-04-25T14:04:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e8b21640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4108940 0x7f58e41a1700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00eaa0 tx=0x7f58dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.169+0000 7f58e17fa640 1 -- 172.21.15.78:0/2199697493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00cdb0 con 0x7f58e4108940 2024-04-25T14:04:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.170+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e41a2060 con 0x7f58e4108940 2024-04-25T14:04:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.170+0000 7f58e17fa640 1 -- 172.21.15.78:0/2199697493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58dc005020 con 0x7f58e4108940 2024-04-25T14:04:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.170+0000 7f58e17fa640 1 -- 172.21.15.78:0/2199697493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0116d0 con 0x7f58e4108940 2024-04-25T14:04:41.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.170+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41a2600 con 0x7f58e4108940 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 Processor -- start 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58e17fa640 1 -- 172.21.15.78:0/2199697493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f58dc0054a0 con 0x7f58e4108940 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 -- start start 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa43410 0x55abcfde88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfde8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdefd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abcfd49b00 con 0x55abcfa22730 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55abcfce5920 con 0x55abcfa43410 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58f143a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55abcfd45ac0 con 0x55abcfa1a550 2024-04-25T14:04:41.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdefd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdefd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47856/0 (socket says 172.21.15.78:47856) 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58e37fe640 1 -- 172.21.15.78:0/3327323424 learned_addr learned my addr 172.21.15.78:0/3327323424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58e3fff640 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfde8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.173+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa43410 0x55abcfde88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e37fe640 1 -- 172.21.15.78:0/3327323424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 msgr2=0x55abcfde8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfde8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e17fa640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f58cc0afaf0 0x7f58cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e37fe640 1 -- 172.21.15.78:0/3327323424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa43410 msgr2=0x55abcfde88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa43410 0x55abcfde88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e37fe640 1 -- 172.21.15.78:0/3327323424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcfb237e0 con 0x55abcfa22730 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e8b21640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f58cc0afaf0 0x7f58cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdefd30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f58d8005f70 tx=0x7f58d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256e668ad0632d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58e17fa640 1 -- 172.21.15.78:0/2199697493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f58dc0ded40 con 0x7f58e4108940 2024-04-25T14:04:41.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58ca7fc640 1 -- 172.21.15.78:0/3327323424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8035040 con 0x55abcfa22730 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58ca7fc640 1 -- 172.21.15.78:0/3327323424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d8031720 con 0x55abcfa22730 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.174+0000 7f58ca7fc640 1 -- 172.21.15.78:0/3327323424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8003af0 con 0x55abcfa22730 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 -- 172.21.15.78:0/3327323424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 msgr2=0x55abcfdefd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58e8b21640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f58cc0afaf0 0x7f58cc0b2000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0a71f0 tx=0x7f58dc0d5900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdefd30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f58d8005f70 tx=0x7f58d8000bc0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 -- 172.21.15.78:0/3327323424 shutdown_connections 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdefd30 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfde8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 --2- 172.21.15.78:0/3327323424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa43410 0x55abcfde88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 -- 172.21.15.78:0/3327323424 >> 172.21.15.78:0/3327323424 conn(0x55abcfa37560 msgr2=0x55abcfddfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 -- 172.21.15.78:0/3327323424 shutdown_connections 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 -- 172.21.15.78:0/3327323424 wait complete. 2024-04-25T14:04:41.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.175+0000 7f58f143a740 1 -- start start 2024-04-25T14:04:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 1 Processor -- start 2024-04-25T14:04:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdf2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa43410 0x55abcfdf9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55abcfdf0720 con 0x55abcfa22730 2024-04-25T14:04:41.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55abcfce35d0 con 0x55abcfa1a550 2024-04-25T14:04:41.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55abcfdf08a0 con 0x55abcfa43410 2024-04-25T14:04:41.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdf2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdf2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47866/0 (socket says 172.21.15.78:47866) 2024-04-25T14:04:41.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58f143a740 10 client.0 _ll_register_callbacks cb 0x55abcfcd1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:41.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 -- 172.21.15.78:0/703065774 learned_addr learned my addr 172.21.15.78:0/703065774 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e37fe640 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa43410 0x55abcfdf9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e37fe640 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 -- 172.21.15.78:0/703065774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa43410 msgr2=0x55abcfdf9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa43410 0x55abcfdf9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 -- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 msgr2=0x55abcfdf2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.176+0000 7f58e3fff640 1 -- 172.21.15.78:0/703065774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcfb83f70 con 0x55abcfa22730 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58e3fff640 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdf2bc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f58d400bad0 tx=0x7f58d400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2a45a345dd377e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58c37fe640 1 -- 172.21.15.78:0/703065774 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4003e20 con 0x55abcfa22730 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58c37fe640 1 -- 172.21.15.78:0/703065774 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f58d4011440 con 0x55abcfa22730 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d8031720 con 0x55abcfa22730 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58c37fe640 1 -- 172.21.15.78:0/703065774 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f58d4011640 con 0x55abcfa22730 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 10 client.16980 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55abcfa1b170 con 0x55abcfa22730 2024-04-25T14:04:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58c37fe640 1 -- 172.21.15.78:0/703065774 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d8031720 con 0x55abcfa22730 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58c37fe640 1 -- 172.21.15.78:0/703065774 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f58d4024050 con 0x55abcfa22730 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 10 client.16980 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 20 client.16980 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcfa021f0 0x55abcfe00030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 4 client.16980 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.177+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abcfd57710 con 0x55abcfa021f0 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.178+0000 7f58e37fe640 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcfa021f0 0x55abcfe00030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.178+0000 7f58c37fe640 10 client.16980 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.178+0000 7f58e37fe640 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcfa021f0 0x55abcfe00030 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a99ffb8de8ab65 server_cookie=850b53fcc142d080 in_seq=0 out_seq=0 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58c37fe640 1 -- 172.21.15.78:0/703065774 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55abcfd57710 con 0x55abcfa021f0 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58c37fe640 10 client.16980 handle_command_reply: tid=0 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 client.16980 shutdown 2024-04-25T14:04:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 msgr2=0x55abcfdf2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdf2bc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f58d400bad0 tx=0x7f58d400bf90 comp rx=0 tx=0).stop 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 shutdown_connections 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcfa021f0 0x55abcfe00030 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa43410 0x55abcfdf9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa22730 0x55abcfdf2bc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 --2- 172.21.15.78:0/703065774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58c37fe640 0 client.16980 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 >> 172.21.15.78:0/703065774 conn(0x55abcfa37560 msgr2=0x55abcfddeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 shutdown_connections 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 1 -- 172.21.15.78:0/703065774 wait complete. 2024-04-25T14:04:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.179+0000 7f58f143a740 20 client.16980 trim_cache size 0 max 16384 2024-04-25T14:04:41.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.197+0000 7f58f143a740 1 Processor -- start 2024-04-25T14:04:41.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.197+0000 7f58f143a740 1 -- start start 2024-04-25T14:04:41.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58f143a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfdf1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58f143a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa22730 0x55abcfded1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58f143a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58f143a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41aba00 con 0x55abcfa43410 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58f143a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41a92d0 con 0x55abcfa22730 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58f143a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41a7120 con 0x55abcfa1a550 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47878/0 (socket says 172.21.15.78:47878) 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 -- 172.21.15.78:0/282431035 learned_addr learned my addr 172.21.15.78:0/282431035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfdf1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e3fff640 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa22730 0x55abcfded1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 -- 172.21.15.78:0/282431035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 msgr2=0x55abcfdf1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfdf1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 -- 172.21.15.78:0/282431035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa22730 msgr2=0x55abcfded1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa22730 0x55abcfded1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 -- 172.21.15.78:0/282431035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcfde0290 con 0x55abcfa43410 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58e37fe640 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf9a40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009590 tx=0x7f58d8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f9ff2307d363c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58c9ffb640 1 -- 172.21.15.78:0/282431035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8035040 con 0x55abcfa43410 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58c9ffb640 1 -- 172.21.15.78:0/282431035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d803d5d0 con 0x55abcfa43410 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.198+0000 7f58c9ffb640 1 -- 172.21.15.78:0/282431035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80554a0 con 0x55abcfa43410 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 -- 172.21.15.78:0/282431035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 msgr2=0x55abcfdf9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf9a40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009590 tx=0x7f58d8009520 comp rx=0 tx=0).stop 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 -- 172.21.15.78:0/282431035 shutdown_connections 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf9a40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa22730 0x55abcfded1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 --2- 172.21.15.78:0/282431035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa1a550 0x55abcfdf1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 -- 172.21.15.78:0/282431035 >> 172.21.15.78:0/282431035 conn(0x55abcfa37560 msgr2=0x55abcfdf0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 -- 172.21.15.78:0/282431035 shutdown_connections 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.199+0000 7f58f143a740 1 -- 172.21.15.78:0/282431035 wait complete. 2024-04-25T14:04:41.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 -- start start 2024-04-25T14:04:41.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 Processor -- start 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa22730 0x55abcfdf8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41a7790 con 0x55abcfa43410 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41a8c00 con 0x55abcfa1a550 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41a7e60 con 0x55abcfa22730 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58f143a740 10 client.0 _ll_register_callbacks cb 0x7f58d40043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58e37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47890/0 (socket says 172.21.15.78:47890) 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58e37fe640 1 -- 172.21.15.78:0/3480163033 learned_addr learned my addr 172.21.15.78:0/3480163033 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58e3fff640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa22730 0x55abcfdf8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.200+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58e37fe640 1 -- 172.21.15.78:0/3480163033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa22730 msgr2=0x55abcfdf8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa22730 0x55abcfdf8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58e37fe640 1 -- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 msgr2=0x55abcfdf7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58e37fe640 1 -- 172.21.15.78:0/3480163033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcfdf9760 con 0x55abcfa43410 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58e3fff640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa22730 0x55abcfdf8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58e37fe640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf91a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f58d802a930 tx=0x7f58d8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853d8fe584705d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58c37fe640 1 -- 172.21.15.78:0/3480163033 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8035040 con 0x55abcfa43410 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58c37fe640 1 -- 172.21.15.78:0/3480163033 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f58d803d980 con 0x55abcfa43410 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d803d5d0 con 0x55abcfa43410 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58c37fe640 1 -- 172.21.15.78:0/3480163033 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f58d8031720 con 0x55abcfa43410 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 10 client.16986 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55abcfe68500 con 0x55abcfa43410 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58c37fe640 1 -- 172.21.15.78:0/3480163033 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d803d5d0 con 0x55abcfa43410 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58c37fe640 1 -- 172.21.15.78:0/3480163033 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f58d805f030 con 0x55abcfa43410 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 10 client.16986 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 20 client.16986 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcf9d7630 0x55abcfe6a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 4 client.16986 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.201+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55abcfd51580 con 0x55abcf9d7630 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.202+0000 7f58e3fff640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcf9d7630 0x55abcfe6a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.202+0000 7f58c37fe640 10 client.16986 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.202+0000 7f58e3fff640 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcf9d7630 0x55abcfe6a940 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67020a1b51c46eb5 server_cookie=93919cd72b5bbe0f in_seq=0 out_seq=0 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.203+0000 7f58c37fe640 1 -- 172.21.15.78:0/3480163033 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55abcfd51580 con 0x55abcf9d7630 2024-04-25T14:04:41.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.203+0000 7f58c37fe640 10 client.16986 handle_command_reply: tid=0 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.203+0000 7f58f143a740 1 client.16986 shutdown 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 msgr2=0x55abcfdf91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf91a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f58d802a930 tx=0x7f58d8031400 comp rx=0 tx=0).stop 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 shutdown_connections 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abcfa43410 0x55abcfdf91a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55abcf9d7630 0x55abcfe6a940 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcfa22730 0x55abcfdf8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58c37fe640 0 client.16986 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 --2- 172.21.15.78:0/3480163033 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55abcfa1a550 0x55abcfdf7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 >> 172.21.15.78:0/3480163033 conn(0x55abcfa37560 msgr2=0x55abcfe04820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 shutdown_connections 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 1 -- 172.21.15.78:0/3480163033 wait complete. 2024-04-25T14:04:41.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58f143a740 20 client.16986 trim_cache size 0 max 16384 2024-04-25T14:04:41.210 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:04:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:04:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "4febbd7c-43b3-4723-8f54-dfe8767f0b98", 2024-04-25T14:04:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:04:41.210 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f58cc0afaf0 msgr2=0x7f58cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.204+0000 7f58ea584640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f58cc0afaf0 0x7f58cc0b2000 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0a71f0 tx=0x7f58dc0d5900 comp rx=0 tx=0).stop 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4108940 msgr2=0x7f58e41a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4108940 0x7f58e41a1700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00eaa0 tx=0x7f58dc00ef60 comp rx=0 tx=0).stop 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 shutdown_connections 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f58cc0afaf0 0x7f58cc0b2000 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58e4108940 0x7f58e41a1700 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58e4103c40 0x7f58e419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 --2- 172.21.15.78:0/2199697493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e4102a00 0x7f58e419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 >> 172.21.15.78:0/2199697493 conn(0x7f58e40fdbf0 msgr2=0x7f58e4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 shutdown_connections 2024-04-25T14:04:41.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.205+0000 7f58ea584640 1 -- 172.21.15.78:0/2199697493 wait complete. 2024-04-25T14:04:41.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:04:41.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 -- 172.21.15.78:0/3632205068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f0103c30 msgr2=0x7f67f0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 --2- 172.21.15.78:0/3632205068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f0103c30 0x7f67f0104040 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009900 tx=0x7f67e402a410 comp rx=0 tx=0).stop 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 -- 172.21.15.78:0/3632205068 shutdown_connections 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 --2- 172.21.15.78:0/3632205068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f0108930 0x7f67f010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 --2- 172.21.15.78:0/3632205068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f0103c30 0x7f67f0104040 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 --2- 172.21.15.78:0/3632205068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67f01029f0 0x7f67f01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 -- 172.21.15.78:0/3632205068 >> 172.21.15.78:0/3632205068 conn(0x7f67f00fdbc0 msgr2=0x7f67f0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 -- 172.21.15.78:0/3632205068 shutdown_connections 2024-04-25T14:04:41.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.327+0000 7f67f741a640 1 -- 172.21.15.78:0/3632205068 wait complete. 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 Processor -- start 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 -- start start 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f01029f0 0x7f67f0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f0103c30 0x7f67f0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67f0108930 0x7f67f01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010e4f0 con 0x7f67f01029f0 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010e370 con 0x7f67f0108930 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f741a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010e670 con 0x7f67f0103c30 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f01029f0 0x7f67f0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f01029f0 0x7f67f0199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47906/0 (socket says 172.21.15.78:47906) 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 -- 172.21.15.78:0/324142567 learned_addr learned my addr 172.21.15.78:0/324142567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f49b5640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f0103c30 0x7f67f0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f59b7640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67f0108930 0x7f67f01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 -- 172.21.15.78:0/324142567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f0103c30 msgr2=0x7f67f0199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f0103c30 0x7f67f0199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 -- 172.21.15.78:0/324142567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67f0108930 msgr2=0x7f67f01a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67f0108930 0x7f67f01a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.328+0000 7f67f51b6640 1 -- 172.21.15.78:0/324142567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f01a0c60 con 0x7f67f01029f0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.329+0000 7f67f51b6640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f01029f0 0x7f67f0199120 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e9f0 tx=0x7f67e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.329+0000 7f67ee7fc640 1 -- 172.21.15.78:0/324142567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800ccc0 con 0x7f67f01029f0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.329+0000 7f67ee7fc640 1 -- 172.21.15.78:0/324142567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e80035f0 con 0x7f67f01029f0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.329+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f01a0f50 con 0x7f67f01029f0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.329+0000 7f67ee7fc640 1 -- 172.21.15.78:0/324142567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800ce60 con 0x7f67f01029f0 2024-04-25T14:04:41.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.329+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01a1410 con 0x7f67f01029f0 2024-04-25T14:04:41.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 Processor -- start 2024-04-25T14:04:41.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 -- start start 2024-04-25T14:04:41.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67ee7fc640 1 -- 172.21.15.78:0/324142567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f67e8003aa0 con 0x7f67f01029f0 2024-04-25T14:04:41.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec799e390 0x561ec7d3fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec79754d0 0x561ec7d3ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d40c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ec7ca4d80 con 0x561ec797d6b0 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ec7c5e8b0 con 0x561ec79754d0 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ec7ca0d40 con 0x561ec799e390 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d40c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f51b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec79754d0 0x561ec7d3ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d40c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47918/0 (socket says 172.21.15.78:47918) 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f59b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec799e390 0x561ec7d3fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 -- 172.21.15.78:0/3027218555 learned_addr learned my addr 172.21.15.78:0/3027218555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 -- 172.21.15.78:0/3027218555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec799e390 msgr2=0x561ec7d3fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec799e390 0x561ec7d3fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 -- 172.21.15.78:0/3027218555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec79754d0 msgr2=0x561ec7d3ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec79754d0 0x561ec7d3ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67ee7fc640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67d80afb70 0x7f67d80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f49b5640 1 -- 172.21.15.78:0/3027218555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ec7c2fa40 con 0x561ec797d6b0 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.332+0000 7f67f59b7640 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec799e390 0x561ec7d3fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.333+0000 7f67f59b7640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67d80afb70 0x7f67d80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.333+0000 7f67f49b5640 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d40c70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f67e4031500 tx=0x7f67e4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da7ed99c12245a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.333+0000 7f67ee7fc640 1 -- 172.21.15.78:0/324142567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f67e80deeb0 con 0x7f67f01029f0 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.333+0000 7f67d77fe640 1 -- 172.21.15.78:0/3027218555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4035030 con 0x561ec797d6b0 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.333+0000 7f67d77fe640 1 -- 172.21.15.78:0/3027218555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e4040070 con 0x561ec797d6b0 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.333+0000 7f67d77fe640 1 -- 172.21.15.78:0/3027218555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4003970 con 0x561ec797d6b0 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3027218555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 msgr2=0x561ec7d40c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d40c70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f67e4031500 tx=0x7f67e4009640 comp rx=0 tx=0).stop 2024-04-25T14:04:41.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67f59b7640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67d80afb70 0x7f67d80b2080 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x561ec7d40ab0 tx=0x7f67e0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3027218555 shutdown_connections 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d40c70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec79754d0 0x561ec7d3ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3027218555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec799e390 0x561ec7d3fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3027218555 >> 172.21.15.78:0/3027218555 conn(0x561ec79924e0 msgr2=0x561ec7d3b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3027218555 shutdown_connections 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3027218555 wait complete. 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 -- start start 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.334+0000 7f67fe3a1740 1 Processor -- start 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec79754d0 0x561ec7d49a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d49f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec799e390 0x561ec7d50e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ec7d41720 con 0x561ec797d6b0 2024-04-25T14:04:41.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ec7c2bd10 con 0x561ec799e390 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ec7d418a0 con 0x561ec79754d0 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67fe3a1740 10 client.0 _ll_register_callbacks cb 0x561ec7c58210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d49f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d49f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47934/0 (socket says 172.21.15.78:47934) 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 -- 172.21.15.78:0/1107828323 learned_addr learned my addr 172.21.15.78:0/1107828323 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f59b7640 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec799e390 0x561ec7d50e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f51b6640 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec79754d0 0x561ec7d49a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 -- 172.21.15.78:0/1107828323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec79754d0 msgr2=0x561ec7d49a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec79754d0 0x561ec7d49a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 -- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec799e390 msgr2=0x561ec7d50e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec799e390 0x561ec7d50e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 -- 172.21.15.78:0/1107828323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ec7a807d0 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67f49b5640 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d49f40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f67e40042f0 tx=0x7f67e4004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b75ddc260cc672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.335+0000 7f67d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67d4ff9640 1 -- 172.21.15.78:0/1107828323 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4035070 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e4040070 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67d4ff9640 1 -- 172.21.15.78:0/1107828323 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f67e40036f0 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67d4ff9640 1 -- 172.21.15.78:0/1107828323 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f67e40038f0 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 10 client.16998 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561ec79b63f0 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67d4ff9640 1 -- 172.21.15.78:0/1107828323 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e4040070 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67d4ff9640 1 -- 172.21.15.78:0/1107828323 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f67e40459e0 con 0x561ec797d6b0 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 10 client.16998 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 20 client.16998 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec795cf60 0x561ec7d56ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 4 client.16998 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ec7cb2990 con 0x561ec795cf60 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.336+0000 7f67f51b6640 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec795cf60 0x561ec7d56ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.337+0000 7f67d4ff9640 10 client.16998 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.337+0000 7f67f51b6640 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec795cf60 0x561ec7d56ef0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea71428f39d02cf3 server_cookie=2b833a4e3ac5747a in_seq=0 out_seq=0 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.337+0000 7f67d4ff9640 1 -- 172.21.15.78:0/1107828323 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ec7cb2990 con 0x561ec795cf60 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.337+0000 7f67d4ff9640 10 client.16998 handle_command_reply: tid=0 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.337+0000 7f67fe3a1740 1 client.16998 shutdown 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 msgr2=0x561ec7d49f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d49f40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f67e40042f0 tx=0x7f67e4004320 comp rx=0 tx=0).stop 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 shutdown_connections 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec799e390 0x561ec7d50e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec797d6b0 0x561ec7d49f40 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec795cf60 0x561ec7d56ef0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67d4ff9640 0 client.16998 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/1107828323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec79754d0 0x561ec7d49a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 >> 172.21.15.78:0/1107828323 conn(0x561ec79924e0 msgr2=0x561ec7d3e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 shutdown_connections 2024-04-25T14:04:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 1 -- 172.21.15.78:0/1107828323 wait complete. 2024-04-25T14:04:41.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.338+0000 7f67fe3a1740 20 client.16998 trim_cache size 0 max 16384 2024-04-25T14:04:41.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 Processor -- start 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 -- start start 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec795cf60 0x561ec7d3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d3b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec797d6b0 0x561ec7d50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01aca40 con 0x561ec79754d0 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01aa310 con 0x561ec797d6b0 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01a8160 con 0x561ec795cf60 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d3b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d3b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47948/0 (socket says 172.21.15.78:47948) 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 -- 172.21.15.78:0/293669338 learned_addr learned my addr 172.21.15.78:0/293669338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f51b6640 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec795cf60 0x561ec7d3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 -- 172.21.15.78:0/293669338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec795cf60 msgr2=0x561ec7d3ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec795cf60 0x561ec7d3ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 -- 172.21.15.78:0/293669338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec797d6b0 msgr2=0x561ec7d50d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec797d6b0 0x561ec7d50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.356+0000 7f67f49b5640 1 -- 172.21.15.78:0/293669338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ec79760f0 con 0x561ec79754d0 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.357+0000 7f67f49b5640 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d3b170 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f67e4045dd0 tx=0x7f67e4046fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86851f5f818b45a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.357+0000 7f67d6ffd640 1 -- 172.21.15.78:0/293669338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4035040 con 0x561ec79754d0 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.357+0000 7f67d6ffd640 1 -- 172.21.15.78:0/293669338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e4040070 con 0x561ec79754d0 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.357+0000 7f67d6ffd640 1 -- 172.21.15.78:0/293669338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4045760 con 0x561ec79754d0 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.357+0000 7f67fe3a1740 1 -- 172.21.15.78:0/293669338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 msgr2=0x561ec7d3b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.357+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d3b170 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f67e4045dd0 tx=0x7f67e4046fd0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- 172.21.15.78:0/293669338 shutdown_connections 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec797d6b0 0x561ec7d50d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d3b170 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/293669338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec795cf60 0x561ec7d3ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- 172.21.15.78:0/293669338 >> 172.21.15.78:0/293669338 conn(0x561ec79924e0 msgr2=0x561ec7d44760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- 172.21.15.78:0/293669338 shutdown_connections 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- 172.21.15.78:0/293669338 wait complete. 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- start start 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 Processor -- start 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec795cf60 0x561ec7d4e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d4e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec797d6b0 0x561ec7dbd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01a87d0 con 0x561ec79754d0 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01a9c40 con 0x561ec795cf60 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01a8ea0 con 0x561ec797d6b0 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67fe3a1740 10 client.0 _ll_register_callbacks cb 0x7f67e80e2860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:41.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d4e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67f49b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d4e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47954/0 (socket says 172.21.15.78:47954) 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.358+0000 7f67f49b5640 1 -- 172.21.15.78:0/3698647913 learned_addr learned my addr 172.21.15.78:0/3698647913 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f59b7640 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec797d6b0 0x561ec7dbd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f51b6640 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec795cf60 0x561ec7d4e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f49b5640 1 -- 172.21.15.78:0/3698647913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec797d6b0 msgr2=0x561ec7dbd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f49b5640 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec797d6b0 0x561ec7dbd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f49b5640 1 -- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec795cf60 msgr2=0x561ec7d4e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f49b5640 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec795cf60 0x561ec7d4e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f49b5640 1 -- 172.21.15.78:0/3698647913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ec7c654b0 con 0x561ec79754d0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67f49b5640 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d4e6e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009900 tx=0x7f67e4003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6281265d0ff85b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67d4ff9640 1 -- 172.21.15.78:0/3698647913 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4035030 con 0x561ec79754d0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e4040070 con 0x561ec79754d0 2024-04-25T14:04:41.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67d4ff9640 1 -- 172.21.15.78:0/3698647913 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f67e40456e0 con 0x561ec79754d0 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67d4ff9640 1 -- 172.21.15.78:0/3698647913 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f67e40458e0 con 0x561ec79754d0 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67fe3a1740 10 client.17004 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.359+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561ec7dbe2d0 con 0x561ec79754d0 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67d4ff9640 1 -- 172.21.15.78:0/3698647913 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e4040070 con 0x561ec79754d0 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67d4ff9640 1 -- 172.21.15.78:0/3698647913 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f67e4061070 con 0x561ec79754d0 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67fe3a1740 10 client.17004 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67fe3a1740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67fe3a1740 20 client.17004 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec799e390 0x561ec7dc0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67fe3a1740 4 client.17004 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ec7cac800 con 0x561ec799e390 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67f51b6640 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec799e390 0x561ec7dc0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:41.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67d4ff9640 10 client.17004 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.360+0000 7f67f51b6640 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec799e390 0x561ec7dc0af0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbfecd947adddc93 server_cookie=bcc35d0c891919ed in_seq=0 out_seq=0 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67d4ff9640 1 -- 172.21.15.78:0/3698647913 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561ec7cac800 con 0x561ec799e390 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67d4ff9640 10 client.17004 handle_command_reply: tid=0 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 client.17004 shutdown 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 msgr2=0x561ec7d4e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d4e6e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009900 tx=0x7f67e4003fd0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 shutdown_connections 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ec799e390 0x561ec7dc0af0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec797d6b0 0x561ec7dbd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ec79754d0 0x561ec7d4e6e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 --2- 172.21.15.78:0/3698647913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ec795cf60 0x561ec7d4e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67d4ff9640 0 client.17004 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 >> 172.21.15.78:0/3698647913 conn(0x561ec79924e0 msgr2=0x561ec7d3ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 shutdown_connections 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 1 -- 172.21.15.78:0/3698647913 wait complete. 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67fe3a1740 20 client.17004 trim_cache size 0 max 16384 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "4febbd7c-43b3-4723-8f54-dfe8767f0b98": { 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "4febbd7c-43b3-4723-8f54-dfe8767f0b98", 2024-04-25T14:04:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:04:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:04:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:04:41.369 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67d80afb70 msgr2=0x7f67d80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67f741a640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67d80afb70 0x7f67d80b2080 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x561ec7d40ab0 tx=0x7f67e0009500 comp rx=0 tx=0).stop 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f01029f0 msgr2=0x7f67f0199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.361+0000 7f67f741a640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f01029f0 0x7f67f0199120 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e9f0 tx=0x7f67e800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 shutdown_connections 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67f0108930 0x7f67f01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67f0103c30 0x7f67f0199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67d80afb70 0x7f67d80b2080 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 --2- 172.21.15.78:0/324142567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f01029f0 0x7f67f0199120 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 >> 172.21.15.78:0/324142567 conn(0x7f67f00fdbc0 msgr2=0x7f67f0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 shutdown_connections 2024-04-25T14:04:41.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:04:41.362+0000 7f67f741a640 1 -- 172.21.15.78:0/324142567 wait complete. 2024-04-25T14:04:41.372 INFO:tasks.cephfs.filesystem:scrub status for tag:4febbd7c-43b3-4723-8f54-dfe8767f0b98 - {'path': '/', 'tag': '4febbd7c-43b3-4723-8f54-dfe8767f0b98', 'options': 'recursive,force'} 2024-04-25T14:04:42.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:42 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:04:42.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:42 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:42.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:42 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:04:42.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:42 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:42 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:04:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:42 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:42 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:04:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:42 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:42 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:04:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:42 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:42 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:04:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:42 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:04:43.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:43 smithi129 ceph-mon[32569]: pgmap v504: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:43 smithi078 ceph-mon[29163]: pgmap v504: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:43.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:43 smithi112 ceph-mon[32617]: pgmap v504: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:44 smithi129 ceph-mon[32569]: pgmap v505: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:44 smithi078 ceph-mon[29163]: pgmap v505: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:44 smithi112 ceph-mon[32617]: pgmap v505: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 -- 172.21.15.78:0/2748842369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94108960 msgr2=0x7f9f9410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 --2- 172.21.15.78:0/2748842369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94108960 0x7f9f9410ad80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009ca0 tx=0x7f9f8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 -- 172.21.15.78:0/2748842369 shutdown_connections 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 --2- 172.21.15.78:0/2748842369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94108960 0x7f9f9410ad80 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 --2- 172.21.15.78:0/2748842369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f94103c60 0x7f9f94104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 --2- 172.21.15.78:0/2748842369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f94102a20 0x7f9f94103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 -- 172.21.15.78:0/2748842369 >> 172.21.15.78:0/2748842369 conn(0x7f9f940fdc10 msgr2=0x7f9f941000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 -- 172.21.15.78:0/2748842369 shutdown_connections 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.318+0000 7f9f9a100640 1 -- 172.21.15.78:0/2748842369 wait complete. 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 Processor -- start 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 -- start start 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94102a20 0x7f9f941990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f94103c60 0x7f9f94199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f94108960 0x7f9f941a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9410e550 con 0x7f9f94102a20 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9410e3d0 con 0x7f9f94103c60 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f9a100640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9410e6d0 con 0x7f9f94108960 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.319+0000 7f9f937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94102a20 0x7f9f941990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94102a20 0x7f9f941990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47972/0 (socket says 172.21.15.78:47972) 2024-04-25T14:04:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 -- 172.21.15.78:0/1818405068 learned_addr learned my addr 172.21.15.78:0/1818405068 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f94103c60 0x7f9f94199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f94108960 0x7f9f941a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 -- 172.21.15.78:0/1818405068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f94108960 msgr2=0x7f9f941a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f94108960 0x7f9f941a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 -- 172.21.15.78:0/1818405068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f94103c60 msgr2=0x7f9f94199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f94103c60 0x7f9f94199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 -- 172.21.15.78:0/1818405068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f941a0c10 con 0x7f9f94102a20 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f94108960 0x7f9f941a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f94103c60 0x7f9f94199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94102a20 0x7f9f941990d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400e9f0 tx=0x7f9f8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f90ff9640 1 -- 172.21.15.78:0/1818405068 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400ccc0 con 0x7f9f94102a20 2024-04-25T14:04:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f941a0f00 con 0x7f9f94102a20 2024-04-25T14:04:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f90ff9640 1 -- 172.21.15.78:0/1818405068 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f840035f0 con 0x7f9f94102a20 2024-04-25T14:04:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f90ff9640 1 -- 172.21.15.78:0/1818405068 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400ce60 con 0x7f9f94102a20 2024-04-25T14:04:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.320+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f941a1360 con 0x7f9f94102a20 2024-04-25T14:04:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f90ff9640 1 -- 172.21.15.78:0/1818405068 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9f84003aa0 con 0x7f9f94102a20 2024-04-25T14:04:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 Processor -- start 2024-04-25T14:04:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 -- start start 2024-04-25T14:04:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e97bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e97c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e983000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55854e8df460 con 0x55854e5b84d0 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55854e899590 con 0x55854e5b02f0 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55854e8db420 con 0x55854e5d91b0 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e983000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e983000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47986/0 (socket says 172.21.15.78:47986) 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1250226911 learned_addr learned my addr 172.21.15.78:0/1250226911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e97bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e97c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f90ff9640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f7c0afae0 0x7f9f7c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1250226911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 msgr2=0x55854e97bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e97bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1250226911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 msgr2=0x55854e97c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e97c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1250226911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55854e8a60d0 con 0x55854e5b84d0 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e97c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.323+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f7c0afae0 0x7f9f7c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.324+0000 7f9f90ff9640 1 -- 172.21.15.78:0/1818405068 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9f840df0c0 con 0x7f9f94102a20 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.324+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e983000 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009cf0 tx=0x7f9f880117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205602e05c561937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.324+0000 7f9f79ffb640 1 -- 172.21.15.78:0/1250226911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88012450 con 0x55854e5b84d0 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.324+0000 7f9f79ffb640 1 -- 172.21.15.78:0/1250226911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f88012aa0 con 0x55854e5b84d0 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.324+0000 7f9f79ffb640 1 -- 172.21.15.78:0/1250226911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800fc20 con 0x55854e5b84d0 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1250226911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 msgr2=0x55854e983000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f7c0afae0 0x7f9f7c0b1ff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0094e0 tx=0x7f9f8c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e983000 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009cf0 tx=0x7f9f880117b0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1250226911 shutdown_connections 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e983000 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e97c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1250226911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e97bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1250226911 >> 172.21.15.78:0/1250226911 conn(0x55854e5cd300 msgr2=0x55854e975300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1250226911 shutdown_connections 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1250226911 wait complete. 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 -- start start 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.325+0000 7f9fa0fb4740 1 Processor -- start 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b02f0 0x55854e985820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e985d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55854e983a50 con 0x55854e5b84d0 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55854e866b60 con 0x55854e5d91b0 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55854e983bd0 con 0x55854e5b02f0 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9fa0fb4740 10 client.0 _ll_register_callbacks cb 0x55854e88b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e985d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e985d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47988/0 (socket says 172.21.15.78:47988) 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 -- 172.21.15.78:0/4109249150 learned_addr learned my addr 172.21.15.78:0/4109249150 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f937fe640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b02f0 0x55854e985820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f93fff640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 -- 172.21.15.78:0/4109249150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b02f0 msgr2=0x55854e985820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b02f0 0x55854e985820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 -- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 msgr2=0x55854e98ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 -- 172.21.15.78:0/4109249150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55854e5f1210 con 0x55854e5b84d0 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f937fe640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b02f0 0x55854e985820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f93fff640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e985d60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9f8801c040 tx=0x7f9f88009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360065b4d029c003 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f72ffd640 1 -- 172.21.15.78:0/4109249150 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88028020 con 0x55854e5b84d0 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.326+0000 7f9f72ffd640 1 -- 172.21.15.78:0/4109249150 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9f88004310 con 0x55854e5b84d0 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f88012aa0 con 0x55854e5b84d0 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9f72ffd640 1 -- 172.21.15.78:0/4109249150 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9f88033050 con 0x55854e5b84d0 2024-04-25T14:04:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 10 client.17016 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55854e6bd620 con 0x55854e5b84d0 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9f72ffd640 1 -- 172.21.15.78:0/4109249150 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f88012aa0 con 0x55854e5b84d0 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9f72ffd640 1 -- 172.21.15.78:0/4109249150 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9f880050e0 con 0x55854e5b84d0 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 10 client.17016 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 20 client.17016 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e597d80 0x55854e993280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 4 client.17016 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.327+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55854e8ed070 con 0x55854e597d80 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.328+0000 7f9f937fe640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e597d80 0x55854e993280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.328+0000 7f9f72ffd640 10 client.17016 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.328+0000 7f9f937fe640 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e597d80 0x55854e993280 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c54929e777be937 server_cookie=898ddcfadb18aa32 in_seq=0 out_seq=0 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9f72ffd640 1 -- 172.21.15.78:0/4109249150 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55854e8ed070 con 0x55854e597d80 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9f72ffd640 10 client.17016 handle_command_reply: tid=0 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 client.17016 shutdown 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 msgr2=0x55854e985d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e985d60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9f8801c040 tx=0x7f9f88009860 comp rx=0 tx=0).stop 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 shutdown_connections 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e985d60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e597d80 0x55854e993280 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/4109249150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b02f0 0x55854e985820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 >> 172.21.15.78:0/4109249150 conn(0x55854e5cd300 msgr2=0x55854e974250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9f72ffd640 0 client.17016 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 shutdown_connections 2024-04-25T14:04:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/4109249150 wait complete. 2024-04-25T14:04:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.329+0000 7f9fa0fb4740 20 client.17016 trim_cache size 0 max 16384 2024-04-25T14:04:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 Processor -- start 2024-04-25T14:04:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 -- start start 2024-04-25T14:04:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e9753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e98cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e983fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f941ab9c0 con 0x55854e5b84d0 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f941a9290 con 0x55854e5b02f0 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.344+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f941a70e0 con 0x55854e5d91b0 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e98cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e98cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48004/0 (socket says 172.21.15.78:48004) 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1976917289 learned_addr learned my addr 172.21.15.78:0/1976917289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e9753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e983fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1976917289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 msgr2=0x55854e983fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e983fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1976917289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 msgr2=0x55854e9753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e9753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 -- 172.21.15.78:0/1976917289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55854e980690 con 0x55854e5b84d0 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e983fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e98cbe0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9f88012fd0 tx=0x7f9f88009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d036cbca3a4923c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f797fa640 1 -- 172.21.15.78:0/1976917289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88028040 con 0x55854e5b84d0 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f797fa640 1 -- 172.21.15.78:0/1976917289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8800f440 con 0x55854e5b84d0 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.345+0000 7f9f797fa640 1 -- 172.21.15.78:0/1976917289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800f960 con 0x55854e5b84d0 2024-04-25T14:04:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1976917289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 msgr2=0x55854e98cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e98cbe0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9f88012fd0 tx=0x7f9f88009280 comp rx=0 tx=0).stop 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1976917289 shutdown_connections 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5d91b0 0x55854e983fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b84d0 0x55854e98cbe0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1976917289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5b02f0 0x55854e9753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1976917289 >> 172.21.15.78:0/1976917289 conn(0x55854e5cd300 msgr2=0x55854e974cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1976917289 shutdown_connections 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1976917289 wait complete. 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.346+0000 7f9fa0fb4740 1 -- start start 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 1 Processor -- start 2024-04-25T14:04:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b02f0 0x55854e989f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b84d0 0x55854e98a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f941a7750 con 0x55854e5b02f0 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f941a8bc0 con 0x55854e5d91b0 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f941a7e20 con 0x55854e5b84d0 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9fa0fb4740 10 client.0 _ll_register_callbacks cb 0x7f9f88009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b02f0 0x55854e989f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b02f0 0x55854e989f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48020/0 (socket says 172.21.15.78:48020) 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 -- 172.21.15.78:0/1184816978 learned_addr learned my addr 172.21.15.78:0/1184816978 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b84d0 0x55854e98a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f93fff640 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 -- 172.21.15.78:0/1184816978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b84d0 msgr2=0x55854e98a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b84d0 0x55854e98a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 -- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 msgr2=0x55854e98b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 -- 172.21.15.78:0/1184816978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55854e98d120 con 0x55854e5b02f0 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f937fe640 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b02f0 0x55854e989f70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9f840dcbb0 tx=0x7f9f840a6e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7705f32f732b5c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.347+0000 7f9f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9f72ffd640 1 -- 172.21.15.78:0/1184816978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f840b0160 con 0x55854e5b02f0 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9f72ffd640 1 -- 172.21.15.78:0/1184816978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9f840e1110 con 0x55854e5b02f0 2024-04-25T14:04:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f8800f440 con 0x55854e5b02f0 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9f72ffd640 1 -- 172.21.15.78:0/1184816978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9f840a7c90 con 0x55854e5b02f0 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 10 client.17022 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55854e98caa0 con 0x55854e5b02f0 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9f72ffd640 1 -- 172.21.15.78:0/1184816978 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8800f440 con 0x55854e5b02f0 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9f72ffd640 1 -- 172.21.15.78:0/1184816978 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9f840a2f00 con 0x55854e5b02f0 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 10 client.17022 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 20 client.17022 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e56d1c0 0x55854e9fd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 4 client.17022 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.348+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55854e8e6ee0 con 0x55854e56d1c0 2024-04-25T14:04:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.349+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e56d1c0 0x55854e9fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.349+0000 7f9f72ffd640 10 client.17022 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.349+0000 7f9f92ffd640 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e56d1c0 0x55854e9fd910 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8937e5a5c4ae6aa2 server_cookie=7b989a0850f0a6fb in_seq=0 out_seq=0 2024-04-25T14:04:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9f72ffd640 1 -- 172.21.15.78:0/1184816978 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55854e8e6ee0 con 0x55854e56d1c0 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9f72ffd640 10 client.17022 handle_command_reply: tid=0 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 client.17022 shutdown 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b02f0 msgr2=0x55854e989f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b02f0 0x55854e989f70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9f840dcbb0 tx=0x7f9f840a6e30 comp rx=0 tx=0).stop 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 shutdown_connections 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55854e5d91b0 0x55854e98b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55854e56d1c0 0x55854e9fd910 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55854e5b84d0 0x55854e98a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9f72ffd640 0 client.17022 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 --2- 172.21.15.78:0/1184816978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55854e5b02f0 0x55854e989f70 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 >> 172.21.15.78:0/1184816978 conn(0x55854e5cd300 msgr2=0x55854e983fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 shutdown_connections 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 1 -- 172.21.15.78:0/1184816978 wait complete. 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.363+0000 7f9fa0fb4740 20 client.17022 trim_cache size 0 max 16384 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:04:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:04:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f7c0afae0 msgr2=0x7f9f7c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f7c0afae0 0x7f9f7c0b1ff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0094e0 tx=0x7f9f8c0095c0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94102a20 msgr2=0x7f9f941990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94102a20 0x7f9f941990d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400e9f0 tx=0x7f9f8400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 shutdown_connections 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f94108960 0x7f9f941a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f94103c60 0x7f9f94199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9f7c0afae0 0x7f9f7c0b1ff0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 --2- 172.21.15.78:0/1818405068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f94102a20 0x7f9f941990d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 >> 172.21.15.78:0/1818405068 conn(0x7f9f940fdc10 msgr2=0x7f9f94104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 shutdown_connections 2024-04-25T14:04:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:46.364+0000 7f9f9a100640 1 -- 172.21.15.78:0/1818405068 wait complete. 2024-04-25T14:04:46.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:46 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:04:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:46 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:04:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:46 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:04:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:47 smithi129 ceph-mon[32569]: pgmap v506: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:47 smithi078 ceph-mon[29163]: pgmap v506: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:47.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:47 smithi112 ceph-mon[32617]: pgmap v506: 97 pgs: 97 active+clean; 3.6 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:48 smithi129 ceph-mon[32569]: pgmap v507: 97 pgs: 97 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:04:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:48 smithi078 ceph-mon[29163]: pgmap v507: 97 pgs: 97 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:04:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:48 smithi112 ceph-mon[32617]: pgmap v507: 97 pgs: 97 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:04:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:50 smithi129 ceph-mon[32569]: pgmap v508: 97 pgs: 97 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:50 smithi078 ceph-mon[29163]: pgmap v508: 97 pgs: 97 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:50 smithi112 ceph-mon[32617]: pgmap v508: 97 pgs: 97 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:52 smithi129 ceph-mon[32569]: pgmap v509: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:04:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:52 smithi078 ceph-mon[29163]: pgmap v509: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:04:52.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:52 smithi112 ceph-mon[32617]: pgmap v509: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:04:53.422 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:189/600: open fd = 3 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:190/600: open fd = 3 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:191/600: open fd = 3 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.423 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:192/600: open fd = 3 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:193/600: open fd = 3 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:194/600: open fd = 3 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:195/600: open fd = 3 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.424 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:196/600: open fd = 3 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:197/600: open fd = 3 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:198/600: open fd = 3 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.425 INFO:tasks.workunit.client.0.smithi078.stdout:199/600: open fd = 3 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:200/600: open fd = 3 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:201/600: open fd = 3 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:202/600: open fd = 3 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.426 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:203/600: open fd = 3 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:204/600: open fd = 3 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:205/600: open fd = 3 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:206/600: open fd = 3 2024-04-25T14:04:53.427 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:207/600: open fd = 3 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:208/600: open fd = 3 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:209/600: open fd = 3 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.428 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:210/600: open fd = 3 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:211/600: open fd = 3 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:212/600: open fd = 3 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:213/600: open fd = 3 2024-04-25T14:04:53.429 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:214/600: open fd = 3 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:215/600: open fd = 3 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:216/600: open fd = 3 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.430 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:217/600: open fd = 3 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:218/600: open fd = 3 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:219/600: open fd = 3 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.431 INFO:tasks.workunit.client.0.smithi078.stdout:220/600: open fd = 3 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:221/600: open fd = 3 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:222/600: open fd = 3 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:223/600: open fd = 3 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.432 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:224/600: open fd = 3 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:225/600: open fd = 3 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:226/600: open fd = 3 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.433 INFO:tasks.workunit.client.0.smithi078.stdout:227/600: open fd = 3 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:228/600: open fd = 3 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:229/600: open fd = 3 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:230/600: open fd = 3 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.434 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:231/600: open fd = 3 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:232/600: open fd = 3 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:233/600: open fd = 3 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:234/600: open fd = 3 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:235/600: open fd = 3 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:236/600: open fd = 3 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:237/600: open fd = 3 2024-04-25T14:04:53.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:238/600: open fd = 3 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:239/600: open fd = 3 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:240/600: open fd = 3 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.437 INFO:tasks.workunit.client.0.smithi078.stdout:241/600: open fd = 3 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:242/600: open fd = 3 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:243/600: open fd = 3 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:244/600: open fd = 3 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:245/600: open fd = 3 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:246/600: open fd = 3 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:247/600: open fd = 3 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:248/600: open fd = 3 2024-04-25T14:04:53.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:249/600: open fd = 3 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:250/600: open fd = 3 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:53.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:04:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:54 smithi129 ceph-mon[32569]: pgmap v510: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:54.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:54 smithi078 ceph-mon[29163]: pgmap v510: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:54.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:54 smithi112 ceph-mon[32617]: pgmap v510: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:04:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:56 smithi129 ceph-mon[32569]: pgmap v511: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:56 smithi078 ceph-mon[29163]: pgmap v511: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:56 smithi112 ceph-mon[32617]: pgmap v511: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:04:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:04:58 smithi129 ceph-mon[32569]: pgmap v512: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:04:58 smithi078 ceph-mon[29163]: pgmap v512: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:58.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:04:58 smithi112 ceph-mon[32617]: pgmap v512: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 -- 172.21.15.78:0/1616651160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 msgr2=0x7f2b20104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 --2- 172.21.15.78:0/1616651160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20104070 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009900 tx=0x7f2b1002a410 comp rx=0 tx=0).stop 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 -- 172.21.15.78:0/1616651160 shutdown_connections 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 --2- 172.21.15.78:0/1616651160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b20108960 0x7f2b2010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 --2- 172.21.15.78:0/1616651160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20104070 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 --2- 172.21.15.78:0/1616651160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b20102a20 0x7f2b20103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 -- 172.21.15.78:0/1616651160 >> 172.21.15.78:0/1616651160 conn(0x7f2b200fdc10 msgr2=0x7f2b201000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 -- 172.21.15.78:0/1616651160 shutdown_connections 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.472+0000 7f2b24e51640 1 -- 172.21.15.78:0/1616651160 wait complete. 2024-04-25T14:04:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 Processor -- start 2024-04-25T14:04:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 -- start start 2024-04-25T14:04:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b20102a20 0x7f2b20199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b20108960 0x7f2b201a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2010e520 con 0x7f2b20103c60 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2010e3a0 con 0x7f2b20102a20 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.473+0000 7f2b24e51640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2010e6a0 con 0x7f2b20108960 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38234/0 (socket says 172.21.15.78:38234) 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 -- 172.21.15.78:0/2399949491 learned_addr learned my addr 172.21.15.78:0/2399949491 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b20102a20 0x7f2b20199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b20108960 0x7f2b201a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 -- 172.21.15.78:0/2399949491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b20108960 msgr2=0x7f2b201a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b20108960 0x7f2b201a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 -- 172.21.15.78:0/2399949491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b20102a20 msgr2=0x7f2b20199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b20102a20 0x7f2b20199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 -- 172.21.15.78:0/2399949491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b201a0c60 con 0x7f2b20103c60 2024-04-25T14:04:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20199660 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b0f7fe640 1 -- 172.21.15.78:0/2399949491 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1003f030 con 0x7f2b20103c60 2024-04-25T14:04:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b0f7fe640 1 -- 172.21.15.78:0/2399949491 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b10031910 con 0x7f2b20103c60 2024-04-25T14:04:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b0f7fe640 1 -- 172.21.15.78:0/2399949491 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10035070 con 0x7f2b20103c60 2024-04-25T14:04:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b201a0ef0 con 0x7f2b20103c60 2024-04-25T14:04:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.474+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201a12a0 con 0x7f2b20103c60 2024-04-25T14:04:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.477+0000 7f2b2bde4740 1 Processor -- start 2024-04-25T14:04:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.477+0000 7f2b2bde4740 1 -- start start 2024-04-25T14:04:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a5641b0 0x56158a904970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a53b2f0 0x56158a904ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56158a86a460 con 0x56158a5641b0 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56158a824590 con 0x56158a5434d0 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56158a866420 con 0x56158a53b2f0 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b0f7fe640 1 -- 172.21.15.78:0/2399949491 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2b10042030 con 0x7f2b20103c60 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a5641b0 0x56158a904970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a5641b0 0x56158a904970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38248/0 (socket says 172.21.15.78:38248) 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 -- 172.21.15.78:0/752246303 learned_addr learned my addr 172.21.15.78:0/752246303 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a53b2f0 0x56158a904ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 -- 172.21.15.78:0/752246303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a53b2f0 msgr2=0x56158a904ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a53b2f0 0x56158a904ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 -- 172.21.15.78:0/752246303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 msgr2=0x56158a90bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 -- 172.21.15.78:0/752246303 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56158a8310d0 con 0x56158a5641b0 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a53b2f0 0x56158a904ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.478+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a5641b0 0x56158a904970 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009fd0 tx=0x7f2b180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bf32d5c0489c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.479+0000 7f2b0f7fe640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2b080afae0 0x7f2b080b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.479+0000 7f2b0cff9640 1 -- 172.21.15.78:0/752246303 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18004d20 con 0x56158a5641b0 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.479+0000 7f2b0cff9640 1 -- 172.21.15.78:0/752246303 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b18004050 con 0x56158a5641b0 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.479+0000 7f2b0cff9640 1 -- 172.21.15.78:0/752246303 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18005280 con 0x56158a5641b0 2024-04-25T14:04:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.479+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2b080afae0 0x7f2b080b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.479+0000 7f2b0f7fe640 1 -- 172.21.15.78:0/2399949491 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2b100ff0d0 con 0x7f2b20103c60 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 -- 172.21.15.78:0/752246303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a5641b0 msgr2=0x56158a904970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2b080afae0 0x7f2b080b1ff0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56158a905d50 tx=0x7f2b14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a5641b0 0x56158a904970 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009fd0 tx=0x7f2b180118d0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 -- 172.21.15.78:0/752246303 shutdown_connections 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a53b2f0 0x56158a904ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/752246303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a5641b0 0x56158a904970 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 -- 172.21.15.78:0/752246303 >> 172.21.15.78:0/752246303 conn(0x56158a558300 msgr2=0x56158a900300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 -- 172.21.15.78:0/752246303 shutdown_connections 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 -- 172.21.15.78:0/752246303 wait complete. 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 -- start start 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.480+0000 7f2b2bde4740 1 Processor -- start 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a90e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a915b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56158a90c840 con 0x56158a53b2f0 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56158a7f1b60 con 0x56158a5434d0 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56158a90c9c0 con 0x56158a5641b0 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b2bde4740 10 client.0 _ll_register_callbacks cb 0x56158a8162b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a90e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a90e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38258/0 (socket says 172.21.15.78:38258) 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 -- 172.21.15.78:0/3419404653 learned_addr learned my addr 172.21.15.78:0/3419404653 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a915b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 -- 172.21.15.78:0/3419404653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 msgr2=0x56158a915b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a915b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 -- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 msgr2=0x56158a90ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 -- 172.21.15.78:0/3419404653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56158a57c210 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a915b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a90e6e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2b1401e9d0 tx=0x7f2b14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f9f7918ff250e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.481+0000 7f2b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b05ffb640 1 -- 172.21.15.78:0/3419404653 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14009bc0 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b05ffb640 1 -- 172.21.15.78:0/3419404653 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2b14018030 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b18004050 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b05ffb640 1 -- 172.21.15.78:0/3419404653 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2b14005020 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 10 client.17034 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56158a648620 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b05ffb640 1 -- 172.21.15.78:0/3419404653 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b18004050 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b05ffb640 1 -- 172.21.15.78:0/3419404653 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2b14029420 con 0x56158a53b2f0 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 10 client.17034 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 20 client.17034 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a522d80 0x56158a91c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 4 client.17034 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.482+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56158a878070 con 0x56158a522d80 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.483+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a522d80 0x56158a91c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.483+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a522d80 0x56158a91c4a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd757bc8a25116c1 server_cookie=e71ac5e781066a9c in_seq=0 out_seq=0 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.483+0000 7f2b05ffb640 10 client.17034 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b05ffb640 1 -- 172.21.15.78:0/3419404653 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56158a878070 con 0x56158a522d80 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b05ffb640 10 client.17034 handle_command_reply: tid=0 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 client.17034 shutdown 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 msgr2=0x56158a90e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a90e6e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2b1401e9d0 tx=0x7f2b14004730 comp rx=0 tx=0).stop 2024-04-25T14:04:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 shutdown_connections 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a522d80 0x56158a91c4a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a915b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/3419404653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a90e6e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b05ffb640 0 client.17034 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 >> 172.21.15.78:0/3419404653 conn(0x56158a558300 msgr2=0x56158a8ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 shutdown_connections 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 1 -- 172.21.15.78:0/3419404653 wait complete. 2024-04-25T14:04:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.484+0000 7f2b2bde4740 20 client.17034 trim_cache size 0 max 16384 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 Processor -- start 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 -- start start 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a90e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201acaf0 con 0x56158a53b2f0 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201aa3c0 con 0x56158a5434d0 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.499+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201a8210 con 0x56158a5641b0 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38272/0 (socket says 172.21.15.78:38272) 2024-04-25T14:04:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 -- 172.21.15.78:0/1530484758 learned_addr learned my addr 172.21.15.78:0/1530484758 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a90e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 -- 172.21.15.78:0/1530484758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 msgr2=0x56158a90e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a90e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 -- 172.21.15.78:0/1530484758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 msgr2=0x56158a90e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 -- 172.21.15.78:0/1530484758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56158a905b70 con 0x56158a53b2f0 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a90e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9002b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000c00 tx=0x7f2b14009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f674584446d0603d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b07fff640 1 -- 172.21.15.78:0/1530484758 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14043070 con 0x56158a53b2f0 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b07fff640 1 -- 172.21.15.78:0/1530484758 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b1403a050 con 0x56158a53b2f0 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.500+0000 7f2b07fff640 1 -- 172.21.15.78:0/1530484758 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14038030 con 0x56158a53b2f0 2024-04-25T14:04:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 -- 172.21.15.78:0/1530484758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 msgr2=0x56158a9002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9002b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000c00 tx=0x7f2b14009bc0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 -- 172.21.15.78:0/1530484758 shutdown_connections 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5641b0 0x56158a90e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5434d0 0x56158a90e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/1530484758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9002b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 -- 172.21.15.78:0/1530484758 >> 172.21.15.78:0/1530484758 conn(0x56158a558300 msgr2=0x56158a8ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 -- 172.21.15.78:0/1530484758 shutdown_connections 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 -- 172.21.15.78:0/1530484758 wait complete. 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 -- start start 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.501+0000 7f2b2bde4740 1 Processor -- start 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5434d0 0x56158a9155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b2bde4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5641b0 0x56158a984b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201a8880 con 0x56158a53b2f0 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201a9cf0 con 0x56158a5641b0 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b2bde4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201a8f50 con 0x56158a5434d0 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b2bde4740 10 client.0 _ll_register_callbacks cb 0x7f2b14012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38282/0 (socket says 172.21.15.78:38282) 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 -- 172.21.15.78:0/2629569959 learned_addr learned my addr 172.21.15.78:0/2629569959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5434d0 0x56158a9155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1ed9d640 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5641b0 0x56158a984b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 -- 172.21.15.78:0/2629569959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5434d0 msgr2=0x56158a9155e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5434d0 0x56158a9155e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 -- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5641b0 msgr2=0x56158a984b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5641b0 0x56158a984b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 -- 172.21.15.78:0/2629569959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56158a985170 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b1e59c640 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9150a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2b14012e50 tx=0x7f2b140051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc6a3ce8140d50b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.502+0000 7f2b05ffb640 1 -- 172.21.15.78:0/2629569959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14038030 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b05ffb640 1 -- 172.21.15.78:0/2629569959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2b14024040 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1403a050 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b05ffb640 1 -- 172.21.15.78:0/2629569959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2b14043050 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 10 client.17040 fetch_fsmap learned FSMap version 22 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56158a986370 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b05ffb640 1 -- 172.21.15.78:0/2629569959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b1403a050 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b05ffb640 1 -- 172.21.15.78:0/2629569959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2b1403e5d0 con 0x56158a53b2f0 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 10 client.17040 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 10 client.17040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 20 client.17040 populate_metadata read hostname 'smithi078' 2024-04-25T14:04:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a4f81c0 0x56158a988d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:04:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 4 client.17040 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:04:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56158a871ee0 con 0x56158a4f81c0 2024-04-25T14:04:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.503+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a4f81c0 0x56158a988d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:04:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.504+0000 7f2b1dd9b640 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a4f81c0 0x56158a988d00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e87b409e45ce283 server_cookie=2db7b6b1b9985600 in_seq=0 out_seq=0 2024-04-25T14:04:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.504+0000 7f2b05ffb640 10 client.17040 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b05ffb640 1 -- 172.21.15.78:0/2629569959 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56158a871ee0 con 0x56158a4f81c0 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b05ffb640 10 client.17040 handle_command_reply: tid=0 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 client.17040 shutdown 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 msgr2=0x56158a9150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9150a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2b14012e50 tx=0x7f2b140051c0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 shutdown_connections 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56158a5641b0 0x56158a984b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56158a4f81c0 0x56158a988d00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56158a5434d0 0x56158a9155e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 --2- 172.21.15.78:0/2629569959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56158a53b2f0 0x56158a9150a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b05ffb640 0 client.17040 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 >> 172.21.15.78:0/2629569959 conn(0x56158a558300 msgr2=0x56158a90e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.518+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 shutdown_connections 2024-04-25T14:04:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b2bde4740 1 -- 172.21.15.78:0/2629569959 wait complete. 2024-04-25T14:04:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b2bde4740 20 client.17040 trim_cache size 0 max 16384 2024-04-25T14:04:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:04:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:04:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:04:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:04:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2b080afae0 msgr2=0x7f2b080b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2b080afae0 0x7f2b080b1ff0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56158a905d50 tx=0x7f2b14005f90 comp rx=0 tx=0).stop 2024-04-25T14:04:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 msgr2=0x7f2b20199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:04:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20199660 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b100042f0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 shutdown_connections 2024-04-25T14:04:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2b20108960 0x7f2b201a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2b080afae0 0x7f2b080b1ff0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b20103c60 0x7f2b20199660 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 --2- 172.21.15.78:0/2399949491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b20102a20 0x7f2b20199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:04:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 >> 172.21.15.78:0/2399949491 conn(0x7f2b200fdc10 msgr2=0x7f2b20104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:04:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 shutdown_connections 2024-04-25T14:04:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:04:59.519+0000 7f2b24e51640 1 -- 172.21.15.78:0/2399949491 wait complete. 2024-04-25T14:05:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:00 smithi129 ceph-mon[32569]: pgmap v513: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:05:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:00 smithi078 ceph-mon[29163]: pgmap v513: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:05:00.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:00 smithi112 ceph-mon[32617]: pgmap v513: 97 pgs: 97 active+clean; 3.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:05:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:02 smithi129 ceph-mon[32569]: pgmap v514: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:05:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:02 smithi078 ceph-mon[29163]: pgmap v514: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:05:02.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:02 smithi112 ceph-mon[32617]: pgmap v514: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:05:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:04 smithi129 ceph-mon[32569]: pgmap v515: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:04 smithi078 ceph-mon[29163]: pgmap v515: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:04.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:04 smithi112 ceph-mon[32617]: pgmap v515: 97 pgs: 97 active+clean; 3.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:06 smithi129 ceph-mon[32569]: pgmap v516: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:05:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:06 smithi078 ceph-mon[29163]: pgmap v516: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:05:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:06 smithi112 ceph-mon[32617]: pgmap v516: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:05:09.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:09 smithi129 ceph-mon[32569]: pgmap v517: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:09 smithi078 ceph-mon[29163]: pgmap v517: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:09.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:09 smithi112 ceph-mon[32617]: pgmap v517: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:11.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:11.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 -- 172.21.15.78:0/2168213124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80102fc0 msgr2=0x7fcb801033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 --2- 172.21.15.78:0/2168213124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80102fc0 0x7fcb801033d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009900 tx=0x7fcb7002a410 comp rx=0 tx=0).stop 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 -- 172.21.15.78:0/2168213124 shutdown_connections 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 --2- 172.21.15.78:0/2168213124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb80104680 0x7fcb8010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 --2- 172.21.15.78:0/2168213124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80103b20 0x7fcb80103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 --2- 172.21.15.78:0/2168213124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80102fc0 0x7fcb801033d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 -- 172.21.15.78:0/2168213124 >> 172.21.15.78:0/2168213124 conn(0x7fcb800fd340 msgr2=0x7fcb800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 -- 172.21.15.78:0/2168213124 shutdown_connections 2024-04-25T14:05:11.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.497+0000 7fcb84abd640 1 -- 172.21.15.78:0/2168213124 wait complete. 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 Processor -- start 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 -- start start 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102fc0 0x7fcb80198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80103b20 0x7fcb801994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8019e3f0 0x7fcb801a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e700 con 0x7fcb80103b20 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e580 con 0x7fcb80102fc0 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb84abd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010e880 con 0x7fcb8019e3f0 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80103b20 0x7fcb801994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80103b20 0x7fcb801994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43944/0 (socket says 172.21.15.78:43944) 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 -- 172.21.15.78:0/435467899 learned_addr learned my addr 172.21.15.78:0/435467899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102fc0 0x7fcb80198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8019e3f0 0x7fcb801a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 -- 172.21.15.78:0/435467899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8019e3f0 msgr2=0x7fcb801a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8019e3f0 0x7fcb801a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 -- 172.21.15.78:0/435467899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102fc0 msgr2=0x7fcb80198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102fc0 0x7fcb80198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 -- 172.21.15.78:0/435467899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb801a0e80 con 0x7fcb80103b20 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8019e3f0 0x7fcb801a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102fc0 0x7fcb80198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.498+0000 7fcb77fff640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80103b20 0x7fcb801994c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800e9f0 tx=0x7fcb6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.499+0000 7fcb777fe640 1 -- 172.21.15.78:0/435467899 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800cd80 con 0x7fcb80103b20 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.499+0000 7fcb777fe640 1 -- 172.21.15.78:0/435467899 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb680108e0 con 0x7fcb80103b20 2024-04-25T14:05:11.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.499+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb801a1170 con 0x7fcb80103b20 2024-04-25T14:05:11.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.499+0000 7fcb777fe640 1 -- 172.21.15.78:0/435467899 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68018980 con 0x7fcb80103b20 2024-04-25T14:05:11.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.499+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb801a16c0 con 0x7fcb80103b20 2024-04-25T14:05:11.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.501+0000 7fcb8b972740 1 Processor -- start 2024-04-25T14:05:11.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.501+0000 7fcb8b972740 1 -- start start 2024-04-25T14:05:11.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb777fe640 1 -- 172.21.15.78:0/435467899 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcb68010430 con 0x7fcb80103b20 2024-04-25T14:05:11.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f8b390 0x564d8432ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d8432cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb8b972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d84291d80 con 0x564d83f624d0 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb8b972740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564d8424b8b0 con 0x564d83f6a6b0 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb8b972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564d8428dd40 con 0x564d83f8b390 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d8432cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d8432cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43946/0 (socket says 172.21.15.78:43946) 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 -- 172.21.15.78:0/2604419490 learned_addr learned my addr 172.21.15.78:0/2604419490 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f8b390 0x564d8432ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb77fff640 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 -- 172.21.15.78:0/2604419490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f8b390 msgr2=0x564d8432ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f8b390 0x564d8432ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 -- 172.21.15.78:0/2604419490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 msgr2=0x564d8432dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 -- 172.21.15.78:0/2604419490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8421ca40 con 0x564d83f624d0 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb77fff640 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d8432cfe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcb700096c0 tx=0x7fcb70032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a4deaf899ee55b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.502+0000 7fcb777fe640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb74ff9640 1 -- 172.21.15.78:0/2604419490 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7003f070 con 0x564d83f624d0 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb74ff9640 1 -- 172.21.15.78:0/2604419490 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb700317a0 con 0x564d83f624d0 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb74ff9640 1 -- 172.21.15.78:0/2604419490 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70035070 con 0x564d83f624d0 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb777fe640 1 -- 172.21.15.78:0/435467899 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fcb680df000 con 0x7fcb80103b20 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb8b972740 1 -- 172.21.15.78:0/2604419490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 msgr2=0x564d8432cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x564d8432dab0 tx=0x7fcb78008630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.503+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d8432cfe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcb700096c0 tx=0x7fcb70032de0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- 172.21.15.78:0/2604419490 shutdown_connections 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d8432cfe0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2604419490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f8b390 0x564d8432ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- 172.21.15.78:0/2604419490 >> 172.21.15.78:0/2604419490 conn(0x564d83f7f4e0 msgr2=0x564d84328270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- 172.21.15.78:0/2604419490 shutdown_connections 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- 172.21.15.78:0/2604419490 wait complete. 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- start start 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 Processor -- start 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d84336900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f6a6b0 0x564d84336e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f8b390 0x564d8433dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d8432e720 con 0x564d83f624d0 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564d84218d10 con 0x564d83f8b390 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564d8432e8a0 con 0x564d83f6a6b0 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb8b972740 10 client.0 _ll_register_callbacks cb 0x564d84245210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.504+0000 7fcb7e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d84336900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d84336900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43962/0 (socket says 172.21.15.78:43962) 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 -- 172.21.15.78:0/906320372 learned_addr learned my addr 172.21.15.78:0/906320372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f8b390 0x564d8433dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb77fff640 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f6a6b0 0x564d84336e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 -- 172.21.15.78:0/906320372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f6a6b0 msgr2=0x564d84336e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f6a6b0 0x564d84336e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 -- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f8b390 msgr2=0x564d8433dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f8b390 0x564d8433dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 -- 172.21.15.78:0/906320372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d8406d7d0 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d84336900 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006010 tx=0x7fcb70005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5299df009e462c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb65ffb640 1 -- 172.21.15.78:0/906320372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7003f030 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb65ffb640 1 -- 172.21.15.78:0/906320372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fcb70004450 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb700317a0 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb65ffb640 1 -- 172.21.15.78:0/906320372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcb70035040 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb8b972740 10 client.17052 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564d83fa33f0 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.505+0000 7fcb65ffb640 1 -- 172.21.15.78:0/906320372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb700317a0 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb65ffb640 1 -- 172.21.15.78:0/906320372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fcb70044640 con 0x564d83f624d0 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb8b972740 10 client.17052 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:11.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb8b972740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb8b972740 20 client.17052 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb8b972740 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f49f60 0x564d84345f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb8b972740 4 client.17052 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d8429f990 con 0x564d83f49f60 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb77fff640 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f49f60 0x564d84345f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb77fff640 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f49f60 0x564d84345f70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2859e6a9ac44d8 server_cookie=5b9eaeb52f46c58b in_seq=0 out_seq=0 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.506+0000 7fcb65ffb640 10 client.17052 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb65ffb640 1 -- 172.21.15.78:0/906320372 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d8429f990 con 0x564d83f49f60 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb65ffb640 10 client.17052 handle_command_reply: tid=0 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 client.17052 shutdown 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 msgr2=0x564d84336900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d84336900 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006010 tx=0x7fcb70005690 comp rx=0 tx=0).stop 2024-04-25T14:05:11.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 shutdown_connections 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f8b390 0x564d8433dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f49f60 0x564d84345f70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f6a6b0 0x564d84336e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 --2- 172.21.15.78:0/906320372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f624d0 0x564d84336900 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb65ffb640 0 client.17052 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 >> 172.21.15.78:0/906320372 conn(0x564d83f7f4e0 msgr2=0x564d8432b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 shutdown_connections 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.507+0000 7fcb8b972740 1 -- 172.21.15.78:0/906320372 wait complete. 2024-04-25T14:05:11.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.508+0000 7fcb8b972740 20 client.17052 trim_cache size 0 max 16384 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 Processor -- start 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 -- start start 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f7f4e0 0x564d84336be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801ab9a0 con 0x564d83f7f4e0 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a9270 con 0x564d83f6a6b0 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb8b972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a70c0 con 0x564d83f624d0 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb7e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f7f4e0 0x564d84336be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb7e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f7f4e0 0x564d84336be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43968/0 (socket says 172.21.15.78:43968) 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.525+0000 7fcb7e59c640 1 -- 172.21.15.78:0/4072776766 learned_addr learned my addr 172.21.15.78:0/4072776766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb77fff640 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb7e59c640 1 -- 172.21.15.78:0/4072776766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 msgr2=0x564d8433dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb7e59c640 1 -- 172.21.15.78:0/4072776766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 msgr2=0x564d8432b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb7e59c640 1 -- 172.21.15.78:0/4072776766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d83f630f0 con 0x564d83f7f4e0 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f7f4e0 0x564d84336be0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcb70044a30 tx=0x7fcb70000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2778c6b6b699800a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb67fff640 1 -- 172.21.15.78:0/4072776766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7003f070 con 0x564d83f7f4e0 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb67fff640 1 -- 172.21.15.78:0/4072776766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7004d070 con 0x564d83f7f4e0 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.526+0000 7fcb67fff640 1 -- 172.21.15.78:0/4072776766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70003d70 con 0x564d83f7f4e0 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 -- 172.21.15.78:0/4072776766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f7f4e0 msgr2=0x564d84336be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f7f4e0 0x564d84336be0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcb70044a30 tx=0x7fcb70000bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 -- 172.21.15.78:0/4072776766 shutdown_connections 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f6a6b0 0x564d8432b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 --2- 172.21.15.78:0/4072776766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f7f4e0 0x564d84336be0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 -- 172.21.15.78:0/4072776766 >> 172.21.15.78:0/4072776766 conn(0x564d843234d0 msgr2=0x564d843364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 -- 172.21.15.78:0/4072776766 shutdown_connections 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 -- 172.21.15.78:0/4072776766 wait complete. 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 -- start start 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 Processor -- start 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.527+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f6a6b0 0x564d8433b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f7f4e0 0x564d8433c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a7730 con 0x564d83f6a6b0 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a8ba0 con 0x564d83f7f4e0 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801a7e00 con 0x564d83f624d0 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 10 client.0 _ll_register_callbacks cb 0x7fcb780084d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f6a6b0 0x564d8433b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f6a6b0 0x564d8433b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43978/0 (socket says 172.21.15.78:43978) 2024-04-25T14:05:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 -- 172.21.15.78:0/2652270055 learned_addr learned my addr 172.21.15.78:0/2652270055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f7f4e0 0x564d8433c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 -- 172.21.15.78:0/2652270055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 msgr2=0x564d8433b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 -- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f7f4e0 msgr2=0x564d8433c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f7f4e0 0x564d8433c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 -- 172.21.15.78:0/2652270055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d842524b0 con 0x564d83f6a6b0 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb7ed9d640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f7f4e0 0x564d8433c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb77fff640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f6a6b0 0x564d8433b690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcb680a7500 tx=0x7fcb68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa22ceeffdf75d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb65ffb640 1 -- 172.21.15.78:0/2652270055 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb680a7570 con 0x564d83f6a6b0 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb65ffb640 1 -- 172.21.15.78:0/2652270055 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fcb68018b20 con 0x564d83f6a6b0 2024-04-25T14:05:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb7004d070 con 0x564d83f6a6b0 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb65ffb640 1 -- 172.21.15.78:0/2652270055 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcb680ab920 con 0x564d83f6a6b0 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 10 client.17058 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.528+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564d8433dcc0 con 0x564d83f6a6b0 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb65ffb640 1 -- 172.21.15.78:0/2652270055 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7004d070 con 0x564d83f6a6b0 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb65ffb640 1 -- 172.21.15.78:0/2652270055 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fcb680e11d0 con 0x564d83f6a6b0 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb8b972740 10 client.17058 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb8b972740 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb8b972740 20 client.17058 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f8b390 0x564d843b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb8b972740 4 client.17058 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d84299800 con 0x564d83f8b390 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f8b390 0x564d843b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb7e59c640 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f8b390 0x564d843b0570 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81617ffe4ab049ad server_cookie=14c5f2bf4c7080e in_seq=0 out_seq=0 2024-04-25T14:05:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.529+0000 7fcb65ffb640 10 client.17058 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb65ffb640 1 -- 172.21.15.78:0/2652270055 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d84299800 con 0x564d83f8b390 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb65ffb640 10 client.17058 handle_command_reply: tid=0 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 client.17058 shutdown 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f6a6b0 msgr2=0x564d8433b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f6a6b0 0x564d8433b690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcb680a7500 tx=0x7fcb68000bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 shutdown_connections 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564d83f7f4e0 0x564d8433c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d83f6a6b0 0x564d8433b690 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564d83f8b390 0x564d843b0570 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 --2- 172.21.15.78:0/2652270055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d83f624d0 0x564d8433b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb65ffb640 0 client.17058 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 >> 172.21.15.78:0/2652270055 conn(0x564d843234d0 msgr2=0x564d8432b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 shutdown_connections 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 1 -- 172.21.15.78:0/2652270055 wait complete. 2024-04-25T14:05:11.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb8b972740 20 client.17058 trim_cache size 0 max 16384 2024-04-25T14:05:11.537 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:11.537 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:05:11.537 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:05:11.537 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb6c0afaf0 msgr2=0x7fcb6c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x564d8432dab0 tx=0x7fcb78008630 comp rx=0 tx=0).stop 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80103b20 msgr2=0x7fcb801994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80103b20 0x7fcb801994c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800e9f0 tx=0x7fcb6800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 shutdown_connections 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8019e3f0 0x7fcb801a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb6c0afaf0 0x7fcb6c0b2000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb80103b20 0x7fcb801994c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 --2- 172.21.15.78:0/435467899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb80102fc0 0x7fcb80198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.530+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 >> 172.21.15.78:0/435467899 conn(0x7fcb800fd340 msgr2=0x7fcb800ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.531+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 shutdown_connections 2024-04-25T14:05:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.531+0000 7fcb84abd640 1 -- 172.21.15.78:0/435467899 wait complete. 2024-04-25T14:05:11.541 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:05:11.541 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 -- 172.21.15.78:0/1020673968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 msgr2=0x7fade41036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 --2- 172.21.15.78:0/1020673968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade41036c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009ca0 tx=0x7fadd402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 -- 172.21.15.78:0/1020673968 shutdown_connections 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 --2- 172.21.15.78:0/1020673968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade4108900 0x7fade410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 --2- 172.21.15.78:0/1020673968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4103c00 0x7fade4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 --2- 172.21.15.78:0/1020673968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade41036c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 -- 172.21.15.78:0/1020673968 >> 172.21.15.78:0/1020673968 conn(0x7fade40fdbd0 msgr2=0x7fade4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 -- 172.21.15.78:0/1020673968 shutdown_connections 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 -- 172.21.15.78:0/1020673968 wait complete. 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.650+0000 7fadea100640 1 Processor -- start 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fadea100640 1 -- start start 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fadea100640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fadea100640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade4103c00 0x7fade419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fadea100640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4108900 0x7fade41a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fadea100640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fade410f5f0 con 0x7fade41029c0 2024-04-25T14:05:11.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fadea100640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade410f470 con 0x7fade4108900 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fadea100640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fade410f770 con 0x7fade4103c00 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43990/0 (socket says 172.21.15.78:43990) 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 -- 172.21.15.78:0/1656000856 learned_addr learned my addr 172.21.15.78:0/1656000856 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade2ffd640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade4103c00 0x7fade419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 -- 172.21.15.78:0/1656000856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade4103c00 msgr2=0x7fade419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade4103c00 0x7fade419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 -- 172.21.15.78:0/1656000856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4108900 msgr2=0x7fade41a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4108900 0x7fade41a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 -- 172.21.15.78:0/1656000856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade41a1d70 con 0x7fade41029c0 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade2ffd640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade4103c00 0x7fade419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.651+0000 7fade37fe640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade419a230 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fadd40098d0 tx=0x7fadd4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.652+0000 7fade0ff9640 1 -- 172.21.15.78:0/1656000856 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4003cb0 con 0x7fade41029c0 2024-04-25T14:05:11.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.652+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade41a2000 con 0x7fade41029c0 2024-04-25T14:05:11.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.652+0000 7fade0ff9640 1 -- 172.21.15.78:0/1656000856 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4003e50 con 0x7fade41029c0 2024-04-25T14:05:11.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.652+0000 7fade0ff9640 1 -- 172.21.15.78:0/1656000856 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd402fc60 con 0x7fade41029c0 2024-04-25T14:05:11.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.652+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a25a0 con 0x7fade41029c0 2024-04-25T14:05:11.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.653+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade4107710 con 0x7fade41029c0 2024-04-25T14:05:11.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.654+0000 7fade0ff9640 1 -- 172.21.15.78:0/1656000856 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fadd4004710 con 0x7fade41029c0 2024-04-25T14:05:11.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.655+0000 7fade0ff9640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadd00afaf0 0x7fadd00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.655+0000 7fade2ffd640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadd00afaf0 0x7fadd00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.655+0000 7fade0ff9640 1 -- 172.21.15.78:0/1656000856 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fadd4100510 con 0x7fade41029c0 2024-04-25T14:05:11.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.655+0000 7fade2ffd640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadd00afaf0 0x7fadd00b2000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fadcc005d00 tx=0x7fadcc005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:11.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.659+0000 7fade0ff9640 1 -- 172.21.15.78:0/1656000856 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadd40c9170 con 0x7fade41029c0 2024-04-25T14:05:11.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:11 smithi129 ceph-mon[32569]: pgmap v518: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:11 smithi078 ceph-mon[29163]: pgmap v518: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:11.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:11 smithi112 ceph-mon[32617]: pgmap v518: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:11.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.871+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fade402db50 con 0x7fade41029c0 2024-04-25T14:05:11.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.872+0000 7fade0ff9640 1 -- 172.21.15.78:0/1656000856 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fadd40c8a70 con 0x7fade41029c0 2024-04-25T14:05:11.873 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:05:11.873 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:05:11.874 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadd00afaf0 msgr2=0x7fadd00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadd00afaf0 0x7fadd00b2000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fadcc005d00 tx=0x7fadcc005c90 comp rx=0 tx=0).stop 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 msgr2=0x7fade419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade419a230 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fadd40098d0 tx=0x7fadd4031840 comp rx=0 tx=0).stop 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 shutdown_connections 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade4108900 0x7fade41a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade4103c00 0x7fade419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadd00afaf0 0x7fadd00b2000 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 --2- 172.21.15.78:0/1656000856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade41029c0 0x7fade419a230 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 >> 172.21.15.78:0/1656000856 conn(0x7fade40fdbd0 msgr2=0x7fade41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 shutdown_connections 2024-04-25T14:05:11.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.874+0000 7fadea100640 1 -- 172.21.15.78:0/1656000856 wait complete. 2024-04-25T14:05:11.885 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 -- 172.21.15.78:0/3037758442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90104d30 msgr2=0x7fab90105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 --2- 172.21.15.78:0/3037758442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90104d30 0x7fab90105140 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fab84009900 tx=0x7fab8402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 -- 172.21.15.78:0/3037758442 shutdown_connections 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 --2- 172.21.15.78:0/3037758442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab90109a30 0x7fab9010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 --2- 172.21.15.78:0/3037758442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90104d30 0x7fab90105140 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 --2- 172.21.15.78:0/3037758442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab90103af0 0x7fab901047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 -- 172.21.15.78:0/3037758442 >> 172.21.15.78:0/3037758442 conn(0x7fab900fece0 msgr2=0x7fab90101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 -- 172.21.15.78:0/3037758442 shutdown_connections 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.995+0000 7fab95700640 1 -- 172.21.15.78:0/3037758442 wait complete. 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 Processor -- start 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 -- start start 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90103af0 0x7fab90077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab90104d30 0x7fab90076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab90109a30 0x7fab90076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab9010f5f0 con 0x7fab90103af0 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab9010f470 con 0x7fab90104d30 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab95700640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab9010f770 con 0x7fab90109a30 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab8effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90103af0 0x7fab90077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab8effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90103af0 0x7fab90077960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44014/0 (socket says 172.21.15.78:44014) 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.996+0000 7fab8effd640 1 -- 172.21.15.78:0/634948366 learned_addr learned my addr 172.21.15.78:0/634948366 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab90104d30 0x7fab90076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab90109a30 0x7fab90076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8effd640 1 -- 172.21.15.78:0/634948366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab90109a30 msgr2=0x7fab90076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8effd640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab90109a30 0x7fab90076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8effd640 1 -- 172.21.15.78:0/634948366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab90104d30 msgr2=0x7fab90076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8effd640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab90104d30 0x7fab90076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8effd640 1 -- 172.21.15.78:0/634948366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab90076be0 con 0x7fab90103af0 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab90109a30 0x7fab90076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab90104d30 0x7fab90076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab8effd640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90103af0 0x7fab90077960 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c970 tx=0x7fab8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab77fff640 1 -- 172.21.15.78:0/634948366 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80007c70 con 0x7fab90103af0 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab901a41f0 con 0x7fab90103af0 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab77fff640 1 -- 172.21.15.78:0/634948366 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab80007e10 con 0x7fab90103af0 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab77fff640 1 -- 172.21.15.78:0/634948366 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80018760 con 0x7fab90103af0 2024-04-25T14:05:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.997+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab901a4650 con 0x7fab90103af0 2024-04-25T14:05:12.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:11.999+0000 7fab9c5b5740 1 Processor -- start 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab9c5b5740 1 -- start start 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcadc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcadcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcadda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab77fff640 1 -- 172.21.15.78:0/634948366 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fab8000f450 con 0x7fab90103af0 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559dfca40d80 con 0x559dfc7196b0 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559dfc9faa20 con 0x559dfc73a390 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559dfca3cd40 con 0x559dfc7114d0 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcadda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcadda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44016/0 (socket says 172.21.15.78:44016) 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3798290606 learned_addr learned my addr 172.21.15.78:0/3798290606 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8effd640 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcadcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcadc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3798290606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 msgr2=0x559dfcadcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcadcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3798290606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 msgr2=0x559dfcadc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcadc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3798290606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfca05c30 con 0x559dfc7196b0 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcadc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8effd640 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcadcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcadda20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fab840096c0 tx=0x7fab84031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7a79f773d4aab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.000+0000 7fab77fff640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab780afaf0 0x7fab780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab757fa640 1 -- 172.21.15.78:0/3798290606 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84035030 con 0x559dfc7196b0 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab780afaf0 0x7fab780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab757fa640 1 -- 172.21.15.78:0/3798290606 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84031860 con 0x559dfc7196b0 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab77fff640 1 -- 172.21.15.78:0/634948366 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fab800ded70 con 0x7fab90103af0 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab757fa640 1 -- 172.21.15.78:0/3798290606 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84003d90 con 0x559dfc7196b0 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3798290606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 msgr2=0x559dfcadda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcadda20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fab840096c0 tx=0x7fab84031e80 comp rx=0 tx=0).stop 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.001+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab780afaf0 0x7fab780b2000 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x559dfcadd860 tx=0x7fab88007490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3798290606 shutdown_connections 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcadda20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcadcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3798290606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcadc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3798290606 >> 172.21.15.78:0/3798290606 conn(0x559dfc72e4e0 msgr2=0x559dfcad8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3798290606 shutdown_connections 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3798290606 wait complete. 2024-04-25T14:05:12.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- start start 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 Processor -- start 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcae6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7196b0 0x559dfcae6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc73a390 0x559dfcaeda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559dfcade4d0 con 0x559dfc73a390 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559dfc9c7dc0 con 0x559dfc7196b0 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559dfcade650 con 0x559dfc7114d0 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab9c5b5740 10 client.0 _ll_register_callbacks cb 0x559dfc9e3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab8f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc73a390 0x559dfcaeda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab8f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc73a390 0x559dfcaeda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44024/0 (socket says 172.21.15.78:44024) 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.002+0000 7fab8f7fe640 1 -- 172.21.15.78:0/4016185360 learned_addr learned my addr 172.21.15.78:0/4016185360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8effd640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcae6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7196b0 0x559dfcae6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8f7fe640 1 -- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 msgr2=0x559dfcae6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcae6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8f7fe640 1 -- 172.21.15.78:0/4016185360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7196b0 msgr2=0x559dfcae6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7196b0 0x559dfcae6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8f7fe640 1 -- 172.21.15.78:0/4016185360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfc7523f0 con 0x559dfc73a390 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8effd640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcae6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc73a390 0x559dfcaeda30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fab880119c0 tx=0x7fab88004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed681ef4e7f4177a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab72ff5640 1 -- 172.21.15.78:0/4016185360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88022070 con 0x559dfc73a390 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab72ff5640 1 -- 172.21.15.78:0/4016185360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fab88010d60 con 0x559dfc73a390 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab84031860 con 0x559dfc73a390 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab72ff5640 1 -- 172.21.15.78:0/4016185360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fab8802e050 con 0x559dfc73a390 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab9c5b5740 10 client.17076 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559dfc81c7d0 con 0x559dfc73a390 2024-04-25T14:05:12.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.003+0000 7fab72ff5640 1 -- 172.21.15.78:0/4016185360 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84031860 con 0x559dfc73a390 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab72ff5640 1 -- 172.21.15.78:0/4016185360 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fab880263d0 con 0x559dfc73a390 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab9c5b5740 10 client.17076 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab9c5b5740 10 client.17076 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab9c5b5740 20 client.17076 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6f8f60 0x559dfcaf4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab9c5b5740 4 client.17076 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dfca4e990 con 0x559dfc6f8f60 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.004+0000 7fab8effd640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6f8f60 0x559dfcaf4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab72ff5640 10 client.17076 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab8effd640 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6f8f60 0x559dfcaf4210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=844dab7f7d38249c server_cookie=367fbd341ebf083b in_seq=0 out_seq=0 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab72ff5640 1 -- 172.21.15.78:0/4016185360 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559dfca4e990 con 0x559dfc6f8f60 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab72ff5640 10 client.17076 handle_command_reply: tid=0 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab9c5b5740 1 client.17076 shutdown 2024-04-25T14:05:12.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc73a390 msgr2=0x559dfcaeda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc73a390 0x559dfcaeda30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fab880119c0 tx=0x7fab88004ab0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.005+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 shutdown_connections 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6f8f60 0x559dfcaf4210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc73a390 0x559dfcaeda30 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7196b0 0x559dfcae6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/4016185360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcae6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab72ff5640 0 client.17076 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 >> 172.21.15.78:0/4016185360 conn(0x559dfc72e4e0 msgr2=0x559dfcadaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 shutdown_connections 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 1 -- 172.21.15.78:0/4016185360 wait complete. 2024-04-25T14:05:12.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.006+0000 7fab9c5b5740 20 client.17076 trim_cache size 0 max 16384 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 Processor -- start 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 -- start start 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7114d0 0x559dfcae1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcae1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc73a390 0x559dfcad79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab901afbd0 con 0x559dfc7196b0 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab901ad4a0 con 0x559dfc7114d0 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab901ab2f0 con 0x559dfc73a390 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcae1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcae1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44032/0 (socket says 172.21.15.78:44032) 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3842259418 learned_addr learned my addr 172.21.15.78:0/3842259418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc73a390 0x559dfcad79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8effd640 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7114d0 0x559dfcae1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3842259418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc73a390 msgr2=0x559dfcad79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc73a390 0x559dfcad79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.019+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3842259418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7114d0 msgr2=0x559dfcae1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.020+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7114d0 0x559dfcae1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.020+0000 7fab8e7fc640 1 -- 172.21.15.78:0/3842259418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfcaf1ee0 con 0x559dfc7196b0 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.020+0000 7fab8effd640 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7114d0 0x559dfcae1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.020+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcae1850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fab84009900 tx=0x7fab8402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d618d4e89bfbe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.020+0000 7fab74ff9640 1 -- 172.21.15.78:0/3842259418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8404d070 con 0x559dfc7196b0 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.020+0000 7fab74ff9640 1 -- 172.21.15.78:0/3842259418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84004240 con 0x559dfc7196b0 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.020+0000 7fab74ff9640 1 -- 172.21.15.78:0/3842259418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84035040 con 0x559dfc7196b0 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3842259418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 msgr2=0x559dfcae1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcae1850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fab84009900 tx=0x7fab8402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3842259418 shutdown_connections 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc73a390 0x559dfcad79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcae1850 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/3842259418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc7114d0 0x559dfcae1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3842259418 >> 172.21.15.78:0/3842259418 conn(0x559dfc72e4e0 msgr2=0x559dfcadc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3842259418 shutdown_connections 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- 172.21.15.78:0/3842259418 wait complete. 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- start start 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 Processor -- start 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcaea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcaeae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcaebd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab901ab960 con 0x559dfc7196b0 2024-04-25T14:05:12.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab901acdd0 con 0x559dfc73a390 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab901ac030 con 0x559dfc7114d0 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab9c5b5740 10 client.0 _ll_register_callbacks cb 0x7fab88015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab8e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcaeae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab8e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcaeae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44034/0 (socket says 172.21.15.78:44034) 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab8e7fc640 1 -- 172.21.15.78:0/345124287 learned_addr learned my addr 172.21.15.78:0/345124287 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab8effd640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcaea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.021+0000 7fab8f7fe640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcaebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab8e7fc640 1 -- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 msgr2=0x559dfcaea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcaea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab8e7fc640 1 -- 172.21.15.78:0/345124287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 msgr2=0x559dfcaebd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcaebd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab8e7fc640 1 -- 172.21.15.78:0/345124287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfcaec300 con 0x559dfc7196b0 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab8effd640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcaea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab8e7fc640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcaeae10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fab8404b040 tx=0x7fab84031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9525e44b81060d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab72ff5640 1 -- 172.21.15.78:0/345124287 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84056060 con 0x559dfc7196b0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab72ff5640 1 -- 172.21.15.78:0/345124287 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fab840035f0 con 0x559dfc7196b0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab84004240 con 0x559dfc7196b0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab72ff5640 1 -- 172.21.15.78:0/345124287 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fab8404d040 con 0x559dfc7196b0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 10 client.17082 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559dfcb5c580 con 0x559dfc7196b0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab72ff5640 1 -- 172.21.15.78:0/345124287 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab84004240 con 0x559dfc7196b0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab72ff5640 1 -- 172.21.15.78:0/345124287 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fab84050070 con 0x559dfc7196b0 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 10 client.17082 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 10 client.17082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 20 client.17082 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6ce3a0 0x559dfcb5ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 4 client.17082 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.022+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559dfca48800 con 0x559dfc6ce3a0 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab8effd640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6ce3a0 0x559dfcb5ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab72ff5640 10 client.17082 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab8effd640 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6ce3a0 0x559dfcb5ea20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33af80603f6d35b6 server_cookie=f78000ce0b6fc47 in_seq=0 out_seq=0 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab72ff5640 1 -- 172.21.15.78:0/345124287 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559dfca48800 con 0x559dfc6ce3a0 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab72ff5640 10 client.17082 handle_command_reply: tid=0 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab9c5b5740 1 client.17082 shutdown 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 msgr2=0x559dfcaeae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcaeae10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fab8404b040 tx=0x7fab84031740 comp rx=0 tx=0).stop 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 shutdown_connections 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559dfc6ce3a0 0x559dfcb5ea20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559dfc73a390 0x559dfcaebd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.023+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559dfc7196b0 0x559dfcaeae10 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab9c5b5740 1 --2- 172.21.15.78:0/345124287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559dfc7114d0 0x559dfcaea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab72ff5640 0 client.17082 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 >> 172.21.15.78:0/345124287 conn(0x559dfc72e4e0 msgr2=0x559dfcadc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 shutdown_connections 2024-04-25T14:05:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab9c5b5740 1 -- 172.21.15.78:0/345124287 wait complete. 2024-04-25T14:05:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab9c5b5740 20 client.17082 trim_cache size 0 max 16384 2024-04-25T14:05:12.031 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab780afaf0 msgr2=0x7fab780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab780afaf0 0x7fab780b2000 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x559dfcadd860 tx=0x7fab88007490 comp rx=0 tx=0).stop 2024-04-25T14:05:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90103af0 msgr2=0x7fab90077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90103af0 0x7fab90077960 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c970 tx=0x7fab8000ce30 comp rx=0 tx=0).stop 2024-04-25T14:05:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 shutdown_connections 2024-04-25T14:05:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab90109a30 0x7fab90076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab90104d30 0x7fab90076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fab780afaf0 0x7fab780b2000 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 --2- 172.21.15.78:0/634948366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab90103af0 0x7fab90077960 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 >> 172.21.15.78:0/634948366 conn(0x7fab900fece0 msgr2=0x7fab90105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 shutdown_connections 2024-04-25T14:05:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.024+0000 7fab95700640 1 -- 172.21.15.78:0/634948366 wait complete. 2024-04-25T14:05:12.035 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:05:12.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 -- 172.21.15.78:0/383776222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441080d0 msgr2=0x7f1e4410a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 --2- 172.21.15.78:0/383776222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441080d0 0x7f1e4410a4f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009ca0 tx=0x7f1e3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 -- 172.21.15.78:0/383776222 shutdown_connections 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 --2- 172.21.15.78:0/383776222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441080d0 0x7f1e4410a4f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 --2- 172.21.15.78:0/383776222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e441033d0 0x7f1e441037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 --2- 172.21.15.78:0/383776222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e44102190 0x7f1e44102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 -- 172.21.15.78:0/383776222 >> 172.21.15.78:0/383776222 conn(0x7f1e440fd380 msgr2=0x7f1e440ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 -- 172.21.15.78:0/383776222 shutdown_connections 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.144+0000 7f1e4c172640 1 -- 172.21.15.78:0/383776222 wait complete. 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 Processor -- start 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 -- start start 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e44102190 0x7f1e441990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441033d0 0x7f1e44199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e441080d0 0x7f1e441a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4410dcb0 con 0x7f1e441033d0 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4410db30 con 0x7f1e441080d0 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4c172640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4410de30 con 0x7f1e44102190 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441033d0 0x7f1e44199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441033d0 0x7f1e44199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44066/0 (socket says 172.21.15.78:44066) 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e4970d640 1 -- 172.21.15.78:0/2307803400 learned_addr learned my addr 172.21.15.78:0/2307803400 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.145+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e44102190 0x7f1e441990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e441080d0 0x7f1e441a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4970d640 1 -- 172.21.15.78:0/2307803400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e44102190 msgr2=0x7f1e441990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4970d640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e44102190 0x7f1e441990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4970d640 1 -- 172.21.15.78:0/2307803400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e441080d0 msgr2=0x7f1e441a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4970d640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e441080d0 0x7f1e441a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4970d640 1 -- 172.21.15.78:0/2307803400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e441a0c20 con 0x7f1e441033d0 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e441080d0 0x7f1e441a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4970d640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441033d0 0x7f1e44199620 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009970 tx=0x7f1e34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e3affd640 1 -- 172.21.15.78:0/2307803400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400d020 con 0x7f1e441033d0 2024-04-25T14:05:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e44109550 con 0x7f1e441033d0 2024-04-25T14:05:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e3affd640 1 -- 172.21.15.78:0/2307803400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3400d1c0 con 0x7f1e441033d0 2024-04-25T14:05:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e3affd640 1 -- 172.21.15.78:0/2307803400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400cc50 con 0x7f1e441033d0 2024-04-25T14:05:12.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.146+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e44109af0 con 0x7f1e441033d0 2024-04-25T14:05:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e3affd640 1 -- 172.21.15.78:0/2307803400 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1e34007860 con 0x7f1e441033d0 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 Processor -- start 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 -- start start 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40e1390 0x563cf4483810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf4483d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4484a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563cf43e7d80 con 0x563cf40b84d0 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cf43a1a20 con 0x563cf40e1390 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e5301d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563cf43e3d40 con 0x563cf40c06b0 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e49f0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf4483d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e49f0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf4483d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44074/0 (socket says 172.21.15.78:44074) 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e49f0e640 1 -- 172.21.15.78:0/1791487435 learned_addr learned my addr 172.21.15.78:0/1791487435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.149+0000 7f1e4970d640 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4484a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40e1390 0x563cf4483810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e3affd640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e300afaf0 0x7f1e300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e300afaf0 0x7f1e300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e49f0e640 1 -- 172.21.15.78:0/1791487435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 msgr2=0x563cf4484a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4484a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e49f0e640 1 -- 172.21.15.78:0/1791487435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40e1390 msgr2=0x563cf4483810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40e1390 0x563cf4483810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e49f0e640 1 -- 172.21.15.78:0/1791487435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf43acc30 con 0x563cf40b84d0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e3affd640 1 -- 172.21.15.78:0/2307803400 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1e340e0090 con 0x7f1e441033d0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf4483d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000bc70 tx=0x7f1e400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871ed44bd5a779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e2ffff640 1 -- 172.21.15.78:0/1791487435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40004220 con 0x563cf40b84d0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e2ffff640 1 -- 172.21.15.78:0/1791487435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4000be20 con 0x563cf40b84d0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.150+0000 7f1e2ffff640 1 -- 172.21.15.78:0/1791487435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40004c90 con 0x563cf40b84d0 2024-04-25T14:05:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 -- 172.21.15.78:0/1791487435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 msgr2=0x563cf4483d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e300afaf0 0x7f1e300b2000 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005e30 tx=0x7f1e3c02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf4483d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000bc70 tx=0x7f1e400107a0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 -- 172.21.15.78:0/1791487435 shutdown_connections 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4484a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf4483d90 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1791487435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40e1390 0x563cf4483810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 -- 172.21.15.78:0/1791487435 >> 172.21.15.78:0/1791487435 conn(0x563cf40d54e0 msgr2=0x563cf447f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 -- 172.21.15.78:0/1791487435 shutdown_connections 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 -- 172.21.15.78:0/1791487435 wait complete. 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.151+0000 7f1e5301d740 1 -- start start 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 1 Processor -- start 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf448d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf448dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40e1390 0x563cf4494a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563cf4485530 con 0x563cf40c06b0 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cf436edc0 con 0x563cf40b84d0 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563cf44856b0 con 0x563cf40e1390 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e5301d740 10 client.0 _ll_register_callbacks cb 0x563cf438a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf448dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf448dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44080/0 (socket says 172.21.15.78:44080) 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 -- 172.21.15.78:0/568466156 learned_addr learned my addr 172.21.15.78:0/568466156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf448d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40e1390 0x563cf4494a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 -- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40e1390 msgr2=0x563cf4494a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40e1390 0x563cf4494a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 -- 172.21.15.78:0/568466156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 msgr2=0x563cf448d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf448d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e4970d640 1 -- 172.21.15.78:0/568466156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf40f93f0 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.152+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf448d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e4970d640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf448dad0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1e340058c0 tx=0x7f1e34007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbcf3a3354b75de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/568466156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e340a7e50 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/568466156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1e340b0ed0 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4000be20 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/568466156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1e34007b50 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 10 client.17094 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563cf41c37d0 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/568466156 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4000be20 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/568466156 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1e340a3e70 con 0x563cf40c06b0 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 10 client.17094 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 10 client.17094 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 20 client.17094 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf409ff60 0x563cf449b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 4 client.17094 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.153+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cf43f5990 con 0x563cf409ff60 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.154+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf409ff60 0x563cf449b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.154+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf409ff60 0x563cf449b320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f58d811ce3fcf765 server_cookie=c8b192ede187a126 in_seq=0 out_seq=0 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.154+0000 7f1e2d7fa640 10 client.17094 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/568466156 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563cf43f5990 con 0x563cf409ff60 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e2d7fa640 10 client.17094 handle_command_reply: tid=0 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 client.17094 shutdown 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 msgr2=0x563cf448dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf448dad0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1e340058c0 tx=0x7f1e34007f90 comp rx=0 tx=0).stop 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 shutdown_connections 2024-04-25T14:05:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf409ff60 0x563cf449b320 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40e1390 0x563cf4494a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e2d7fa640 0 client.17094 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf448dad0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 --2- 172.21.15.78:0/568466156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf448d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 >> 172.21.15.78:0/568466156 conn(0x563cf40d54e0 msgr2=0x563cf4481f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 shutdown_connections 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 1 -- 172.21.15.78:0/568466156 wait complete. 2024-04-25T14:05:12.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.155+0000 7f1e5301d740 20 client.17094 trim_cache size 0 max 16384 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 Processor -- start 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 -- start start 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf409ff60 0x563cf4482530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf4494950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf44a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441ab9b0 con 0x563cf40c06b0 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a9280 con 0x563cf40b84d0 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e5301d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a70d0 con 0x563cf409ff60 2024-04-25T14:05:12.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e4a70f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf44a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e4a70f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf44a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44096/0 (socket says 172.21.15.78:44096) 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e4a70f640 1 -- 172.21.15.78:0/918521900 learned_addr learned my addr 172.21.15.78:0/918521900 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e4970d640 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf4494950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.171+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf409ff60 0x563cf4482530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e4a70f640 1 -- 172.21.15.78:0/918521900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf409ff60 msgr2=0x563cf4482530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf409ff60 0x563cf4482530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e4a70f640 1 -- 172.21.15.78:0/918521900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 msgr2=0x563cf4494950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf4494950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e4a70f640 1 -- 172.21.15.78:0/918521900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf4495e00 con 0x563cf40c06b0 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf44a0ae0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0317f0 tx=0x7f1e3c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc26627c47819fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e2f7fe640 1 -- 172.21.15.78:0/918521900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c004240 con 0x563cf40c06b0 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e2f7fe640 1 -- 172.21.15.78:0/918521900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c043040 con 0x563cf40c06b0 2024-04-25T14:05:12.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.172+0000 7f1e2f7fe640 1 -- 172.21.15.78:0/918521900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0305c0 con 0x563cf40c06b0 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 -- 172.21.15.78:0/918521900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 msgr2=0x563cf44a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf44a0ae0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0317f0 tx=0x7f1e3c005c50 comp rx=0 tx=0).stop 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 -- 172.21.15.78:0/918521900 shutdown_connections 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40c06b0 0x563cf44a0ae0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf40b84d0 0x563cf4494950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 --2- 172.21.15.78:0/918521900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf409ff60 0x563cf4482530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 -- 172.21.15.78:0/918521900 >> 172.21.15.78:0/918521900 conn(0x563cf40d54e0 msgr2=0x563cf4481e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 -- 172.21.15.78:0/918521900 shutdown_connections 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 -- 172.21.15.78:0/918521900 wait complete. 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 -- start start 2024-04-25T14:05:12.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.173+0000 7f1e5301d740 1 Processor -- start 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf409ff60 0x563cf4493f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf44944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e5301d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4501b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e5301d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a7740 con 0x563cf40b84d0 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e5301d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a8bb0 con 0x563cf409ff60 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e5301d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a7e10 con 0x563cf40c06b0 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e5301d740 10 client.0 _ll_register_callbacks cb 0x7f1e3400d370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf44944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf44944d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44106/0 (socket says 172.21.15.78:44106) 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 -- 172.21.15.78:0/1415325207 learned_addr learned my addr 172.21.15.78:0/1415325207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf409ff60 0x563cf4493f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4a70f640 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4501b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 -- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 msgr2=0x563cf4501b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4501b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 -- 172.21.15.78:0/1415325207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf409ff60 msgr2=0x563cf4493f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf409ff60 0x563cf4493f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 -- 172.21.15.78:0/1415325207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf4502160 con 0x563cf40b84d0 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e4970d640 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf44944d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1e340a8200 tx=0x7f1e34005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b28691e2780172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.174+0000 7f1e2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/1415325207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e340a7c70 con 0x563cf40b84d0 2024-04-25T14:05:12.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/1415325207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1e340a71d0 con 0x563cf40b84d0 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c043040 con 0x563cf40b84d0 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/1415325207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1e340a7e50 con 0x563cf40b84d0 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 10 client.17100 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563cf4503410 con 0x563cf40b84d0 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/1415325207 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e340c1030 con 0x563cf40b84d0 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/1415325207 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1e340a73d0 con 0x563cf40b84d0 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 10 client.17100 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 10 client.17100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 20 client.17100 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf40e1390 0x563cf4505e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 4 client.17100 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cf43ef800 con 0x563cf40e1390 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.175+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf40e1390 0x563cf4505e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e2d7fa640 10 client.17100 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e49f0e640 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf40e1390 0x563cf4505e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ab294fa59dd0603 server_cookie=309aff13dd55acf0 in_seq=0 out_seq=0 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e2d7fa640 1 -- 172.21.15.78:0/1415325207 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cf43ef800 con 0x563cf40e1390 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e2d7fa640 10 client.17100 handle_command_reply: tid=0 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 client.17100 shutdown 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 msgr2=0x563cf44944d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf44944d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1e340a8200 tx=0x7f1e34005bf0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 shutdown_connections 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563cf40e1390 0x563cf4505e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cf40c06b0 0x563cf4501b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cf40b84d0 0x563cf44944d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 --2- 172.21.15.78:0/1415325207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cf409ff60 0x563cf4493f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e2d7fa640 0 client.17100 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 >> 172.21.15.78:0/1415325207 conn(0x563cf40d54e0 msgr2=0x563cf448bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 shutdown_connections 2024-04-25T14:05:12.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 1 -- 172.21.15.78:0/1415325207 wait complete. 2024-04-25T14:05:12.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.176+0000 7f1e5301d740 20 client.17100 trim_cache size 0 max 16384 2024-04-25T14:05:12.183 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e300afaf0 msgr2=0x7f1e300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e300afaf0 0x7f1e300b2000 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005e30 tx=0x7f1e3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441033d0 msgr2=0x7f1e44199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441033d0 0x7f1e44199620 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009970 tx=0x7f1e34009e30 comp rx=0 tx=0).stop 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 shutdown_connections 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e441080d0 0x7f1e441a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1e300afaf0 0x7f1e300b2000 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1e441033d0 0x7f1e44199620 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 --2- 172.21.15.78:0/2307803400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e44102190 0x7f1e441990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 >> 172.21.15.78:0/2307803400 conn(0x7f1e440fd380 msgr2=0x7f1e440fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 shutdown_connections 2024-04-25T14:05:12.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.177+0000 7f1e4c172640 1 -- 172.21.15.78:0/2307803400 wait complete. 2024-04-25T14:05:12.188 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:05:12.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.297+0000 7f9a33acd640 1 -- 172.21.15.78:0/1016585758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 msgr2=0x7f9a2c10a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 --2- 172.21.15.78:0/1016585758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c10a4e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009ca0 tx=0x7f9a2402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 -- 172.21.15.78:0/1016585758 shutdown_connections 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 --2- 172.21.15.78:0/1016585758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c10a4e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 --2- 172.21.15.78:0/1016585758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a2c1033c0 0x7f9a2c1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 --2- 172.21.15.78:0/1016585758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a2c102180 0x7f9a2c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 -- 172.21.15.78:0/1016585758 >> 172.21.15.78:0/1016585758 conn(0x7f9a2c0fd350 msgr2=0x7f9a2c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 -- 172.21.15.78:0/1016585758 shutdown_connections 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 -- 172.21.15.78:0/1016585758 wait complete. 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 Processor -- start 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 -- start start 2024-04-25T14:05:12.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.298+0000 7f9a33acd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a2c102180 0x7f9a2c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a33acd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a2c1033c0 0x7f9a2c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a33acd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a33acd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c10d410 con 0x7f9a2c1080c0 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a33acd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c10d290 con 0x7f9a2c1033c0 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a33acd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c10d590 con 0x7f9a2c102180 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c1a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44126/0 (socket says 172.21.15.78:44126) 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 -- 172.21.15.78:0/3557132415 learned_addr learned my addr 172.21.15.78:0/3557132415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a31869640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a2c102180 0x7f9a2c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a31068640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a2c1033c0 0x7f9a2c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 -- 172.21.15.78:0/3557132415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a2c102180 msgr2=0x7f9a2c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a2c102180 0x7f9a2c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 -- 172.21.15.78:0/3557132415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a2c1033c0 msgr2=0x7f9a2c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a2c1033c0 0x7f9a2c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 -- 172.21.15.78:0/3557132415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c1a1c30 con 0x7f9a2c1080c0 2024-04-25T14:05:12.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a3206a640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c1a1670 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9a240097b0 tx=0x7f9a24003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a22ffd640 1 -- 172.21.15.78:0/3557132415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24004240 con 0x7f9a2c1080c0 2024-04-25T14:05:12.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a22ffd640 1 -- 172.21.15.78:0/3557132415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a2402f380 con 0x7f9a2c1080c0 2024-04-25T14:05:12.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c1a1ec0 con 0x7f9a2c1080c0 2024-04-25T14:05:12.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a22ffd640 1 -- 172.21.15.78:0/3557132415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2402fc30 con 0x7f9a2c1080c0 2024-04-25T14:05:12.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.299+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a2320 con 0x7f9a2c1080c0 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 Processor -- start 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a22ffd640 1 -- 172.21.15.78:0/3557132415 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9a240043e0 con 0x7f9a2c1080c0 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 -- start start 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b3ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b3f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b3fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561e60a9ed80 con 0x561e607776b0 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e60a58a20 con 0x561e60798390 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.302+0000 7f9a3a978740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561e60a9ad40 con 0x561e6076f4d0 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b3fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b3fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44136/0 (socket says 172.21.15.78:44136) 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 -- 172.21.15.78:0/3782738528 learned_addr learned my addr 172.21.15.78:0/3782738528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b3ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a22ffd640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9a180afae0 0x7f9a180b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31869640 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b3f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a3206a640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9a180afae0 0x7f9a180b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 -- 172.21.15.78:0/3782738528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 msgr2=0x561e60b3f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b3f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 -- 172.21.15.78:0/3782738528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 msgr2=0x561e60b3ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b3ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 -- 172.21.15.78:0/3782738528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e60a63c30 con 0x561e607776b0 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31869640 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b3f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a22ffd640 1 -- 172.21.15.78:0/3557132415 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9a24100450 con 0x7f9a2c1080c0 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a31068640 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b3fe30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00efd0 tx=0x7f9a1c00e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310fe451506e69af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a17fff640 1 -- 172.21.15.78:0/3782738528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c005020 con 0x561e607776b0 2024-04-25T14:05:12.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a3206a640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9a180afae0 0x7f9a180b1ff0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9a240c5b00 tx=0x7f9a240973e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a17fff640 1 -- 172.21.15.78:0/3782738528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c017070 con 0x561e607776b0 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.303+0000 7f9a17fff640 1 -- 172.21.15.78:0/3782738528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00b670 con 0x561e607776b0 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 -- 172.21.15.78:0/3782738528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 msgr2=0x561e60b3fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b3fe30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00efd0 tx=0x7f9a1c00e3d0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 -- 172.21.15.78:0/3782738528 shutdown_connections 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b3fe30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b3f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 --2- 172.21.15.78:0/3782738528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b3ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 -- 172.21.15.78:0/3782738528 >> 172.21.15.78:0/3782738528 conn(0x561e6078c4e0 msgr2=0x561e60b36020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 -- 172.21.15.78:0/3782738528 shutdown_connections 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.304+0000 7f9a3a978740 1 -- 172.21.15.78:0/3782738528 wait complete. 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 -- start start 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 Processor -- start 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b4fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561e60b40940 con 0x561e607776b0 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e60a25dc0 con 0x561e60798390 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561e60b40ac0 con 0x561e6076f4d0 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a3a978740 10 client.0 _ll_register_callbacks cb 0x561e60a41600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a31869640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a31869640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44150/0 (socket says 172.21.15.78:44150) 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a31869640 1 -- 172.21.15.78:0/2066286764 learned_addr learned my addr 172.21.15.78:0/2066286764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a31068640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.305+0000 7f9a31068640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b4fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a31869640 1 -- 172.21.15.78:0/2066286764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 msgr2=0x561e60b48970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a31869640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a31869640 1 -- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 msgr2=0x561e60b4fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a31869640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b4fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a31869640 1 -- 172.21.15.78:0/2066286764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e607b03f0 con 0x561e607776b0 2024-04-25T14:05:12.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a31068640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a31869640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b48eb0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9a28010740 tx=0x7f9a28010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a634a971015e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a157fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a157fa640 1 -- 172.21.15.78:0/2066286764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28004440 con 0x561e607776b0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a157fa640 1 -- 172.21.15.78:0/2066286764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9a2800e930 con 0x561e607776b0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c017070 con 0x561e607776b0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a157fa640 1 -- 172.21.15.78:0/2066286764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9a28010e50 con 0x561e607776b0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 10 client.17112 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561e6087a7d0 con 0x561e607776b0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a157fa640 1 -- 172.21.15.78:0/2066286764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c017070 con 0x561e607776b0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a157fa640 1 -- 172.21.15.78:0/2066286764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9a28011440 con 0x561e607776b0 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 10 client.17112 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 20 client.17112 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e60756f60 0x561e60b56750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 4 client.17112 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.306+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e60aac990 con 0x561e60756f60 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.307+0000 7f9a31068640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e60756f60 0x561e60b56750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.307+0000 7f9a157fa640 10 client.17112 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.307+0000 7f9a31068640 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e60756f60 0x561e60b56750 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4febca22c299afb2 server_cookie=731d609a4eeeba7c in_seq=0 out_seq=0 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a157fa640 1 -- 172.21.15.78:0/2066286764 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e60aac990 con 0x561e60756f60 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a157fa640 10 client.17112 handle_command_reply: tid=0 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 client.17112 shutdown 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 msgr2=0x561e60b48eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b48eb0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9a28010740 tx=0x7f9a28010c00 comp rx=0 tx=0).stop 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 shutdown_connections 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e60798390 0x561e60b4fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e60756f60 0x561e60b56750 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 --2- 172.21.15.78:0/2066286764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 >> 172.21.15.78:0/2066286764 conn(0x561e6078c4e0 msgr2=0x561e60b35040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a157fa640 0 client.17112 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 shutdown_connections 2024-04-25T14:05:12.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 1 -- 172.21.15.78:0/2066286764 wait complete. 2024-04-25T14:05:12.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.308+0000 7f9a3a978740 20 client.17112 trim_cache size 0 max 16384 2024-04-25T14:05:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 Processor -- start 2024-04-25T14:05:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 -- start start 2024-04-25T14:05:12.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60b4fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1ab890 con 0x561e60798390 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1a9160 con 0x561e607776b0 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a3a978740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1a6fb0 con 0x561e6076f4d0 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60b4fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60b4fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44154/0 (socket says 172.21.15.78:44154) 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 -- 172.21.15.78:0/1129406003 learned_addr learned my addr 172.21.15.78:0/1129406003 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31869640 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 -- 172.21.15.78:0/1129406003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 msgr2=0x561e60b48970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 -- 172.21.15.78:0/1129406003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 msgr2=0x561e60b48eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 -- 172.21.15.78:0/1129406003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e60b50270 con 0x561e60798390 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31869640 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.322+0000 7f9a31068640 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60b4fd30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005d00 tx=0x7f9a1c00e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd03868210dcca5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a177fe640 1 -- 172.21.15.78:0/1129406003 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c02c7e0 con 0x561e60798390 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a177fe640 1 -- 172.21.15.78:0/1129406003 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c017040 con 0x561e60798390 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a177fe640 1 -- 172.21.15.78:0/1129406003 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c02b450 con 0x561e60798390 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 -- 172.21.15.78:0/1129406003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 msgr2=0x561e60b4fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60b4fd30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005d00 tx=0x7f9a1c00e3d0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 -- 172.21.15.78:0/1129406003 shutdown_connections 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60b4fd30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b48eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 --2- 172.21.15.78:0/1129406003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b48970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 -- 172.21.15.78:0/1129406003 >> 172.21.15.78:0/1129406003 conn(0x561e6078c4e0 msgr2=0x561e60b470c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.323+0000 7f9a3a978740 1 -- 172.21.15.78:0/1129406003 shutdown_connections 2024-04-25T14:05:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 -- 172.21.15.78:0/1129406003 wait complete. 2024-04-25T14:05:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 -- start start 2024-04-25T14:05:12.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 Processor -- start 2024-04-25T14:05:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b4c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b4ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60bbcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1a7620 con 0x561e60798390 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1a8a90 con 0x561e607776b0 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.324+0000 7f9a3a978740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1a7cf0 con 0x561e6076f4d0 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a3a978740 10 client.0 _ll_register_callbacks cb 0x7f9a2800c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31068640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60bbcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31869640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b4ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31068640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60bbcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44162/0 (socket says 172.21.15.78:44162) 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31068640 1 -- 172.21.15.78:0/541569040 learned_addr learned my addr 172.21.15.78:0/541569040 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31068640 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b4c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31869640 1 -- 172.21.15.78:0/541569040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 msgr2=0x561e60b4c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31869640 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b4c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31869640 1 -- 172.21.15.78:0/541569040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 msgr2=0x561e60bbcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31869640 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60bbcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31869640 1 -- 172.21.15.78:0/541569040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e60b4fe40 con 0x561e607776b0 2024-04-25T14:05:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31068640 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60bbcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a31869640 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b4ce80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a280118b0 tx=0x7f9a2800cfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daec64aa3fdbc9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.325+0000 7f9a157fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T14:05:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.328+0000 7f9a157fa640 1 -- 172.21.15.78:0/541569040 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28034060 con 0x561e607776b0 2024-04-25T14:05:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.329+0000 7f9a157fa640 1 -- 172.21.15.78:0/541569040 <== mon.2 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9a280052b0 con 0x561e607776b0 2024-04-25T14:05:12.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.329+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c017040 con 0x561e607776b0 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.332+0000 7f9a157fa640 1 -- 172.21.15.78:0/541569040 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9a280054b0 con 0x561e607776b0 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.332+0000 7f9a3a978740 10 client.26186 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.332+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561e60bbde10 con 0x561e607776b0 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.332+0000 7f9a157fa640 1 -- 172.21.15.78:0/541569040 <== mon.2 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c017040 con 0x561e607776b0 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a157fa640 1 -- 172.21.15.78:0/541569040 <== mon.2 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9a28038020 con 0x561e607776b0 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a3a978740 10 client.26186 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a3a978740 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a3a978740 20 client.26186 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a3a978740 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e6072c3a0 0x561e60bc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a3a978740 4 client.26186 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e60aa6800 con 0x561e6072c3a0 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a31068640 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e6072c3a0 0x561e60bc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a157fa640 10 client.26186 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.333+0000 7f9a31068640 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e6072c3a0 0x561e60bc0800 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2507dd1bba6ba5bb server_cookie=d80698f752324c1f in_seq=0 out_seq=0 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a157fa640 1 -- 172.21.15.78:0/541569040 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e60aa6800 con 0x561e6072c3a0 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a157fa640 10 client.26186 handle_command_reply: tid=0 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 client.26186 shutdown 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 msgr2=0x561e60b4ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b4ce80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a280118b0 tx=0x7f9a2800cfd0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 shutdown_connections 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e60798390 0x561e60bbcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e607776b0 0x561e60b4ce80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e6072c3a0 0x561e60bc0800 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 --2- 172.21.15.78:0/541569040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e6076f4d0 0x561e60b4c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a157fa640 0 client.26186 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 >> 172.21.15.78:0/541569040 conn(0x561e6078c4e0 msgr2=0x561e60b48890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 shutdown_connections 2024-04-25T14:05:12.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 1 -- 172.21.15.78:0/541569040 wait complete. 2024-04-25T14:05:12.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a3a978740 20 client.26186 trim_cache size 0 max 16384 2024-04-25T14:05:12.336 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:12.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9a180afae0 msgr2=0x7f9a180b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a33acd640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9a180afae0 0x7f9a180b1ff0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9a240c5b00 tx=0x7f9a240973e0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 msgr2=0x7f9a2c1a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.334+0000 7f9a33acd640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c1a1670 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9a240097b0 tx=0x7f9a24003c20 comp rx=0 tx=0).stop 2024-04-25T14:05:12.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 shutdown_connections 2024-04-25T14:05:12.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9a180afae0 0x7f9a180b1ff0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c1080c0 0x7f9a2c1a1670 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a2c1033c0 0x7f9a2c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 --2- 172.21.15.78:0/3557132415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a2c102180 0x7f9a2c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 >> 172.21.15.78:0/3557132415 conn(0x7f9a2c0fd350 msgr2=0x7f9a2c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 shutdown_connections 2024-04-25T14:05:12.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:12.335+0000 7f9a33acd640 1 -- 172.21.15.78:0/3557132415 wait complete. 2024-04-25T14:05:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1656000856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 -- 172.21.15.78:0/2749145380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d01046c0 msgr2=0x7f11d010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2749145380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d01046c0 0x7f11d010ae10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009900 tx=0x7f11c802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 -- 172.21.15.78:0/2749145380 shutdown_connections 2024-04-25T14:05:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2749145380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d01046c0 0x7f11d010ae10 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2749145380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d0103b60 0x7f11d0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2749145380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d0103000 0x7f11d0103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 -- 172.21.15.78:0/2749145380 >> 172.21.15.78:0/2749145380 conn(0x7f11d00fd3a0 msgr2=0x7f11d00ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 -- 172.21.15.78:0/2749145380 shutdown_connections 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.616+0000 7f11d5be3640 1 -- 172.21.15.78:0/2749145380 wait complete. 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 Processor -- start 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 -- start start 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0103000 0x7f11d0102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d0103b60 0x7f11d019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d019f210 0x7f11d01a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11d010e790 con 0x7f11d0103000 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11d010e610 con 0x7f11d019f210 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11d5be3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d010e910 con 0x7f11d0103b60 2024-04-25T14:05:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0103000 0x7f11d0102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0103000 0x7f11d0102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44182/0 (socket says 172.21.15.78:44182) 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 -- 172.21.15.78:0/2848958758 learned_addr learned my addr 172.21.15.78:0/2848958758 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cffff640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d019f210 0x7f11d01a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11ceffd640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d0103b60 0x7f11d019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 -- 172.21.15.78:0/2848958758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d0103b60 msgr2=0x7f11d019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d0103b60 0x7f11d019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 -- 172.21.15.78:0/2848958758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d019f210 msgr2=0x7f11d01a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d019f210 0x7f11d01a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.617+0000 7f11cf7fe640 1 -- 172.21.15.78:0/2848958758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d01a1eb0 con 0x7f11d0103000 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.618+0000 7f11cf7fe640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0103000 0x7f11d0102300 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f11c000ea80 tx=0x7f11c000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.618+0000 7f11ccff9640 1 -- 172.21.15.78:0/2848958758 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c000ce40 con 0x7f11d0103000 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.618+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d01a21a0 con 0x7f11d0103000 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.618+0000 7f11ccff9640 1 -- 172.21.15.78:0/2848958758 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c0011920 con 0x7f11d0103000 2024-04-25T14:05:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.618+0000 7f11ccff9640 1 -- 172.21.15.78:0/2848958758 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c001aa20 con 0x7f11d0103000 2024-04-25T14:05:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.618+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a26f0 con 0x7f11d0103000 2024-04-25T14:05:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 Processor -- start 2024-04-25T14:05:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11ccff9640 1 -- 172.21.15.78:0/2848958758 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f11c001abc0 con 0x7f11d0103000 2024-04-25T14:05:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 -- start start 2024-04-25T14:05:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7ea1b0 0x5654eab8ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab8f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 0x5654eab96200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654eaaf0460 con 0x5654ea7ea1b0 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654eaaaa590 con 0x5654ea7c12f0 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11dcb69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5654eaaec420 con 0x5654ea7c94d0 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.620+0000 7f11cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7ea1b0 0x5654eab8ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7ea1b0 0x5654eab8ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44192/0 (socket says 172.21.15.78:44192) 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 -- 172.21.15.78:0/2244334179 learned_addr learned my addr 172.21.15.78:0/2244334179 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cf7fe640 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab8f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 -- 172.21.15.78:0/2244334179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 msgr2=0x5654eab96200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11ceffd640 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 0x5654eab96200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 0x5654eab96200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 -- 172.21.15.78:0/2244334179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 msgr2=0x5654eab8f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab8f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 -- 172.21.15.78:0/2244334179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654eaab70d0 con 0x5654ea7ea1b0 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11ccff9640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11b80afae0 0x7f11b80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cf7fe640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11b80afae0 0x7f11b80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11cffff640 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7ea1b0 0x5654eab8ed80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009900 tx=0x7f11c8031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f32fb4a1dbf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11ccff9640 1 -- 172.21.15.78:0/2848958758 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f11c00df0f0 con 0x7f11d0103000 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11b5ffb640 1 -- 172.21.15.78:0/2244334179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8035030 con 0x5654ea7ea1b0 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11b5ffb640 1 -- 172.21.15.78:0/2244334179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c8040070 con 0x5654ea7ea1b0 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.621+0000 7f11b5ffb640 1 -- 172.21.15.78:0/2244334179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8003b10 con 0x5654ea7ea1b0 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 -- 172.21.15.78:0/2244334179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7ea1b0 msgr2=0x5654eab8ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11cf7fe640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11b80afae0 0x7f11b80b1ff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5654eab90160 tx=0x7f11c00e2bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7ea1b0 0x5654eab8ed80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009900 tx=0x7f11c8031c40 comp rx=0 tx=0).stop 2024-04-25T14:05:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 -- 172.21.15.78:0/2244334179 shutdown_connections 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 0x5654eab96200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab8f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 --2- 172.21.15.78:0/2244334179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7ea1b0 0x5654eab8ed80 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 -- 172.21.15.78:0/2244334179 >> 172.21.15.78:0/2244334179 conn(0x5654ea7de300 msgr2=0x5654eab86300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 -- 172.21.15.78:0/2244334179 shutdown_connections 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.622+0000 7f11dcb69740 1 -- 172.21.15.78:0/2244334179 wait complete. 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 -- start start 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 Processor -- start 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c12f0 0x5654eab98a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab98f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 0x5654eab9fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654eab96bf0 con 0x5654ea7c94d0 2024-04-25T14:05:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654eaa77b60 con 0x5654ea7ea1b0 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5654eab96d70 con 0x5654ea7c12f0 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11dcb69740 10 client.0 _ll_register_callbacks cb 0x5654eaa9c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab98f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab98f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44204/0 (socket says 172.21.15.78:44204) 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11ceffd640 1 -- 172.21.15.78:0/793187065 learned_addr learned my addr 172.21.15.78:0/793187065 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11cffff640 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c12f0 0x5654eab98a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11cffff640 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 0x5654eab9fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11ceffd640 1 -- 172.21.15.78:0/793187065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c12f0 msgr2=0x5654eab98a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11ceffd640 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c12f0 0x5654eab98a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.623+0000 7f11ceffd640 1 -- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 msgr2=0x5654eab9fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11ceffd640 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 0x5654eab9fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11ceffd640 1 -- 172.21.15.78:0/793187065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ea802210 con 0x5654ea7c94d0 2024-04-25T14:05:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11ceffd640 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab98f90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f11c400b940 tx=0x7f11c400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfaeb0366cb6af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11aeffd640 1 -- 172.21.15.78:0/793187065 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4005020 con 0x5654ea7c94d0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11aeffd640 1 -- 172.21.15.78:0/793187065 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f11c40051c0 con 0x5654ea7c94d0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c8040070 con 0x5654ea7c94d0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11aeffd640 1 -- 172.21.15.78:0/793187065 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f11c40053c0 con 0x5654ea7c94d0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 10 client.17130 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5654ea8ce620 con 0x5654ea7c94d0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11aeffd640 1 -- 172.21.15.78:0/793187065 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c8040070 con 0x5654ea7c94d0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11aeffd640 1 -- 172.21.15.78:0/793187065 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f11c4012650 con 0x5654ea7c94d0 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 10 client.17130 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 20 client.17130 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea7a8d80 0x5654eaba6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 4 client.17130 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.624+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654eaafe070 con 0x5654ea7a8d80 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.625+0000 7f11cffff640 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea7a8d80 0x5654eaba6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.625+0000 7f11aeffd640 10 client.17130 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.625+0000 7f11cffff640 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea7a8d80 0x5654eaba6820 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f4a70a3434bcdd server_cookie=bc1bc3e71548e590 in_seq=0 out_seq=0 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11aeffd640 1 -- 172.21.15.78:0/793187065 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654eaafe070 con 0x5654ea7a8d80 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11aeffd640 10 client.17130 handle_command_reply: tid=0 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 client.17130 shutdown 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 msgr2=0x5654eab98f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab98f90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f11c400b940 tx=0x7f11c400be00 comp rx=0 tx=0).stop 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 shutdown_connections 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 0x5654eab9fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab98f90 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea7a8d80 0x5654eaba6820 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 --2- 172.21.15.78:0/793187065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c12f0 0x5654eab98a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11aeffd640 0 client.17130 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 >> 172.21.15.78:0/793187065 conn(0x5654ea7de300 msgr2=0x5654eab85250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 shutdown_connections 2024-04-25T14:05:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 1 -- 172.21.15.78:0/793187065 wait complete. 2024-04-25T14:05:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.626+0000 7f11dcb69740 20 client.17130 trim_cache size 0 max 16384 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 Processor -- start 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 -- start start 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c12f0 0x5654eab85be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 0x5654eab9fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 0x5654eab86120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01ab830 con 0x5654ea7c12f0 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01a9100 con 0x5654ea7ea1b0 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11dcb69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01a6f70 con 0x5654ea7c94d0 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c12f0 0x5654eab85be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c12f0 0x5654eab85be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44214/0 (socket says 172.21.15.78:44214) 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.641+0000 7f11cffff640 1 -- 172.21.15.78:0/117365233 learned_addr learned my addr 172.21.15.78:0/117365233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11cffff640 1 -- 172.21.15.78:0/117365233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 msgr2=0x5654eab9fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11cffff640 1 --2- 172.21.15.78:0/117365233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 0x5654eab9fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11cffff640 1 -- 172.21.15.78:0/117365233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 msgr2=0x5654eab86120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11cffff640 1 --2- 172.21.15.78:0/117365233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 0x5654eab86120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11cffff640 1 -- 172.21.15.78:0/117365233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654eaba0350 con 0x5654ea7c12f0 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11cffff640 1 --2- 172.21.15.78:0/117365233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c12f0 0x5654eab85be0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c8031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741c03a3e5f87dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11b57fa640 1 -- 172.21.15.78:0/117365233 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8035040 con 0x5654ea7c12f0 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11b57fa640 1 -- 172.21.15.78:0/117365233 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c8040040 con 0x5654ea7c12f0 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.642+0000 7f11b57fa640 1 -- 172.21.15.78:0/117365233 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8045470 con 0x5654ea7c12f0 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- 172.21.15.78:0/117365233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c12f0 msgr2=0x5654eab85be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 --2- 172.21.15.78:0/117365233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c12f0 0x5654eab85be0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c8031c40 comp rx=0 tx=0).stop 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- 172.21.15.78:0/117365233 shutdown_connections 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 --2- 172.21.15.78:0/117365233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7ea1b0 0x5654eab86120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 --2- 172.21.15.78:0/117365233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7c94d0 0x5654eab9fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 --2- 172.21.15.78:0/117365233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c12f0 0x5654eab85be0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- 172.21.15.78:0/117365233 >> 172.21.15.78:0/117365233 conn(0x5654ea7de300 msgr2=0x5654eab92f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- 172.21.15.78:0/117365233 shutdown_connections 2024-04-25T14:05:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- 172.21.15.78:0/117365233 wait complete. 2024-04-25T14:05:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- start start 2024-04-25T14:05:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 Processor -- start 2024-04-25T14:05:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab9ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab9d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7ea1b0 0x5654eac0cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01a75c0 con 0x5654ea7c94d0 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01a8a30 con 0x5654ea7c12f0 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.643+0000 7f11dcb69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01a7c90 con 0x5654ea7ea1b0 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11dcb69740 10 client.0 _ll_register_callbacks cb 0x7f11c4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab9d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab9d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44222/0 (socket says 172.21.15.78:44222) 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 -- 172.21.15.78:0/1344267186 learned_addr learned my addr 172.21.15.78:0/1344267186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11cffff640 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab9ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 -- 172.21.15.78:0/1344267186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7ea1b0 msgr2=0x5654eac0cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7ea1b0 0x5654eac0cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 -- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 msgr2=0x5654eab9ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab9ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 -- 172.21.15.78:0/1344267186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654eac0d100 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11ceffd640 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab9d3c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f11c4001eb0 tx=0x7f11c4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc7e946dc2a1bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11aeffd640 1 -- 172.21.15.78:0/1344267186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4033060 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11aeffd640 1 -- 172.21.15.78:0/1344267186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f11c4003d60 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c8040040 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11aeffd640 1 -- 172.21.15.78:0/1344267186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f11c402a040 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11dcb69740 10 client.17136 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5654eac0e3d0 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.644+0000 7f11aeffd640 1 -- 172.21.15.78:0/1344267186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c8040040 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11aeffd640 1 -- 172.21.15.78:0/1344267186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f11c401aa70 con 0x5654ea7c94d0 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11dcb69740 10 client.17136 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11dcb69740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11dcb69740 20 client.17136 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11dcb69740 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea77e1c0 0x5654eac10d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11dcb69740 4 client.17136 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654eaaf7ee0 con 0x5654ea77e1c0 2024-04-25T14:05:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11cffff640 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea77e1c0 0x5654eac10d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11aeffd640 10 client.17136 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.645+0000 7f11cffff640 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea77e1c0 0x5654eac10d70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7330798c9a37c683 server_cookie=91722677f4b0e76c in_seq=0 out_seq=0 2024-04-25T14:05:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11aeffd640 1 -- 172.21.15.78:0/1344267186 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654eaaf7ee0 con 0x5654ea77e1c0 2024-04-25T14:05:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11aeffd640 10 client.17136 handle_command_reply: tid=0 2024-04-25T14:05:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 client.17136 shutdown 2024-04-25T14:05:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 msgr2=0x5654eab9d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab9d3c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f11c4001eb0 tx=0x7f11c4004530 comp rx=0 tx=0).stop 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 shutdown_connections 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654ea77e1c0 0x5654eac10d70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654ea7ea1b0 0x5654eac0cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ea7c94d0 0x5654eab9d3c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 --2- 172.21.15.78:0/1344267186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654ea7c12f0 0x5654eab9ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11aeffd640 0 client.17136 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 >> 172.21.15.78:0/1344267186 conn(0x5654ea7de300 msgr2=0x5654eab85be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 shutdown_connections 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.660+0000 7f11dcb69740 1 -- 172.21.15.78:0/1344267186 wait complete. 2024-04-25T14:05:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11dcb69740 20 client.17136 trim_cache size 0 max 16384 2024-04-25T14:05:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:05:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:05:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:05:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11b80afae0 msgr2=0x7f11b80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11b80afae0 0x7f11b80b1ff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5654eab90160 tx=0x7f11c00e2bd0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0103000 msgr2=0x7f11d0102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0103000 0x7f11d0102300 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f11c000ea80 tx=0x7f11c000ef40 comp rx=0 tx=0).stop 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 shutdown_connections 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d019f210 0x7f11d01a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11d0103b60 0x7f11d019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f11b80afae0 0x7f11b80b1ff0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 --2- 172.21.15.78:0/2848958758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d0103000 0x7f11d0102300 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 >> 172.21.15.78:0/2848958758 conn(0x7f11d00fd3a0 msgr2=0x7f11d00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 shutdown_connections 2024-04-25T14:05:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:12.661+0000 7f11d5be3640 1 -- 172.21.15.78:0/2848958758 wait complete. 2024-04-25T14:05:12.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1656000856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:12.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1656000856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:13.346 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:05:13.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:13.346 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 -- 172.21.15.78:0/2710456334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558101e50 msgr2=0x7f0558106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 --2- 172.21.15.78:0/2710456334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558101e50 0x7f0558106b10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f054c009900 tx=0x7f054c02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 -- 172.21.15.78:0/2710456334 shutdown_connections 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 --2- 172.21.15.78:0/2710456334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0558109770 0x7f055810bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 --2- 172.21.15.78:0/2710456334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0558102350 0x7f05581090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 --2- 172.21.15.78:0/2710456334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558101e50 0x7f0558106b10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 -- 172.21.15.78:0/2710456334 >> 172.21.15.78:0/2710456334 conn(0x7f05580fd340 msgr2=0x7f05580ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 -- 172.21.15.78:0/2710456334 shutdown_connections 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 -- 172.21.15.78:0/2710456334 wait complete. 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.467+0000 7f055f05a640 1 Processor -- start 2024-04-25T14:05:13.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055f05a640 1 -- start start 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055f05a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0558101e50 0x7f0558104320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055f05a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0558102350 0x7f05581029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055f05a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558109770 0x7f0558102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055f05a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f055810f5b0 con 0x7f0558109770 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055f05a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f055810f430 con 0x7f0558101e50 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055f05a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f055810f730 con 0x7f0558102350 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558109770 0x7f0558102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558109770 0x7f0558102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44240/0 (socket says 172.21.15.78:44240) 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 -- 172.21.15.78:0/4283998831 learned_addr learned my addr 172.21.15.78:0/4283998831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055cdf6640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0558101e50 0x7f0558104320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f0557fff640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0558102350 0x7f05581029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 -- 172.21.15.78:0/4283998831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0558102350 msgr2=0x7f05581029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0558102350 0x7f05581029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 -- 172.21.15.78:0/4283998831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0558101e50 msgr2=0x7f0558104320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0558101e50 0x7f0558104320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 -- 172.21.15.78:0/4283998831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0558103630 con 0x7f0558109770 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f055d5f7640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558109770 0x7f0558102f00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f055000eaa0 tx=0x7f055000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.468+0000 7f0555ffb640 1 -- 172.21.15.78:0/4283998831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000cdb0 con 0x7f0558109770 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.469+0000 7f0555ffb640 1 -- 172.21.15.78:0/4283998831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0550005020 con 0x7f0558109770 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.469+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05581a42d0 con 0x7f0558109770 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.469+0000 7f0555ffb640 1 -- 172.21.15.78:0/4283998831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500116d0 con 0x7f0558109770 2024-04-25T14:05:13.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.469+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05581a4740 con 0x7f0558109770 2024-04-25T14:05:13.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.471+0000 7f0555ffb640 1 -- 172.21.15.78:0/4283998831 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f05500051d0 con 0x7f0558109770 2024-04-25T14:05:13.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.471+0000 7f0565fea740 1 Processor -- start 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.471+0000 7f0565fea740 1 -- start start 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0565fea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1be4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0565fea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559df4550 0x55f55a1bea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0565fea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559dfc730 0x55f55a1c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0565fea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f55a123b00 con 0x55f559dfc730 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0565fea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f55a0bf920 con 0x55f559df4550 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0565fea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f55a11fac0 con 0x55f559e1d410 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559dfc730 0x55f55a1c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559dfc730 0x55f55a1c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44242/0 (socket says 172.21.15.78:44242) 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 -- 172.21.15.78:0/2249312983 learned_addr learned my addr 172.21.15.78:0/2249312983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f055cdf6640 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559df4550 0x55f55a1bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0555ffb640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05480afaf0 0x7f05480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1be4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f055d5f7640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05480afaf0 0x7f05480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 -- 172.21.15.78:0/2249312983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 msgr2=0x55f55a1be4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1be4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 -- 172.21.15.78:0/2249312983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559df4550 msgr2=0x55f55a1bea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559df4550 0x55f55a1bea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 -- 172.21.15.78:0/2249312983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f559efd7e0 con 0x55f559dfc730 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0555ffb640 1 -- 172.21.15.78:0/4283998831 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f05500dee50 con 0x7f0558109770 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0557fff640 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559dfc730 0x55f55a1c5920 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f054400bfc0 tx=0x7f054400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67acf62af9f8dc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0542ffd640 1 -- 172.21.15.78:0/2249312983 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544019030 con 0x55f559dfc730 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.472+0000 7f0542ffd640 1 -- 172.21.15.78:0/2249312983 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0544011970 con 0x55f559dfc730 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0542ffd640 1 -- 172.21.15.78:0/2249312983 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054401d800 con 0x55f559dfc730 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 -- 172.21.15.78:0/2249312983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559dfc730 msgr2=0x55f55a1c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f055d5f7640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05480afaf0 0x7f05480b2000 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f05500a33a0 tx=0x7f05500d5a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559dfc730 0x55f55a1c5920 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f054400bfc0 tx=0x7f054400b670 comp rx=0 tx=0).stop 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 -- 172.21.15.78:0/2249312983 shutdown_connections 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559dfc730 0x55f55a1c5920 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559df4550 0x55f55a1bea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 --2- 172.21.15.78:0/2249312983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1be4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 -- 172.21.15.78:0/2249312983 >> 172.21.15.78:0/2249312983 conn(0x55f559e11560 msgr2=0x55f55a1b9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 -- 172.21.15.78:0/2249312983 shutdown_connections 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.473+0000 7f0565fea740 1 -- 172.21.15.78:0/2249312983 wait complete. 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 -- start start 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 Processor -- start 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f55a1c6370 con 0x55f559df4550 2024-04-25T14:05:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f55a0bd5d0 con 0x55f559dfc730 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f55a1c64f0 con 0x55f559e1d410 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0565fea740 10 client.0 _ll_register_callbacks cb 0x55f55a0ab080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44258/0 (socket says 172.21.15.78:44258) 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0557fff640 1 -- 172.21.15.78:0/4183188722 learned_addr learned my addr 172.21.15.78:0/4183188722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f0557fff640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.474+0000 7f055cdf6640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0557fff640 1 -- 172.21.15.78:0/4183188722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 msgr2=0x55f55a1cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0557fff640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0557fff640 1 -- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 msgr2=0x55f55a1c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0557fff640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0557fff640 1 -- 172.21.15.78:0/4183188722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f559f5df70 con 0x55f559df4550 2024-04-25T14:05:13.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f055cdf6640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0557fff640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8240 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f054400b640 tx=0x7f054400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78591390dec49e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0537fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0537fff640 1 -- 172.21.15.78:0/4183188722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544019040 con 0x55f559df4550 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0537fff640 1 -- 172.21.15.78:0/4183188722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0544011e20 con 0x55f559df4550 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0544011970 con 0x55f559df4550 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0537fff640 1 -- 172.21.15.78:0/4183188722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0544005040 con 0x55f559df4550 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0565fea740 10 client.17148 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f559df5170 con 0x55f559df4550 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0537fff640 1 -- 172.21.15.78:0/4183188722 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0544011970 con 0x55f559df4550 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0537fff640 1 -- 172.21.15.78:0/4183188722 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0544004c70 con 0x55f559df4550 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0565fea740 10 client.17148 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0565fea740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.475+0000 7f0565fea740 20 client.17148 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.476+0000 7f0565fea740 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559ddc1f0 0x55f55a1d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.476+0000 7f0565fea740 4 client.17148 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.476+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f55a131710 con 0x55f559ddc1f0 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.476+0000 7f055cdf6640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559ddc1f0 0x55f55a1d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.476+0000 7f055cdf6640 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559ddc1f0 0x55f55a1d5870 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef6aa62047da6e9d server_cookie=ee9a75d982f82441 in_seq=0 out_seq=0 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.476+0000 7f0537fff640 10 client.17148 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0537fff640 1 -- 172.21.15.78:0/4183188722 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f55a131710 con 0x55f559ddc1f0 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0537fff640 10 client.17148 handle_command_reply: tid=0 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 client.17148 shutdown 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 msgr2=0x55f55a1c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8240 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f054400b640 tx=0x7f054400b7c0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 shutdown_connections 2024-04-25T14:05:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559ddc1f0 0x55f55a1d5870 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559e1d410 0x55f55a1cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 --2- 172.21.15.78:0/4183188722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8240 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0537fff640 0 client.17148 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 >> 172.21.15.78:0/4183188722 conn(0x55f559e11560 msgr2=0x55f55a1b8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 shutdown_connections 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 1 -- 172.21.15.78:0/4183188722 wait complete. 2024-04-25T14:05:13.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.477+0000 7f0565fea740 20 client.17148 trim_cache size 0 max 16384 2024-04-25T14:05:13.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 Processor -- start 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 -- start start 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559dfc730 0x55f55a1cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559e1d410 0x55f55a1b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05581afda0 con 0x55f559df4550 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05581ad670 con 0x55f559e1d410 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0565fea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05581ab4c0 con 0x55f559dfc730 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44260/0 (socket says 172.21.15.78:44260) 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f0557fff640 1 -- 172.21.15.78:0/47621345 learned_addr learned my addr 172.21.15.78:0/47621345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.495+0000 7f055cdf6640 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559dfc730 0x55f55a1cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f0557fff640 1 -- 172.21.15.78:0/47621345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559dfc730 msgr2=0x55f55a1cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f0557fff640 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559dfc730 0x55f55a1cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f0557fff640 1 -- 172.21.15.78:0/47621345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559e1d410 msgr2=0x55f55a1b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f0557fff640 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559e1d410 0x55f55a1b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f0557fff640 1 -- 172.21.15.78:0/47621345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55a1b5600 con 0x55f559df4550 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f055cdf6640 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559dfc730 0x55f55a1cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f0557fff640 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8520 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0544013e40 tx=0x7f054400b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb64dbf6c7d2e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f05427fc640 1 -- 172.21.15.78:0/47621345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544019030 con 0x55f559df4550 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f05427fc640 1 -- 172.21.15.78:0/47621345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054401d480 con 0x55f559df4550 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.496+0000 7f05427fc640 1 -- 172.21.15.78:0/47621345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440118a0 con 0x55f559df4550 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- 172.21.15.78:0/47621345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 msgr2=0x55f55a1c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8520 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0544013e40 tx=0x7f054400b510 comp rx=0 tx=0).stop 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- 172.21.15.78:0/47621345 shutdown_connections 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559e1d410 0x55f55a1b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559dfc730 0x55f55a1cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 --2- 172.21.15.78:0/47621345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559df4550 0x55f55a1c8520 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- 172.21.15.78:0/47621345 >> 172.21.15.78:0/47621345 conn(0x55f559e11560 msgr2=0x55f55a1c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- 172.21.15.78:0/47621345 shutdown_connections 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- 172.21.15.78:0/47621345 wait complete. 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- start start 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 Processor -- start 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559df4550 0x55f55a1cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559e1d410 0x55f55a1ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05581abb30 con 0x55f559e1d410 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05581acfa0 con 0x55f559dfc730 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.497+0000 7f0565fea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05581ac200 con 0x55f559df4550 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0565fea740 10 client.0 _ll_register_callbacks cb 0x7f0544011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:13.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559e1d410 0x55f55a1ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559e1d410 0x55f55a1ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44272/0 (socket says 172.21.15.78:44272) 2024-04-25T14:05:13.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 -- 172.21.15.78:0/2573660169 learned_addr learned my addr 172.21.15.78:0/2573660169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f055cdf6640 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559df4550 0x55f55a1cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 -- 172.21.15.78:0/2573660169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559df4550 msgr2=0x55f55a1cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559df4550 0x55f55a1cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 -- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 msgr2=0x55f55a1cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 -- 172.21.15.78:0/2573660169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55a1ceca0 con 0x55f559e1d410 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0557fff640 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559e1d410 0x55f55a1ce6e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f054400be40 tx=0x7f0544013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c5993292867c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0537fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:13.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0537fff640 1 -- 172.21.15.78:0/2573660169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544019070 con 0x55f559e1d410 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0537fff640 1 -- 172.21.15.78:0/2573660169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0544023040 con 0x55f559e1d410 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054401d480 con 0x55f559e1d410 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0537fff640 1 -- 172.21.15.78:0/2573660169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0544011ab0 con 0x55f559e1d410 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0565fea740 10 client.17154 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.498+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f55a23df50 con 0x55f559e1d410 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0537fff640 1 -- 172.21.15.78:0/2573660169 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054401d480 con 0x55f559e1d410 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0537fff640 1 -- 172.21.15.78:0/2573660169 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0544036020 con 0x55f559e1d410 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0565fea740 10 client.17154 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0565fea740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0565fea740 20 client.17154 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0565fea740 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559db1630 0x55f55a2408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0565fea740 4 client.17154 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:05:13.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f55a12b580 con 0x55f559db1630 2024-04-25T14:05:13.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f055cdf6640 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559db1630 0x55f55a2408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f0537fff640 10 client.17154 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.499+0000 7f055cdf6640 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559db1630 0x55f55a2408c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522e3ed1eb84f999 server_cookie=df80000a9b0919ed in_seq=0 out_seq=0 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0537fff640 1 -- 172.21.15.78:0/2573660169 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f55a12b580 con 0x55f559db1630 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0537fff640 10 client.17154 handle_command_reply: tid=0 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 client.17154 shutdown 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559e1d410 msgr2=0x55f55a1ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559e1d410 0x55f55a1ce6e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f054400be40 tx=0x7f0544013f90 comp rx=0 tx=0).stop 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 shutdown_connections 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f559e1d410 0x55f55a1ce6e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f559dfc730 0x55f55a1cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f559db1630 0x55f55a2408c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 --2- 172.21.15.78:0/2573660169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f559df4550 0x55f55a1cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 >> 172.21.15.78:0/2573660169 conn(0x55f559e11560 msgr2=0x55f55a1c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0537fff640 0 client.17154 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 shutdown_connections 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 1 -- 172.21.15.78:0/2573660169 wait complete. 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f0565fea740 20 client.17154 trim_cache size 0 max 16384 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "52e6584e-d2ff-4a86-9b6b-32ac30ed5bd7", 2024-04-25T14:05:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:05:13.510 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:13.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05480afaf0 msgr2=0x7f05480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.501+0000 7f055f05a640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05480afaf0 0x7f05480b2000 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f05500a33a0 tx=0x7f05500d5a70 comp rx=0 tx=0).stop 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558109770 msgr2=0x7f0558102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558109770 0x7f0558102f00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f055000eaa0 tx=0x7f055000ef60 comp rx=0 tx=0).stop 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 shutdown_connections 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05480afaf0 0x7f05480b2000 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0558109770 0x7f0558102f00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0558102350 0x7f05581029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 --2- 172.21.15.78:0/4283998831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0558101e50 0x7f0558104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 >> 172.21.15.78:0/4283998831 conn(0x7f05580fd340 msgr2=0x7f0558106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 shutdown_connections 2024-04-25T14:05:13.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.502+0000 7f055f05a640 1 -- 172.21.15.78:0/4283998831 wait complete. 2024-04-25T14:05:13.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:13.512 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 -- 172.21.15.78:0/262696178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c101760 msgr2=0x7f1a3c101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 --2- 172.21.15.78:0/262696178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c101760 0x7f1a3c101bd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009900 tx=0x7f1a2c02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 -- 172.21.15.78:0/262696178 shutdown_connections 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 --2- 172.21.15.78:0/262696178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c102260 0x7f1a3c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 --2- 172.21.15.78:0/262696178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c101760 0x7f1a3c101bd0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 --2- 172.21.15.78:0/262696178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3c1050b0 0x7f1a3c1054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 -- 172.21.15.78:0/262696178 >> 172.21.15.78:0/262696178 conn(0x7f1a3c0fd380 msgr2=0x7f1a3c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 -- 172.21.15.78:0/262696178 shutdown_connections 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.623+0000 7f1a43492640 1 -- 172.21.15.78:0/262696178 wait complete. 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 Processor -- start 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 -- start start 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3c101760 0x7f1a3c104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c102260 0x7f1a3c102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1050b0 0x7f1a3c102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c10f640 con 0x7f1a3c102260 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c10f4c0 con 0x7f1a3c1050b0 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a43492640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c10f7c0 con 0x7f1a3c101760 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a40a2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c102260 0x7f1a3c102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a40a2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c102260 0x7f1a3c102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44286/0 (socket says 172.21.15.78:44286) 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a40a2d640 1 -- 172.21.15.78:0/1419573844 learned_addr learned my addr 172.21.15.78:0/1419573844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.624+0000 7f1a4122e640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3c101760 0x7f1a3c104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1050b0 0x7f1a3c102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a40a2d640 1 -- 172.21.15.78:0/1419573844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3c101760 msgr2=0x7f1a3c104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3c101760 0x7f1a3c104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a40a2d640 1 -- 172.21.15.78:0/1419573844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1050b0 msgr2=0x7f1a3c102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1050b0 0x7f1a3c102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a40a2d640 1 -- 172.21.15.78:0/1419573844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c103570 con 0x7f1a3c102260 2024-04-25T14:05:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c102260 0x7f1a3c102900 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006010 tx=0x7f1a2c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a327fc640 1 -- 172.21.15.78:0/1419573844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c03f030 con 0x7f1a3c102260 2024-04-25T14:05:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a327fc640 1 -- 172.21.15.78:0/1419573844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a2c003cb0 con 0x7f1a3c102260 2024-04-25T14:05:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c1a4330 con 0x7f1a3c102260 2024-04-25T14:05:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a327fc640 1 -- 172.21.15.78:0/1419573844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c035070 con 0x7f1a3c102260 2024-04-25T14:05:13.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.625+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c1a4730 con 0x7f1a3c102260 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 Processor -- start 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a327fc640 1 -- 172.21.15.78:0/1419573844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1a2c042030 con 0x7f1a3c102260 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 -- start start 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caa93a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa93fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caa94c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5646ca9f8d80 con 0x5646ca6c94d0 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646ca9b28b0 con 0x5646ca6f2390 2024-04-25T14:05:13.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4a415740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5646ca9f4d40 con 0x5646ca6d16b0 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4122e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa93fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4122e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa93fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44296/0 (socket says 172.21.15.78:44296) 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4122e640 1 -- 172.21.15.78:0/1782245243 learned_addr learned my addr 172.21.15.78:0/1782245243 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caa94c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caa93a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.628+0000 7f1a4122e640 1 -- 172.21.15.78:0/1782245243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 msgr2=0x5646caa94c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a4122e640 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caa94c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a4122e640 1 -- 172.21.15.78:0/1782245243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 msgr2=0x5646caa93a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a4122e640 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caa93a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a4122e640 1 -- 172.21.15.78:0/1782245243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ca983a40 con 0x5646ca6c94d0 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a327fc640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a280afaf0 0x7f1a280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a280afaf0 0x7f1a280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a4122e640 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa93fe0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800bc70 tx=0x7f1a380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f0ebe98e5efdcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a327fc640 1 -- 172.21.15.78:0/1419573844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1a2c0ff2c0 con 0x7f1a3c102260 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a277fe640 1 -- 172.21.15.78:0/1782245243 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3801c680 con 0x5646ca6c94d0 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a277fe640 1 -- 172.21.15.78:0/1782245243 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3800be20 con 0x5646ca6c94d0 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.629+0000 7f1a277fe640 1 -- 172.21.15.78:0/1782245243 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3801a660 con 0x5646ca6c94d0 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 -- 172.21.15.78:0/1782245243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 msgr2=0x5646caa93fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a280afaf0 0x7f1a280b2000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x5646caa94ab0 tx=0x7f1a34009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa93fe0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800bc70 tx=0x7f1a380107a0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 -- 172.21.15.78:0/1782245243 shutdown_connections 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caa94c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa93fe0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 --2- 172.21.15.78:0/1782245243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caa93a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 -- 172.21.15.78:0/1782245243 >> 172.21.15.78:0/1782245243 conn(0x5646ca6e64e0 msgr2=0x5646caa8f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 -- 172.21.15.78:0/1782245243 shutdown_connections 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 -- 172.21.15.78:0/1782245243 wait complete. 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 -- start start 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.630+0000 7f1a4a415740 1 Processor -- start 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6c94d0 0x5646caa9d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6d16b0 0x5646caa9deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caaa4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4a415740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5646caa95780 con 0x5646ca6d16b0 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4a415740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5646ca97fd10 con 0x5646ca6f2390 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4a415740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5646caa95900 con 0x5646ca6c94d0 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4a415740 10 client.0 _ll_register_callbacks cb 0x5646ca9ac210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6d16b0 0x5646caa9deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6d16b0 0x5646caa9deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44312/0 (socket says 172.21.15.78:44312) 2024-04-25T14:05:13.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 -- 172.21.15.78:0/3722663881 learned_addr learned my addr 172.21.15.78:0/3722663881 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a4122e640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6c94d0 0x5646caa9d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caaa4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 -- 172.21.15.78:0/3722663881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6c94d0 msgr2=0x5646caa9d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6c94d0 0x5646caa9d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 -- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 msgr2=0x5646caaa4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caaa4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 -- 172.21.15.78:0/3722663881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ca7d47d0 con 0x5646ca6d16b0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caaa4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6d16b0 0x5646caa9deb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0fcdb0 tx=0x7f1a2c102990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb291493ff90a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3722663881 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c049070 con 0x5646ca6d16b0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.631+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3722663881 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1a2c0b0110 con 0x5646ca6d16b0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3800be20 con 0x5646ca6d16b0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3722663881 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1a2c0c3650 con 0x5646ca6d16b0 2024-04-25T14:05:13.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 10 client.17166 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5646ca70a3f0 con 0x5646ca6d16b0 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3722663881 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3800be20 con 0x5646ca6d16b0 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3722663881 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1a2c03bc20 con 0x5646ca6d16b0 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 10 client.17166 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 20 client.17166 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6b0f60 0x5646caaab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 4 client.17166 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646caa06990 con 0x5646ca6b0f60 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.632+0000 7f1a4122e640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6b0f60 0x5646caaab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.633+0000 7f1a24ff9640 10 client.17166 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.633+0000 7f1a4122e640 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6b0f60 0x5646caaab2b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5293ef649244fbda server_cookie=67db84695a37b420 in_seq=0 out_seq=0 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3722663881 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646caa06990 con 0x5646ca6b0f60 2024-04-25T14:05:13.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a24ff9640 10 client.17166 handle_command_reply: tid=0 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 client.17166 shutdown 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6d16b0 msgr2=0x5646caa9deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6d16b0 0x5646caa9deb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0fcdb0 tx=0x7f1a2c102990 comp rx=0 tx=0).stop 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 shutdown_connections 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6f2390 0x5646caaa4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6d16b0 0x5646caa9deb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6b0f60 0x5646caaab2b0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3722663881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6c94d0 0x5646caa9d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a24ff9640 0 client.17166 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 >> 172.21.15.78:0/3722663881 conn(0x5646ca6e64e0 msgr2=0x5646caa92180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 shutdown_connections 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 1 -- 172.21.15.78:0/3722663881 wait complete. 2024-04-25T14:05:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.634+0000 7f1a4a415740 20 client.17166 trim_cache size 0 max 16384 2024-04-25T14:05:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 Processor -- start 2024-04-25T14:05:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 -- start start 2024-04-25T14:05:13.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6d16b0 0x5646caaa4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6f2390 0x5646caa8eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1afcf0 con 0x5646ca6c94d0 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1ad5c0 con 0x5646ca6d16b0 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4a415740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1ab410 con 0x5646ca6f2390 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa9c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44316/0 (socket says 172.21.15.78:44316) 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 -- 172.21.15.78:0/4209071962 learned_addr learned my addr 172.21.15.78:0/4209071962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6f2390 0x5646caa8eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6d16b0 0x5646caaa4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 -- 172.21.15.78:0/4209071962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6f2390 msgr2=0x5646caa8eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6f2390 0x5646caa8eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 -- 172.21.15.78:0/4209071962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6d16b0 msgr2=0x5646caaa4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6d16b0 0x5646caaa4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a4122e640 1 -- 172.21.15.78:0/4209071962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ca6ca0f0 con 0x5646ca6c94d0 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.652+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6f2390 0x5646caa8eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.653+0000 7f1a4122e640 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa9c680 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800bb90 tx=0x7f1a3800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ec197f9f4f573b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.653+0000 7f1a26ffd640 1 -- 172.21.15.78:0/4209071962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3802e020 con 0x5646ca6c94d0 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.653+0000 7f1a26ffd640 1 -- 172.21.15.78:0/4209071962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a38004660 con 0x5646ca6c94d0 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.653+0000 7f1a26ffd640 1 -- 172.21.15.78:0/4209071962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3801c6e0 con 0x5646ca6c94d0 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.653+0000 7f1a4a415740 1 -- 172.21.15.78:0/4209071962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 msgr2=0x5646caa9c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.653+0000 7f1a4a415740 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa9c680 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800bb90 tx=0x7f1a3800b390 comp rx=0 tx=0).stop 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- 172.21.15.78:0/4209071962 shutdown_connections 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6f2390 0x5646caa8eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6d16b0 0x5646caaa4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 --2- 172.21.15.78:0/4209071962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6c94d0 0x5646caa9c680 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- 172.21.15.78:0/4209071962 >> 172.21.15.78:0/4209071962 conn(0x5646ca6e64e0 msgr2=0x5646caa9bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- 172.21.15.78:0/4209071962 shutdown_connections 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- 172.21.15.78:0/4209071962 wait complete. 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- start start 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 Processor -- start 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6c94d0 0x5646caaa1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caaa2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6f2390 0x5646cab11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1aba80 con 0x5646ca6f2390 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1acef0 con 0x5646ca6c94d0 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1ac150 con 0x5646ca6d16b0 2024-04-25T14:05:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a4a415740 10 client.0 _ll_register_callbacks cb 0x7f1a2c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a41a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6f2390 0x5646cab11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a41a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6f2390 0x5646cab11600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44328/0 (socket says 172.21.15.78:44328) 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.654+0000 7f1a41a2f640 1 -- 172.21.15.78:0/3650407380 learned_addr learned my addr 172.21.15.78:0/3650407380 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a4122e640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6c94d0 0x5646caaa1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caaa2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a41a2f640 1 -- 172.21.15.78:0/3650407380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 msgr2=0x5646caaa2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caaa2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a41a2f640 1 -- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6c94d0 msgr2=0x5646caaa1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6c94d0 0x5646caaa1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a41a2f640 1 -- 172.21.15.78:0/3650407380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ca9b94b0 con 0x5646ca6f2390 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a40a2d640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caaa2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a41a2f640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6f2390 0x5646cab11600 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1a34005f40 tx=0x7f1a34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520b88ce1e58504d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3650407380 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3402d060 con 0x5646ca6f2390 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3650407380 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1a3401d540 con 0x5646ca6f2390 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a38004660 con 0x5646ca6f2390 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3650407380 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1a3401f320 con 0x5646ca6f2390 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a4a415740 10 client.17172 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5646cab12cd0 con 0x5646ca6f2390 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.655+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3650407380 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a38004660 con 0x5646ca6f2390 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3650407380 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1a3400f040 con 0x5646ca6f2390 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4a415740 10 client.17172 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4a415740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4a415740 20 client.17172 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6863a0 0x5646cab156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4a415740 4 client.17172 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646caa00800 con 0x5646ca6863a0 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4122e640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6863a0 0x5646cab156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a24ff9640 10 client.17172 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.656+0000 7f1a4122e640 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6863a0 0x5646cab156f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4c4f0aee81a9abc server_cookie=4a7a862a3a15a66 in_seq=0 out_seq=0 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a24ff9640 1 -- 172.21.15.78:0/3650407380 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5646caa00800 con 0x5646ca6863a0 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a24ff9640 10 client.17172 handle_command_reply: tid=0 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 client.17172 shutdown 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6f2390 msgr2=0x5646cab11600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6f2390 0x5646cab11600 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1a34005f40 tx=0x7f1a34004a40 comp rx=0 tx=0).stop 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 shutdown_connections 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646ca6f2390 0x5646cab11600 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5646ca6863a0 0x5646cab156f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646ca6d16b0 0x5646caaa2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a24ff9640 0 client.17172 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 --2- 172.21.15.78:0/3650407380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5646ca6c94d0 0x5646caaa1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 >> 172.21.15.78:0/3650407380 conn(0x5646ca6e64e0 msgr2=0x5646caa8ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 shutdown_connections 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 1 -- 172.21.15.78:0/3650407380 wait complete. 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a4a415740 20 client.17172 trim_cache size 0 max 16384 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "52e6584e-d2ff-4a86-9b6b-32ac30ed5bd7": { 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "52e6584e-d2ff-4a86-9b6b-32ac30ed5bd7", 2024-04-25T14:05:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:05:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:05:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:05:13.666 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:13.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a280afaf0 msgr2=0x7f1a280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a280afaf0 0x7f1a280b2000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x5646caa94ab0 tx=0x7f1a34009740 comp rx=0 tx=0).stop 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c102260 msgr2=0x7f1a3c102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c102260 0x7f1a3c102900 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006010 tx=0x7f1a2c004340 comp rx=0 tx=0).stop 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 shutdown_connections 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a3c1050b0 0x7f1a3c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1a280afaf0 0x7f1a280b2000 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a3c102260 0x7f1a3c102900 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.657+0000 7f1a43492640 1 --2- 172.21.15.78:0/1419573844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a3c101760 0x7f1a3c104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.658+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 >> 172.21.15.78:0/1419573844 conn(0x7f1a3c0fd380 msgr2=0x7f1a3c105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.658+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 shutdown_connections 2024-04-25T14:05:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:13.658+0000 7f1a43492640 1 -- 172.21.15.78:0/1419573844 wait complete. 2024-04-25T14:05:13.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:13 smithi078 ceph-mon[29163]: pgmap v519: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:05:13.668 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:13 smithi129 ceph-mon[32569]: pgmap v519: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:05:13.669 INFO:tasks.cephfs.filesystem:scrub status for tag:52e6584e-d2ff-4a86-9b6b-32ac30ed5bd7 - {'path': '/', 'tag': '52e6584e-d2ff-4a86-9b6b-32ac30ed5bd7', 'options': 'recursive,force'} 2024-04-25T14:05:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:13 smithi112 ceph-mon[32617]: pgmap v519: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:05:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:14 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:05:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:14 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:14 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:05:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:14 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:14.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:14 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:05:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:14 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:14 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:05:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:14 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:14 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:05:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:14 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:14 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:05:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:14 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:16 smithi078 ceph-mon[29163]: pgmap v520: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:05:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:16 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:05:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:16 smithi112 ceph-mon[32617]: pgmap v520: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:05:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:16 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:05:16.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:16 smithi129 ceph-mon[32569]: pgmap v520: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:05:16.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:16 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:05:18.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:18 smithi078 ceph-mon[29163]: pgmap v521: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:05:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:18 smithi112 ceph-mon[32617]: pgmap v521: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:05:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:18 smithi129 ceph-mon[32569]: pgmap v521: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:05:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:20 smithi078 ceph-mon[29163]: pgmap v522: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:20.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:20 smithi112 ceph-mon[32617]: pgmap v522: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:20 smithi129 ceph-mon[32569]: pgmap v522: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:22 smithi078 ceph-mon[29163]: pgmap v523: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:22.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:22 smithi112 ceph-mon[32617]: pgmap v523: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:22 smithi129 ceph-mon[32569]: pgmap v523: 97 pgs: 97 active+clean; 4.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:24 smithi078 ceph-mon[29163]: pgmap v524: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:05:24.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:24 smithi112 ceph-mon[32617]: pgmap v524: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:05:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:24 smithi129 ceph-mon[32569]: pgmap v524: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:05:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 -- 172.21.15.78:0/3734789395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248102a10 msgr2=0x7fc248103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 --2- 172.21.15.78:0/3734789395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248102a10 0x7fc248103710 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc238009900 tx=0x7fc23802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 -- 172.21.15.78:0/3734789395 shutdown_connections 2024-04-25T14:05:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 --2- 172.21.15.78:0/3734789395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc248108950 0x7fc24810ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 --2- 172.21.15.78:0/3734789395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc248103c50 0x7fc248104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 --2- 172.21.15.78:0/3734789395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248102a10 0x7fc248103710 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 -- 172.21.15.78:0/3734789395 >> 172.21.15.78:0/3734789395 conn(0x7fc2480fdc00 msgr2=0x7fc2481000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 -- 172.21.15.78:0/3734789395 shutdown_connections 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.765+0000 7fc24cf84640 1 -- 172.21.15.78:0/3734789395 wait complete. 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 Processor -- start 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 -- start start 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc248102a10 0x7fc24819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc248103c50 0x7fc24819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248108950 0x7fc2481a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc24810f640 con 0x7fc248108950 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc24810f4c0 con 0x7fc248102a10 2024-04-25T14:05:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.766+0000 7fc24cf84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc24810f7c0 con 0x7fc248103c50 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248108950 0x7fc2481a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248108950 0x7fc2481a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51698/0 (socket says 172.21.15.78:51698) 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 -- 172.21.15.78:0/1777922498 learned_addr learned my addr 172.21.15.78:0/1777922498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc23ffff640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc248103c50 0x7fc24819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc24659c640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc248102a10 0x7fc24819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 -- 172.21.15.78:0/1777922498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc248103c50 msgr2=0x7fc24819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc248103c50 0x7fc24819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 -- 172.21.15.78:0/1777922498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc248102a10 msgr2=0x7fc24819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc248102a10 0x7fc24819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 -- 172.21.15.78:0/1777922498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2481a1d70 con 0x7fc248108950 2024-04-25T14:05:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc23ffff640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc248103c50 0x7fc24819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc246d9d640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248108950 0x7fc2481a1700 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc24000eb10 tx=0x7fc24000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc23f7fe640 1 -- 172.21.15.78:0/1777922498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240004b20 con 0x7fc248108950 2024-04-25T14:05:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc23f7fe640 1 -- 172.21.15.78:0/1777922498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc240004cc0 con 0x7fc248108950 2024-04-25T14:05:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc23f7fe640 1 -- 172.21.15.78:0/1777922498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240010670 con 0x7fc248108950 2024-04-25T14:05:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2481a2060 con 0x7fc248108950 2024-04-25T14:05:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.767+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2481a2560 con 0x7fc248108950 2024-04-25T14:05:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 Processor -- start 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc23f7fe640 1 -- 172.21.15.78:0/1777922498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc240005020 con 0x7fc248108950 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 -- start start 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533d6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc53330460 con 0x55cc530094d0 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc532ea590 con 0x55cc530012f0 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc253e38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cc5332c420 con 0x55cc5302a1b0 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.770+0000 7fc23ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533d6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51704/0 (socket says 172.21.15.78:51704) 2024-04-25T14:05:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 -- 172.21.15.78:0/3807850850 learned_addr learned my addr 172.21.15.78:0/3807850850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc24659c640 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23f7fe640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc2340afae0 0x7fc2340b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc246d9d640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc2340afae0 0x7fc2340b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 -- 172.21.15.78:0/3807850850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 msgr2=0x55cc533ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 -- 172.21.15.78:0/3807850850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 msgr2=0x55cc533cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 -- 172.21.15.78:0/3807850850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc532f70d0 con 0x55cc530094d0 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23f7fe640 1 -- 172.21.15.78:0/1777922498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc2400df310 con 0x7fc248108950 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23ffff640 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533d6200 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc23000bfd0 tx=0x7fc23000b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d792838a49b3bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23cff9640 1 -- 172.21.15.78:0/3807850850 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230004030 con 0x55cc530094d0 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc246d9d640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc2340afae0 0x7fc2340b1ff0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2400a3860 tx=0x7fc240076330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23cff9640 1 -- 172.21.15.78:0/3807850850 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2300041d0 con 0x55cc530094d0 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.771+0000 7fc23cff9640 1 -- 172.21.15.78:0/3807850850 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230003850 con 0x55cc530094d0 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 -- 172.21.15.78:0/3807850850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 msgr2=0x55cc533d6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533d6200 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc23000bfd0 tx=0x7fc23000b590 comp rx=0 tx=0).stop 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 -- 172.21.15.78:0/3807850850 shutdown_connections 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533d6200 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 --2- 172.21.15.78:0/3807850850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 -- 172.21.15.78:0/3807850850 >> 172.21.15.78:0/3807850850 conn(0x55cc5301e300 msgr2=0x55cc533c6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 -- 172.21.15.78:0/3807850850 shutdown_connections 2024-04-25T14:05:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.772+0000 7fc253e38740 1 -- 172.21.15.78:0/3807850850 wait complete. 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 -- start start 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 Processor -- start 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530012f0 0x55cc533d8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc530094d0 0x55cc533d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc5302a1b0 0x55cc533dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc533d6bf0 con 0x55cc530012f0 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc532b7b60 con 0x55cc5302a1b0 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cc533d6d70 con 0x55cc530094d0 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc253e38740 10 client.0 _ll_register_callbacks cb 0x55cc532dc2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530012f0 0x55cc533d8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530012f0 0x55cc533d8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51718/0 (socket says 172.21.15.78:51718) 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 -- 172.21.15.78:0/2829561552 learned_addr learned my addr 172.21.15.78:0/2829561552 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc24659c640 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc530094d0 0x55cc533d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc5302a1b0 0x55cc533dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 -- 172.21.15.78:0/2829561552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc530094d0 msgr2=0x55cc533d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc530094d0 0x55cc533d8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 -- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc5302a1b0 msgr2=0x55cc533dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc5302a1b0 0x55cc533dfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.773+0000 7fc23ffff640 1 -- 172.21.15.78:0/2829561552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc53042210 con 0x55cc530012f0 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530012f0 0x55cc533d8a30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc230004430 tx=0x7fc23000b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3836541e8cc0a946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc22dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc22dffb640 1 -- 172.21.15.78:0/2829561552 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000f8d0 con 0x55cc530012f0 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc22dffb640 1 -- 172.21.15.78:0/2829561552 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc230005030 con 0x55cc530012f0 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2300041d0 con 0x55cc530012f0 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc22dffb640 1 -- 172.21.15.78:0/2829561552 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc23000fa70 con 0x55cc530012f0 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 10 client.17184 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cc5310e620 con 0x55cc530012f0 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc22dffb640 1 -- 172.21.15.78:0/2829561552 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2300041d0 con 0x55cc530012f0 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc22dffb640 1 -- 172.21.15.78:0/2829561552 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc230004c60 con 0x55cc530012f0 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 10 client.17184 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 20 client.17184 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fe8d80 0x55cc533e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 4 client.17184 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.774+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc5333e070 con 0x55cc52fe8d80 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.775+0000 7fc24659c640 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fe8d80 0x55cc533e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.775+0000 7fc22dffb640 10 client.17184 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.775+0000 7fc24659c640 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fe8d80 0x55cc533e63c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6ab5d7bbeae4c server_cookie=5989ac2e54e8f362 in_seq=0 out_seq=0 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc22dffb640 1 -- 172.21.15.78:0/2829561552 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc5333e070 con 0x55cc52fe8d80 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc22dffb640 10 client.17184 handle_command_reply: tid=0 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 client.17184 shutdown 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530012f0 msgr2=0x55cc533d8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530012f0 0x55cc533d8a30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc230004430 tx=0x7fc23000b2c0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 shutdown_connections 2024-04-25T14:05:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc5302a1b0 0x55cc533dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fe8d80 0x55cc533e63c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc530094d0 0x55cc533d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 --2- 172.21.15.78:0/2829561552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530012f0 0x55cc533d8a30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc22dffb640 0 client.17184 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 >> 172.21.15.78:0/2829561552 conn(0x55cc5301e300 msgr2=0x55cc533c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 shutdown_connections 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 1 -- 172.21.15.78:0/2829561552 wait complete. 2024-04-25T14:05:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.776+0000 7fc253e38740 20 client.17184 trim_cache size 0 max 16384 2024-04-25T14:05:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 Processor -- start 2024-04-25T14:05:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 -- start start 2024-04-25T14:05:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533c6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481ab9f0 con 0x55cc530094d0 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481a92c0 con 0x55cc530012f0 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc253e38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481a7110 con 0x55cc5302a1b0 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc24659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc24659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51726/0 (socket says 172.21.15.78:51726) 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.791+0000 7fc24659c640 1 -- 172.21.15.78:0/2463629703 learned_addr learned my addr 172.21.15.78:0/2463629703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc24659c640 1 -- 172.21.15.78:0/2463629703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 msgr2=0x55cc533c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc24659c640 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc24659c640 1 -- 172.21.15.78:0/2463629703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 msgr2=0x55cc533d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc24659c640 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc24659c640 1 -- 172.21.15.78:0/2463629703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc533cf730 con 0x55cc530094d0 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc24659c640 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dfdf0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc238009230 tx=0x7fc2380314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3375f54cbca00758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc22ffff640 1 -- 172.21.15.78:0/2463629703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23804a070 con 0x55cc530094d0 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc22ffff640 1 -- 172.21.15.78:0/2463629703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2380054f0 con 0x55cc530094d0 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.792+0000 7fc22ffff640 1 -- 172.21.15.78:0/2463629703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238035040 con 0x55cc530094d0 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 -- 172.21.15.78:0/2463629703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 msgr2=0x55cc533dfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dfdf0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc238009230 tx=0x7fc2380314a0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 -- 172.21.15.78:0/2463629703 shutdown_connections 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dfdf0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 --2- 172.21.15.78:0/2463629703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 -- 172.21.15.78:0/2463629703 >> 172.21.15.78:0/2463629703 conn(0x55cc5301e300 msgr2=0x55cc533c5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 -- 172.21.15.78:0/2463629703 shutdown_connections 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 -- 172.21.15.78:0/2463629703 wait complete. 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 -- start start 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 Processor -- start 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533de550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.793+0000 7fc253e38740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481a7780 con 0x55cc530094d0 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc253e38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481a8bf0 con 0x55cc530012f0 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc253e38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481a7e50 con 0x55cc5302a1b0 2024-04-25T14:05:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc253e38740 10 client.0 _ll_register_callbacks cb 0x7fc2400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51728/0 (socket says 172.21.15.78:51728) 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 -- 172.21.15.78:0/2457799945 learned_addr learned my addr 172.21.15.78:0/2457799945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 -- 172.21.15.78:0/2457799945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 msgr2=0x55cc533de550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533de550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 -- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 msgr2=0x55cc533dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533dd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 -- 172.21.15.78:0/2457799945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc533deb10 con 0x55cc530094d0 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533de550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc24659c640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dd620 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc238032fd0 tx=0x7fc238032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a619b5eb2b9fa2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc22dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc22dffb640 1 -- 172.21.15.78:0/2457799945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238053060 con 0x55cc530094d0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc22dffb640 1 -- 172.21.15.78:0/2457799945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc238004380 con 0x55cc530094d0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2380054f0 con 0x55cc530094d0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc22dffb640 1 -- 172.21.15.78:0/2457799945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc23804a040 con 0x55cc530094d0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc253e38740 10 client.17190 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.794+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cc533dfd40 con 0x55cc530094d0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc22dffb640 1 -- 172.21.15.78:0/2457799945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2380054f0 con 0x55cc530094d0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc22dffb640 1 -- 172.21.15.78:0/2457799945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc238003990 con 0x55cc530094d0 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc253e38740 10 client.17190 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc253e38740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc253e38740 20 client.17190 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc253e38740 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fbe1c0 0x55cc534509c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc253e38740 4 client.17190 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc53337ee0 con 0x55cc52fbe1c0 2024-04-25T14:05:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fbe1c0 0x55cc534509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc22dffb640 10 client.17190 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.795+0000 7fc23ffff640 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fbe1c0 0x55cc534509c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390615fe9904969 server_cookie=28fc86ab91e95d6a in_seq=0 out_seq=0 2024-04-25T14:05:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc22dffb640 1 -- 172.21.15.78:0/2457799945 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc53337ee0 con 0x55cc52fbe1c0 2024-04-25T14:05:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc22dffb640 10 client.17190 handle_command_reply: tid=0 2024-04-25T14:05:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 client.17190 shutdown 2024-04-25T14:05:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 msgr2=0x55cc533dd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dd620 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc238032fd0 tx=0x7fc238032e20 comp rx=0 tx=0).stop 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 shutdown_connections 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cc52fbe1c0 0x55cc534509c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc5302a1b0 0x55cc533de550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc530094d0 0x55cc533dd620 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc22dffb640 0 client.17190 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 --2- 172.21.15.78:0/2457799945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc530012f0 0x55cc533dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 >> 172.21.15.78:0/2457799945 conn(0x55cc5301e300 msgr2=0x55cc533d8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 shutdown_connections 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 1 -- 172.21.15.78:0/2457799945 wait complete. 2024-04-25T14:05:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.811+0000 7fc253e38740 20 client.17190 trim_cache size 0 max 16384 2024-04-25T14:05:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:05:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:05:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:05:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:05:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc2340afae0 msgr2=0x7fc2340b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc2340afae0 0x7fc2340b1ff0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2400a3860 tx=0x7fc240076330 comp rx=0 tx=0).stop 2024-04-25T14:05:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248108950 msgr2=0x7fc2481a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248108950 0x7fc2481a1700 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc24000eb10 tx=0x7fc24000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 shutdown_connections 2024-04-25T14:05:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc2340afae0 0x7fc2340b1ff0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc248108950 0x7fc2481a1700 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc248103c50 0x7fc24819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 --2- 172.21.15.78:0/1777922498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc248102a10 0x7fc24819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 >> 172.21.15.78:0/1777922498 conn(0x7fc2480fdc00 msgr2=0x7fc248104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 shutdown_connections 2024-04-25T14:05:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:25.812+0000 7fc24cf84640 1 -- 172.21.15.78:0/1777922498 wait complete. 2024-04-25T14:05:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:26 smithi078 ceph-mon[29163]: pgmap v525: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:26.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:26 smithi112 ceph-mon[32617]: pgmap v525: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:26.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:26 smithi129 ceph-mon[32569]: pgmap v525: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:28 smithi078 ceph-mon[29163]: pgmap v526: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:05:28.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:28 smithi112 ceph-mon[32617]: pgmap v526: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:05:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:28 smithi129 ceph-mon[32569]: pgmap v526: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:05:30.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:30 smithi078 ceph-mon[29163]: pgmap v527: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:30 smithi112 ceph-mon[32617]: pgmap v527: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:30 smithi129 ceph-mon[32569]: pgmap v527: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:05:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:32 smithi078 ceph-mon[29163]: pgmap v528: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:05:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:32 smithi112 ceph-mon[32617]: pgmap v528: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:05:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:32 smithi129 ceph-mon[32569]: pgmap v528: 97 pgs: 97 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:05:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:34 smithi078 ceph-mon[29163]: pgmap v529: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:05:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:34 smithi112 ceph-mon[32617]: pgmap v529: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:05:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:34 smithi129 ceph-mon[32569]: pgmap v529: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:05:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:36 smithi078 ceph-mon[29163]: pgmap v530: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:36.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:36 smithi112 ceph-mon[32617]: pgmap v530: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:36 smithi129 ceph-mon[32569]: pgmap v530: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:05:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:38 smithi078 ceph-mon[29163]: pgmap v531: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:05:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:38 smithi112 ceph-mon[32617]: pgmap v531: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:05:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:38 smithi129 ceph-mon[32569]: pgmap v531: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:05:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 -- 172.21.15.78:0/3242215702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 msgr2=0x7f9634104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 --2- 172.21.15.78:0/3242215702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f9634104850 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9624009900 tx=0x7f962402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 -- 172.21.15.78:0/3242215702 shutdown_connections 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 --2- 172.21.15.78:0/3242215702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9634109a90 0x7f963410beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 --2- 172.21.15.78:0/3242215702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634104d90 0x7f96341051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 --2- 172.21.15.78:0/3242215702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f9634104850 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 -- 172.21.15.78:0/3242215702 >> 172.21.15.78:0/3242215702 conn(0x7f96340fed20 msgr2=0x7f96341011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 -- 172.21.15.78:0/3242215702 shutdown_connections 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.920+0000 7f963a60d640 1 -- 172.21.15.78:0/3242215702 wait complete. 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 Processor -- start 2024-04-25T14:05:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 -- start start 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f963419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9634104d90 0x7f963419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634109a90 0x7f96341a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f963410f650 con 0x7f9634103b50 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f963410f4d0 con 0x7f9634109a90 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f963a60d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f963410f7d0 con 0x7f9634104d90 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f9633fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f963419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f9633fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f963419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55170/0 (socket says 172.21.15.78:55170) 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f9633fff640 1 -- 172.21.15.78:0/3488147363 learned_addr learned my addr 172.21.15.78:0/3488147363 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f96337fe640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9634104d90 0x7f963419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.921+0000 7f9638baa640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634109a90 0x7f96341a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f9633fff640 1 -- 172.21.15.78:0/3488147363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9634104d90 msgr2=0x7f963419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f9633fff640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9634104d90 0x7f963419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f9633fff640 1 -- 172.21.15.78:0/3488147363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634109a90 msgr2=0x7f96341a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f9633fff640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634109a90 0x7f96341a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f9633fff640 1 -- 172.21.15.78:0/3488147363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96341a1d90 con 0x7f9634103b50 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f9633fff640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f963419a250 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9624006010 tx=0x7f9624004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f96317fa640 1 -- 172.21.15.78:0/3488147363 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962403f030 con 0x7f9634103b50 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f96317fa640 1 -- 172.21.15.78:0/3488147363 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9624003aa0 con 0x7f9634103b50 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96341a2020 con 0x7f9634103b50 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f96317fa640 1 -- 172.21.15.78:0/3488147363 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624035070 con 0x7f9634103b50 2024-04-25T14:05:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.922+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96341a2510 con 0x7f9634103b50 2024-04-25T14:05:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.924+0000 7f96317fa640 1 -- 172.21.15.78:0/3488147363 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9624042030 con 0x7f9634103b50 2024-04-25T14:05:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.924+0000 7f9641595740 1 Processor -- start 2024-04-25T14:05:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9641595740 1 -- start start 2024-04-25T14:05:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9641595740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328be970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9641595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328beef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9641595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9641595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558732824460 con 0x5587324fd4d0 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9641595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587327de590 con 0x5587324f52f0 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9641595740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558732820420 con 0x55873251e1b0 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55180/0 (socket says 172.21.15.78:55180) 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 -- 172.21.15.78:0/2585858799 learned_addr learned my addr 172.21.15.78:0/2585858799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9638baa640 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9633fff640 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328beef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 -- 172.21.15.78:0/2585858799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 msgr2=0x5587328be970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96317fa640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f961c0afae0 0x7f961c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328be970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 -- 172.21.15.78:0/2585858799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 msgr2=0x5587328beef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328beef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f96337fe640 1 -- 172.21.15.78:0/2585858799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587327eb0d0 con 0x5587324fd4d0 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.925+0000 7f9638baa640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f961c0afae0 0x7f961c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.926+0000 7f96317fa640 1 -- 172.21.15.78:0/3488147363 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f96240ff260 con 0x7f9634103b50 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.926+0000 7f96337fe640 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328c5df0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9628009c90 tx=0x7f9628011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10312485d3e572ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.926+0000 7f961a7fc640 1 -- 172.21.15.78:0/2585858799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628005220 con 0x5587324fd4d0 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.926+0000 7f961a7fc640 1 -- 172.21.15.78:0/2585858799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9628004c50 con 0x5587324fd4d0 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.926+0000 7f961a7fc640 1 -- 172.21.15.78:0/2585858799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628019690 con 0x5587324fd4d0 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 -- 172.21.15.78:0/2585858799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 msgr2=0x5587328c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9638baa640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f961c0afae0 0x7f961c0b1ff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5587328bfb40 tx=0x7f962c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328c5df0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9628009c90 tx=0x7f9628011810 comp rx=0 tx=0).stop 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 -- 172.21.15.78:0/2585858799 shutdown_connections 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328c5df0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328beef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 --2- 172.21.15.78:0/2585858799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 -- 172.21.15.78:0/2585858799 >> 172.21.15.78:0/2585858799 conn(0x558732512300 msgr2=0x5587328ba300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 -- 172.21.15.78:0/2585858799 shutdown_connections 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 -- 172.21.15.78:0/2585858799 wait complete. 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 -- start start 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.927+0000 7f9641595740 1 Processor -- start 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324fd4d0 0x5587328c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55873251e1b0 0x5587328cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5587328c6840 con 0x55873251e1b0 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587327abb60 con 0x5587324fd4d0 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5587328c69c0 con 0x5587324f52f0 2024-04-25T14:05:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 10 client.0 _ll_register_callbacks cb 0x5587327d02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55873251e1b0 0x5587328cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55873251e1b0 0x5587328cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55186/0 (socket says 172.21.15.78:55186) 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 -- 172.21.15.78:0/3511515520 learned_addr learned my addr 172.21.15.78:0/3511515520 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f96337fe640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324fd4d0 0x5587328c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9633fff640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 -- 172.21.15.78:0/3511515520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 msgr2=0x5587328c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 -- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324fd4d0 msgr2=0x5587328c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324fd4d0 0x5587328c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 -- 172.21.15.78:0/3511515520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558732536210 con 0x55873251e1b0 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f96337fe640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324fd4d0 0x5587328c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9638baa640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55873251e1b0 0x5587328cfab0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f962c013890 tx=0x7f962c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f28c8d4e0c713f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f96137fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f96137fe640 1 -- 172.21.15.78:0/3511515520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c01ed70 con 0x55873251e1b0 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f96137fe640 1 -- 172.21.15.78:0/3511515520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f962c028470 con 0x55873251e1b0 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.928+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9628004c50 con 0x55873251e1b0 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f96137fe640 1 -- 172.21.15.78:0/3511515520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f962c004b20 con 0x55873251e1b0 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 10 client.17202 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558732602620 con 0x55873251e1b0 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f96137fe640 1 -- 172.21.15.78:0/3511515520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9628004c50 con 0x55873251e1b0 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f96137fe640 1 -- 172.21.15.78:0/3511515520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f962c00f040 con 0x55873251e1b0 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 10 client.17202 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 20 client.17202 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324dcd80 0x5587328d6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 4 client.17202 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558732832070 con 0x5587324dcd80 2024-04-25T14:05:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.929+0000 7f9633fff640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324dcd80 0x5587328d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.930+0000 7f9633fff640 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324dcd80 0x5587328d6450 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ae8d6aaed523bb server_cookie=fcd1c393ebbb1a3f in_seq=0 out_seq=0 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.930+0000 7f96137fe640 10 client.17202 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f96137fe640 1 -- 172.21.15.78:0/3511515520 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558732832070 con 0x5587324dcd80 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f96137fe640 10 client.17202 handle_command_reply: tid=0 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 client.17202 shutdown 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55873251e1b0 msgr2=0x5587328cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55873251e1b0 0x5587328cfab0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f962c013890 tx=0x7f962c004ab0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 shutdown_connections 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55873251e1b0 0x5587328cfab0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324fd4d0 0x5587328c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324dcd80 0x5587328d6450 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 --2- 172.21.15.78:0/3511515520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f96137fe640 0 client.17202 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 >> 172.21.15.78:0/3511515520 conn(0x558732512300 msgr2=0x5587328bd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 shutdown_connections 2024-04-25T14:05:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 1 -- 172.21.15.78:0/3511515520 wait complete. 2024-04-25T14:05:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.931+0000 7f9641595740 20 client.17202 trim_cache size 0 max 16384 2024-04-25T14:05:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 Processor -- start 2024-04-25T14:05:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 -- start start 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55873251e1b0 0x5587328bd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96341adbc0 con 0x5587324fd4d0 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96341ab490 con 0x55873251e1b0 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9641595740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96341a92e0 con 0x5587324f52f0 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55190/0 (socket says 172.21.15.78:55190) 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 -- 172.21.15.78:0/652803108 learned_addr learned my addr 172.21.15.78:0/652803108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9633fff640 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9638baa640 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55873251e1b0 0x5587328bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 -- 172.21.15.78:0/652803108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 msgr2=0x5587328b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 -- 172.21.15.78:0/652803108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55873251e1b0 msgr2=0x5587328bd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55873251e1b0 0x5587328bd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f96337fe640 1 -- 172.21.15.78:0/652803108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587328c9210 con 0x5587324fd4d0 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.946+0000 7f9638baa640 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55873251e1b0 0x5587328bd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f96337fe640 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ba110 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f962801b040 tx=0x7f9628009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9431b00743731c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9619ffb640 1 -- 172.21.15.78:0/652803108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628005240 con 0x5587324fd4d0 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9619ffb640 1 -- 172.21.15.78:0/652803108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96280053e0 con 0x5587324fd4d0 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9619ffb640 1 -- 172.21.15.78:0/652803108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96280236b0 con 0x5587324fd4d0 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9641595740 1 -- 172.21.15.78:0/652803108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 msgr2=0x5587328ba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9641595740 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ba110 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f962801b040 tx=0x7f9628009930 comp rx=0 tx=0).stop 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9641595740 1 -- 172.21.15.78:0/652803108 shutdown_connections 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9641595740 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55873251e1b0 0x5587328bd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9641595740 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ba110 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9641595740 1 --2- 172.21.15.78:0/652803108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587324f52f0 0x5587328b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.947+0000 7f9641595740 1 -- 172.21.15.78:0/652803108 >> 172.21.15.78:0/652803108 conn(0x558732512300 msgr2=0x5587328c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 -- 172.21.15.78:0/652803108 shutdown_connections 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 -- 172.21.15.78:0/652803108 wait complete. 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 -- start start 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 Processor -- start 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96341a9950 con 0x5587324fd4d0 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96341aadc0 con 0x5587324f52f0 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96341aa020 con 0x55873251e1b0 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f9641595740 10 client.0 _ll_register_callbacks cb 0x7f962c009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f96337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f96337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55206/0 (socket says 172.21.15.78:55206) 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.948+0000 7f96337fe640 1 -- 172.21.15.78:0/4235336963 learned_addr learned my addr 172.21.15.78:0/4235336963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f9638baa640 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f9633fff640 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96337fe640 1 -- 172.21.15.78:0/4235336963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 msgr2=0x5587328cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96337fe640 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96337fe640 1 -- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 msgr2=0x5587328cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96337fe640 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96337fe640 1 -- 172.21.15.78:0/4235336963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587328ce0c0 con 0x5587324fd4d0 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96337fe640 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ccc40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9628011850 tx=0x7f9628005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6fb4b477c5fad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96137fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96137fe640 1 -- 172.21.15.78:0/4235336963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628035070 con 0x5587324fd4d0 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96137fe640 1 -- 172.21.15.78:0/4235336963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9628004a30 con 0x5587324fd4d0 2024-04-25T14:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96280053e0 con 0x5587324fd4d0 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96137fe640 1 -- 172.21.15.78:0/4235336963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9628041050 con 0x5587324fd4d0 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f9641595740 10 client.17208 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5587328cf2c0 con 0x5587324fd4d0 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.949+0000 7f96137fe640 1 -- 172.21.15.78:0/4235336963 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96280053e0 con 0x5587324fd4d0 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f96137fe640 1 -- 172.21.15.78:0/4235336963 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f96280194c0 con 0x5587324fd4d0 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9641595740 10 client.17208 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9641595740 10 client.17208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9641595740 20 client.17208 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9641595740 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324b21c0 0x558732940cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9641595740 4 client.17208 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55873282bee0 con 0x5587324b21c0 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9633fff640 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324b21c0 0x558732940cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f96137fe640 10 client.17208 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.950+0000 7f9633fff640 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324b21c0 0x558732940cd0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343b823b0cd88eee server_cookie=1efc8b1d3030b077 in_seq=0 out_seq=0 2024-04-25T14:05:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f96137fe640 1 -- 172.21.15.78:0/4235336963 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55873282bee0 con 0x5587324b21c0 2024-04-25T14:05:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f96137fe640 10 client.17208 handle_command_reply: tid=0 2024-04-25T14:05:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 client.17208 shutdown 2024-04-25T14:05:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 msgr2=0x5587328ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ccc40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9628011850 tx=0x7f9628005fa0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 shutdown_connections 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5587324b21c0 0x558732940cd0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55873251e1b0 0x5587328cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587324fd4d0 0x5587328ccc40 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 --2- 172.21.15.78:0/4235336963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587324f52f0 0x5587328cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f96137fe640 0 client.17208 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 >> 172.21.15.78:0/4235336963 conn(0x558732512300 msgr2=0x5587328bd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 shutdown_connections 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 1 -- 172.21.15.78:0/4235336963 wait complete. 2024-04-25T14:05:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f9641595740 20 client.17208 trim_cache size 0 max 16384 2024-04-25T14:05:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:05:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:05:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:05:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f961c0afae0 msgr2=0x7f961c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f963a60d640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f961c0afae0 0x7f961c0b1ff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5587328bfb40 tx=0x7f962c004a40 comp rx=0 tx=0).stop 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 msgr2=0x7f963419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.966+0000 7f963a60d640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f963419a250 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9624006010 tx=0x7f9624004270 comp rx=0 tx=0).stop 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 shutdown_connections 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634109a90 0x7f96341a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9634104d90 0x7f963419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f961c0afae0 0x7f961c0b1ff0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 --2- 172.21.15.78:0/3488147363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9634103b50 0x7f963419a250 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 >> 172.21.15.78:0/3488147363 conn(0x7f96340fed20 msgr2=0x7f9634105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 shutdown_connections 2024-04-25T14:05:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:38.967+0000 7f963a60d640 1 -- 172.21.15.78:0/3488147363 wait complete. 2024-04-25T14:05:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:39 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:39 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:39 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: pgmap v532: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: pgmap v532: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:05:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: pgmap v532: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:05:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:05:40.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:05:40.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:05:40.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:05:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:41 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-25T14:05:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:41 smithi112 ceph-mon[32617]: Unable to set osd_memory_target on smithi078 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T14:05:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:41 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-25T14:05:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:41 smithi129 ceph-mon[32569]: Unable to set osd_memory_target on smithi078 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T14:05:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:41 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi078 to 176.2M 2024-04-25T14:05:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:41 smithi078 ceph-mon[29163]: Unable to set osd_memory_target on smithi078 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-04-25T14:05:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:42 smithi112 ceph-mon[32617]: pgmap v533: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:05:42.564 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:42 smithi129 ceph-mon[32569]: pgmap v533: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:05:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:42 smithi078 ceph-mon[29163]: pgmap v533: 97 pgs: 97 active+clean; 4.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:05:43.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:43.670 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3286276480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81050b0 msgr2=0x7f40b81054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3286276480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81050b0 0x7f40b81054c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3286276480 shutdown_connections 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3286276480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b8102260 0x7f40b810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3286276480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b8101760 0x7f40b8101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3286276480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81050b0 0x7f40b81054c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3286276480 >> 172.21.15.78:0/3286276480 conn(0x7f40b80fd380 msgr2=0x7f40b80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3286276480 shutdown_connections 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.790+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3286276480 wait complete. 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 Processor -- start 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 -- start start 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b8101760 0x7f40b819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b8102260 0x7f40b819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b81050b0 0x7f40b81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40b810f600 con 0x7f40b8102260 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b810f480 con 0x7f40b81050b0 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bf8d8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b810f780 con 0x7f40b8101760 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bce73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b8102260 0x7f40b819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bce73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b8102260 0x7f40b819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55224/0 (socket says 172.21.15.78:55224) 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bce73640 1 -- 172.21.15.78:0/3061419096 learned_addr learned my addr 172.21.15.78:0/3061419096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bde75640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b81050b0 0x7f40b81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.791+0000 7f40bd674640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b8101760 0x7f40b819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bce73640 1 -- 172.21.15.78:0/3061419096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b8101760 msgr2=0x7f40b819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bce73640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b8101760 0x7f40b819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bce73640 1 -- 172.21.15.78:0/3061419096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b81050b0 msgr2=0x7f40b81a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bce73640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b81050b0 0x7f40b81a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bce73640 1 -- 172.21.15.78:0/3061419096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b81a1d90 con 0x7f40b8102260 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bd674640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b8101760 0x7f40b819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bce73640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b8102260 0x7f40b819a790 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f40b400d950 tx=0x7f40b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40ae7fc640 1 -- 172.21.15.78:0/3061419096 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4017070 con 0x7f40b8102260 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40ae7fc640 1 -- 172.21.15.78:0/3061419096 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b4009c30 con 0x7f40b8102260 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b81a2080 con 0x7f40b8102260 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40ae7fc640 1 -- 172.21.15.78:0/3061419096 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b400f620 con 0x7f40b8102260 2024-04-25T14:05:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.792+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a25d0 con 0x7f40b8102260 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 Processor -- start 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 -- start start 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40ae7fc640 1 -- 172.21.15.78:0/3061419096 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f40b4010460 con 0x7f40b8102260 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fe6603bd80 con 0x55fe65d35390 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fe65ff58b0 con 0x55fe65d0c4d0 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40c6783740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe66037d40 con 0x55fe65d146b0 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660d6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55238/0 (socket says 172.21.15.78:55238) 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 -- 172.21.15.78:0/2152831643 learned_addr learned my addr 172.21.15.78:0/2152831643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bd674640 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bce73640 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 -- 172.21.15.78:0/2152831643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 msgr2=0x55fe660d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 -- 172.21.15.78:0/2152831643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 msgr2=0x55fe660d6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660d6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40bde75640 1 -- 172.21.15.78:0/2152831643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe65fc6a40 con 0x55fe65d35390 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.795+0000 7f40ae7fc640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a40afaf0 0x7f40a40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.796+0000 7f40bd674640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a40afaf0 0x7f40a40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.796+0000 7f40bde75640 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660d6a60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f40b0010370 tx=0x7f40b0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6e27212be093b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.796+0000 7f40ae7fc640 1 -- 172.21.15.78:0/3061419096 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f40b40de8a0 con 0x7f40b8102260 2024-04-25T14:05:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.796+0000 7f40a37fe640 1 -- 172.21.15.78:0/2152831643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0016070 con 0x55fe65d35390 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.796+0000 7f40a37fe640 1 -- 172.21.15.78:0/2152831643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0004d20 con 0x55fe65d35390 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.796+0000 7f40a37fe640 1 -- 172.21.15.78:0/2152831643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0005420 con 0x55fe65d35390 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 -- 172.21.15.78:0/2152831643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 msgr2=0x55fe660d6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40bd674640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a40afaf0 0x7f40a40b2000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009230 tx=0x7f40a8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660d6a60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f40b0010370 tx=0x7f40b0010830 comp rx=0 tx=0).stop 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 -- 172.21.15.78:0/2152831643 shutdown_connections 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660d6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 --2- 172.21.15.78:0/2152831643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660d6a60 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 -- 172.21.15.78:0/2152831643 >> 172.21.15.78:0/2152831643 conn(0x55fe65d294e0 msgr2=0x55fe660d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 -- 172.21.15.78:0/2152831643 shutdown_connections 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 -- 172.21.15.78:0/2152831643 wait complete. 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 -- start start 2024-04-25T14:05:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.797+0000 7f40c6783740 1 Processor -- start 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fe660d8780 con 0x55fe65d35390 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fe65fc2d10 con 0x55fe65d0c4d0 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe660d8900 con 0x55fe65d146b0 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 10 client.0 _ll_register_callbacks cb 0x55fe65fef210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660e7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55254/0 (socket says 172.21.15.78:55254) 2024-04-25T14:05:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 -- 172.21.15.78:0/1566518685 learned_addr learned my addr 172.21.15.78:0/1566518685 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bce73640 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bd674640 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 -- 172.21.15.78:0/1566518685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 msgr2=0x55fe660e0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660e0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 -- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 msgr2=0x55fe660e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 -- 172.21.15.78:0/1566518685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe65e177d0 con 0x55fe65d35390 2024-04-25T14:05:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40bde75640 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660e7ec0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006010 tx=0x7f40b0009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad3ef1c74a079f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1566518685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0016070 con 0x55fe65d35390 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1566518685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f40b0012510 con 0x55fe65d35390 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.798+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0004d20 con 0x55fe65d35390 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1566518685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f40b0012710 con 0x55fe65d35390 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 10 client.17220 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fe65d4d3f0 con 0x55fe65d35390 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1566518685 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0004d20 con 0x55fe65d35390 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1566518685 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f40b000f040 con 0x55fe65d35390 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 10 client.17220 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 20 client.17220 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cf3f60 0x55fe660ee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 4 client.17220 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe66049990 con 0x55fe65cf3f60 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.799+0000 7f40bd674640 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cf3f60 0x55fe660ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.800+0000 7f40a0ff9640 10 client.17220 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.800+0000 7f40bd674640 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cf3f60 0x55fe660ee030 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f7a42c3c9a4eb4 server_cookie=7b65f3a7d7292d62 in_seq=0 out_seq=0 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1566518685 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe66049990 con 0x55fe65cf3f60 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40a0ff9640 10 client.17220 handle_command_reply: tid=0 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 client.17220 shutdown 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 msgr2=0x55fe660e7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660e7ec0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006010 tx=0x7f40b0009cc0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 shutdown_connections 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d35390 0x55fe660e7ec0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cf3f60 0x55fe660ee030 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d146b0 0x55fe660e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 --2- 172.21.15.78:0/1566518685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d0c4d0 0x55fe660e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40a0ff9640 0 client.17220 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 >> 172.21.15.78:0/1566518685 conn(0x55fe65d294e0 msgr2=0x55fe660d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 shutdown_connections 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 1 -- 172.21.15.78:0/1566518685 wait complete. 2024-04-25T14:05:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.801+0000 7f40c6783740 20 client.17220 trim_cache size 0 max 16384 2024-04-25T14:05:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 Processor -- start 2024-04-25T14:05:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 -- start start 2024-04-25T14:05:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d0c4d0 0x55fe660df720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d146b0 0x55fe660dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d35390 0x55fe660d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81adcd0 con 0x55fe65d0c4d0 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81ab5a0 con 0x55fe65d146b0 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40c6783740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81a93f0 con 0x55fe65d35390 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d0c4d0 0x55fe660df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d0c4d0 0x55fe660df720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55260/0 (socket says 172.21.15.78:55260) 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 -- 172.21.15.78:0/4293537439 learned_addr learned my addr 172.21.15.78:0/4293537439 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:43.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bce73640 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d146b0 0x55fe660dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 -- 172.21.15.78:0/4293537439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d35390 msgr2=0x55fe660d5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d35390 0x55fe660d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 -- 172.21.15.78:0/4293537439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d146b0 msgr2=0x55fe660dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d146b0 0x55fe660dba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 -- 172.21.15.78:0/4293537439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe65d0d0f0 con 0x55fe65d0c4d0 2024-04-25T14:05:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.819+0000 7f40bd674640 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d0c4d0 0x55fe660df720 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d97f68431142458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40a2ffd640 1 -- 172.21.15.78:0/4293537439 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8035040 con 0x55fe65d0c4d0 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40a2ffd640 1 -- 172.21.15.78:0/4293537439 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8055070 con 0x55fe65d0c4d0 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40a2ffd640 1 -- 172.21.15.78:0/4293537439 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8004ae0 con 0x55fe65d0c4d0 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 -- 172.21.15.78:0/4293537439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d0c4d0 msgr2=0x55fe660df720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d0c4d0 0x55fe660df720 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009900 tx=0x7f40a803e000 comp rx=0 tx=0).stop 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 -- 172.21.15.78:0/4293537439 shutdown_connections 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d35390 0x55fe660d5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d146b0 0x55fe660dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 --2- 172.21.15.78:0/4293537439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d0c4d0 0x55fe660df720 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 -- 172.21.15.78:0/4293537439 >> 172.21.15.78:0/4293537439 conn(0x55fe65d294e0 msgr2=0x55fe660d5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 -- 172.21.15.78:0/4293537439 shutdown_connections 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.820+0000 7f40c6783740 1 -- 172.21.15.78:0/4293537439 wait complete. 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 -- start start 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 Processor -- start 2024-04-25T14:05:43.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d0c4d0 0x55fe660e8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d146b0 0x55fe660e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d35390 0x55fe660e7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81a9a60 con 0x55fe65d146b0 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81aaed0 con 0x55fe65d35390 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81aa130 con 0x55fe65d0c4d0 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40c6783740 10 client.0 _ll_register_callbacks cb 0x7f40a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40bce73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d146b0 0x55fe660e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40bce73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d146b0 0x55fe660e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55276/0 (socket says 172.21.15.78:55276) 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40bce73640 1 -- 172.21.15.78:0/1471491995 learned_addr learned my addr 172.21.15.78:0/1471491995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40bde75640 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d35390 0x55fe660e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.821+0000 7f40bd674640 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d0c4d0 0x55fe660e8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40bce73640 1 -- 172.21.15.78:0/1471491995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d0c4d0 msgr2=0x55fe660e8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40bce73640 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d0c4d0 0x55fe660e8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40bce73640 1 -- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d35390 msgr2=0x55fe660e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40bce73640 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d35390 0x55fe660e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40bce73640 1 -- 172.21.15.78:0/1471491995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe65ffc4b0 con 0x55fe65d146b0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40bce73640 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d146b0 0x55fe660e6cc0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f40b40a6a70 tx=0x7f40b40a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e825f0ca886589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1471491995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4020040 con 0x55fe65d146b0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1471491995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f40b40a61e0 con 0x55fe65d146b0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a8055070 con 0x55fe65d146b0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1471491995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f40b4009de0 con 0x55fe65d146b0 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40c6783740 10 client.17226 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:43.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fe66155160 con 0x55fe65d146b0 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1471491995 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8055070 con 0x55fe65d146b0 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1471491995 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f40b40c1030 con 0x55fe65d146b0 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40c6783740 10 client.17226 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40c6783740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40c6783740 20 client.17226 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.822+0000 7f40c6783740 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cc93a0 0x55fe66157b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40c6783740 4 client.17226 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe66043800 con 0x55fe65cc93a0 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40bd674640 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cc93a0 0x55fe66157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40a0ff9640 10 client.17226 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40bd674640 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cc93a0 0x55fe66157b80 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6edf2c72407d1f2 server_cookie=d54122a4f991ddee in_seq=0 out_seq=0 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40a0ff9640 1 -- 172.21.15.78:0/1471491995 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe66043800 con 0x55fe65cc93a0 2024-04-25T14:05:43.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40a0ff9640 10 client.17226 handle_command_reply: tid=0 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.823+0000 7f40c6783740 1 client.17226 shutdown 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d146b0 msgr2=0x55fe660e6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d146b0 0x55fe660e6cc0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f40b40a6a70 tx=0x7f40b40a64e0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 shutdown_connections 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fe65d35390 0x55fe660e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fe65d146b0 0x55fe660e6cc0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55fe65cc93a0 0x55fe66157b80 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 --2- 172.21.15.78:0/1471491995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe65d0c4d0 0x55fe660e8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40a0ff9640 0 client.17226 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 >> 172.21.15.78:0/1471491995 conn(0x55fe65d294e0 msgr2=0x55fe660d5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 shutdown_connections 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 1 -- 172.21.15.78:0/1471491995 wait complete. 2024-04-25T14:05:43.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40c6783740 20 client.17226 trim_cache size 0 max 16384 2024-04-25T14:05:43.832 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:43.832 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:05:43.832 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:05:43.832 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a40afaf0 msgr2=0x7f40a40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a40afaf0 0x7f40a40b2000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009230 tx=0x7f40a8005a20 comp rx=0 tx=0).stop 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b8102260 msgr2=0x7f40b819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b8102260 0x7f40b819a790 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f40b400d950 tx=0x7f40b400de10 comp rx=0 tx=0).stop 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 shutdown_connections 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b81050b0 0x7f40b81a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a40afaf0 0x7f40a40b2000 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b8102260 0x7f40b819a790 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 --2- 172.21.15.78:0/3061419096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b8101760 0x7f40b819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 >> 172.21.15.78:0/3061419096 conn(0x7f40b80fd380 msgr2=0x7f40b8105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 shutdown_connections 2024-04-25T14:05:43.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.824+0000 7f40bf8d8640 1 -- 172.21.15.78:0/3061419096 wait complete. 2024-04-25T14:05:43.835 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:05:43.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 -- 172.21.15.78:0/1608521826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924104450 msgr2=0x7f392410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 --2- 172.21.15.78:0/1608521826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924104450 0x7f392410ab90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f391c009ca0 tx=0x7f391c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 -- 172.21.15.78:0/1608521826 shutdown_connections 2024-04-25T14:05:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 --2- 172.21.15.78:0/1608521826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924104450 0x7f392410ab90 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 --2- 172.21.15.78:0/1608521826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3924103970 0x7f3924103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 --2- 172.21.15.78:0/1608521826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3924102f90 0x7f39241033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 -- 172.21.15.78:0/1608521826 >> 172.21.15.78:0/1608521826 conn(0x7f39240fd330 msgr2=0x7f39240ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 -- 172.21.15.78:0/1608521826 shutdown_connections 2024-04-25T14:05:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.944+0000 7f3929ad8640 1 -- 172.21.15.78:0/1608521826 wait complete. 2024-04-25T14:05:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 Processor -- start 2024-04-25T14:05:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 -- start start 2024-04-25T14:05:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3924102f90 0x7f39241990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924103970 0x7f3924199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3924104450 0x7f39241a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f392410e520 con 0x7f3924103970 2024-04-25T14:05:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f392410e3a0 con 0x7f3924102f90 2024-04-25T14:05:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3929ad8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f392410e6a0 con 0x7f3924104450 2024-04-25T14:05:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924103970 0x7f3924199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924103970 0x7f3924199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55304/0 (socket says 172.21.15.78:55304) 2024-04-25T14:05:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 -- 172.21.15.78:0/2195653244 learned_addr learned my addr 172.21.15.78:0/2195653244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f39237fe640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3924102f90 0x7f39241990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f3923fff640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3924104450 0x7f39241a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 -- 172.21.15.78:0/2195653244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3924104450 msgr2=0x7f39241a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3924104450 0x7f39241a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 -- 172.21.15.78:0/2195653244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3924102f90 msgr2=0x7f39241990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3924102f90 0x7f39241990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 -- 172.21.15.78:0/2195653244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39241a0c00 con 0x7f3924103970 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.945+0000 7f391bfff640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924103970 0x7f3924199600 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f390c00e9f0 tx=0x7f390c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.946+0000 7f39217fa640 1 -- 172.21.15.78:0/2195653244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c00cd80 con 0x7f3924103970 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.946+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39241a0ef0 con 0x7f3924103970 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.946+0000 7f39217fa640 1 -- 172.21.15.78:0/2195653244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390c0108e0 con 0x7f3924103970 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.946+0000 7f39217fa640 1 -- 172.21.15.78:0/2195653244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c018980 con 0x7f3924103970 2024-04-25T14:05:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.946+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39241a13b0 con 0x7f3924103970 2024-04-25T14:05:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.947+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3924109940 con 0x7f3924103970 2024-04-25T14:05:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.948+0000 7f39217fa640 1 -- 172.21.15.78:0/2195653244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f390c010430 con 0x7f3924103970 2024-04-25T14:05:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.949+0000 7f39217fa640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39100afaf0 0x7f39100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.949+0000 7f39237fe640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39100afaf0 0x7f39100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.949+0000 7f39217fa640 1 -- 172.21.15.78:0/2195653244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f390c0df540 con 0x7f3924103970 2024-04-25T14:05:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.949+0000 7f39237fe640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39100afaf0 0x7f39100b2000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3924069b90 tx=0x7f3914005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:43.952+0000 7f39217fa640 1 -- 172.21.15.78:0/2195653244 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f390c0a8110 con 0x7f3924103970 2024-04-25T14:05:44.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.169+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f392402db50 con 0x7f3924103970 2024-04-25T14:05:44.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.170+0000 7f39217fa640 1 -- 172.21.15.78:0/2195653244 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f390c0b0190 con 0x7f3924103970 2024-04-25T14:05:44.170 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:05:44.170 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:05:44.170 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39100afaf0 msgr2=0x7f39100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39100afaf0 0x7f39100b2000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3924069b90 tx=0x7f3914005f90 comp rx=0 tx=0).stop 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924103970 msgr2=0x7f3924199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924103970 0x7f3924199600 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f390c00e9f0 tx=0x7f390c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 shutdown_connections 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3924104450 0x7f39241a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f39100afaf0 0x7f39100b2000 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3924103970 0x7f3924199600 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 --2- 172.21.15.78:0/2195653244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3924102f90 0x7f39241990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 >> 172.21.15.78:0/2195653244 conn(0x7f39240fd330 msgr2=0x7f3924106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 shutdown_connections 2024-04-25T14:05:44.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.172+0000 7f3929ad8640 1 -- 172.21.15.78:0/2195653244 wait complete. 2024-04-25T14:05:44.183 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:05:44.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 -- 172.21.15.78:0/3410723853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 msgr2=0x7f1d2c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 --2- 172.21.15.78:0/3410723853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c104030 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009900 tx=0x7f1d1c02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:44.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 -- 172.21.15.78:0/3410723853 shutdown_connections 2024-04-25T14:05:44.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 --2- 172.21.15.78:0/3410723853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d2c108920 0x7f1d2c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 --2- 172.21.15.78:0/3410723853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c104030 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 --2- 172.21.15.78:0/3410723853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2c1029e0 0x7f1d2c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 -- 172.21.15.78:0/3410723853 >> 172.21.15.78:0/3410723853 conn(0x7f1d2c0fdbd0 msgr2=0x7f1d2c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 -- 172.21.15.78:0/3410723853 shutdown_connections 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.294+0000 7f1d34506640 1 -- 172.21.15.78:0/3410723853 wait complete. 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 Processor -- start 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 -- start start 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2c1029e0 0x7f1d2c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d2c108920 0x7f1d2c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c10f600 con 0x7f1d2c103c20 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c10f480 con 0x7f1d2c1029e0 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d34506640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c10f780 con 0x7f1d2c108920 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55322/0 (socket says 172.21.15.78:55322) 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 -- 172.21.15.78:0/3334379839 learned_addr learned my addr 172.21.15.78:0/3334379839 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d2c108920 0x7f1d2c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 -- 172.21.15.78:0/3334379839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d2c108920 msgr2=0x7f1d2c1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d322a2640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2c1029e0 0x7f1d2c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d2c108920 0x7f1d2c1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 -- 172.21.15.78:0/3334379839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2c1029e0 msgr2=0x7f1d2c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2c1029e0 0x7f1d2c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 -- 172.21.15.78:0/3334379839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c1a2070 con 0x7f1d2c103c20 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.295+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c19aa70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c006010 tx=0x7f1d1c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.296+0000 7f1d237fe640 1 -- 172.21.15.78:0/3334379839 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c03f030 con 0x7f1d2c103c20 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.296+0000 7f1d237fe640 1 -- 172.21.15.78:0/3334379839 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c003cb0 con 0x7f1d2c103c20 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.296+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c1a2300 con 0x7f1d2c103c20 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.296+0000 7f1d237fe640 1 -- 172.21.15.78:0/3334379839 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c035070 con 0x7f1d2c103c20 2024-04-25T14:05:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.296+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c1a27f0 con 0x7f1d2c103c20 2024-04-25T14:05:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.298+0000 7f1d3b3bb740 1 Processor -- start 2024-04-25T14:05:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.298+0000 7f1d237fe640 1 -- 172.21.15.78:0/3334379839 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1d1c042030 con 0x7f1d2c103c20 2024-04-25T14:05:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.298+0000 7f1d3b3bb740 1 -- start start 2024-04-25T14:05:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ceb6b0 0x5644770afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564477012d80 con 0x564476ceb6b0 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564476fcca20 con 0x564476d0c390 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56447700ed40 con 0x564476ce34d0 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ceb6b0 0x5644770afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ceb6b0 0x5644770afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55336/0 (socket says 172.21.15.78:55336) 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 -- 172.21.15.78:0/240448432 learned_addr learned my addr 172.21.15.78:0/240448432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d322a2640 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d237fe640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d180afaf0 0x7f1d180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 -- 172.21.15.78:0/240448432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 msgr2=0x5644770aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 -- 172.21.15.78:0/240448432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 msgr2=0x5644770ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 -- 172.21.15.78:0/240448432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564476fd7c30 con 0x564476ceb6b0 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d180afaf0 0x7f1d180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d237fe640 1 -- 172.21.15.78:0/3334379839 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1d1c0ff370 con 0x7f1d2c103c20 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.299+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ceb6b0 0x5644770afa20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c008ad0 tx=0x7f1d1c04b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429ea1856e830a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.300+0000 7f1d20bfe640 1 -- 172.21.15.78:0/240448432 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0c7190 con 0x564476ceb6b0 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.300+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d180afaf0 0x7f1d180b2000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x5644770af860 tx=0x7f1d24004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.300+0000 7f1d20bfe640 1 -- 172.21.15.78:0/240448432 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c001970 con 0x564476ceb6b0 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.300+0000 7f1d20bfe640 1 -- 172.21.15.78:0/240448432 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c02e260 con 0x564476ceb6b0 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.300+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/240448432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ceb6b0 msgr2=0x5644770afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.300+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ceb6b0 0x5644770afa20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c008ad0 tx=0x7f1d1c04b000 comp rx=0 tx=0).stop 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/240448432 shutdown_connections 2024-04-25T14:05:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ceb6b0 0x5644770afa20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/240448432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/240448432 >> 172.21.15.78:0/240448432 conn(0x564476d004e0 msgr2=0x5644770aa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/240448432 shutdown_connections 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/240448432 wait complete. 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- start start 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 Processor -- start 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ce34d0 0x5644770b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ceb6b0 0x5644770b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5644770b02f0 con 0x564476ce34d0 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564476f99dc0 con 0x564476d0c390 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5644770b0470 con 0x564476ceb6b0 2024-04-25T14:05:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d3b3bb740 10 client.0 _ll_register_callbacks cb 0x564476fb5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d322a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ce34d0 0x5644770b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d322a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ce34d0 0x5644770b8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55348/0 (socket says 172.21.15.78:55348) 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.301+0000 7f1d322a2640 1 -- 172.21.15.78:0/380201715 learned_addr learned my addr 172.21.15.78:0/380201715 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ceb6b0 0x5644770b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d322a2640 1 -- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ceb6b0 msgr2=0x5644770b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d322a2640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ceb6b0 0x5644770b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d322a2640 1 -- 172.21.15.78:0/380201715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 msgr2=0x5644770bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d322a2640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d322a2640 1 -- 172.21.15.78:0/380201715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564476d243f0 con 0x564476ce34d0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d322a2640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ce34d0 0x5644770b8250 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009d40 tx=0x7f1d280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a89a9051d4b1627 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d15ffb640 1 -- 172.21.15.78:0/380201715 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28012860 con 0x564476ce34d0 2024-04-25T14:05:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d15ffb640 1 -- 172.21.15.78:0/380201715 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1d28012a00 con 0x564476ce34d0 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564477020990 con 0x564476ce34d0 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d15ffb640 1 -- 172.21.15.78:0/380201715 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1d28009e30 con 0x564476ce34d0 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d3b3bb740 10 client.17244 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564476dee7d0 con 0x564476ce34d0 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d15ffb640 1 -- 172.21.15.78:0/380201715 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564477020990 con 0x564476ce34d0 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.302+0000 7f1d15ffb640 1 -- 172.21.15.78:0/380201715 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1d2800f040 con 0x564476ce34d0 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d3b3bb740 10 client.17244 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d3b3bb740 10 client.17244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d3b3bb740 20 client.17244 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ccaf60 0x5644770c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d3b3bb740 4 client.17244 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56447701a800 con 0x564476ccaf60 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ccaf60 0x5644770c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d15ffb640 10 client.17244 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.303+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ccaf60 0x5644770c6040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd3b6c159b7f4fa8 server_cookie=a217cfea0a498b19 in_seq=0 out_seq=0 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d15ffb640 1 -- 172.21.15.78:0/380201715 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56447701a800 con 0x564476ccaf60 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d15ffb640 10 client.17244 handle_command_reply: tid=0 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 client.17244 shutdown 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ce34d0 msgr2=0x5644770b8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ce34d0 0x5644770b8250 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009d40 tx=0x7f1d280117b0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 shutdown_connections 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ccaf60 0x5644770c6040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476d0c390 0x5644770bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ceb6b0 0x5644770b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/380201715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476ce34d0 0x5644770b8250 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.305+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 >> 172.21.15.78:0/380201715 conn(0x564476d004e0 msgr2=0x5644770acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.304+0000 7f1d15ffb640 0 client.17244 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:44.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.305+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 shutdown_connections 2024-04-25T14:05:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.305+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/380201715 wait complete. 2024-04-25T14:05:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.305+0000 7f1d3b3bb740 20 client.17244 trim_cache size 0 max 16384 2024-04-25T14:05:44.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 Processor -- start 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 -- start start 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x5644770ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1adbb0 con 0x564476d0c390 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1ab480 con 0x564476ceb6b0 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1a92d0 con 0x564476ce34d0 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d32aa3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x5644770ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d32aa3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x5644770ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55362/0 (socket says 172.21.15.78:55362) 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d32aa3640 1 -- 172.21.15.78:0/1730068610 learned_addr learned my addr 172.21.15.78:0/1730068610 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.318+0000 7f1d322a2640 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d32aa3640 1 -- 172.21.15.78:0/1730068610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 msgr2=0x5644770b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d32aa3640 1 -- 172.21.15.78:0/1730068610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 msgr2=0x5644770bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d32aa3640 1 -- 172.21.15.78:0/1730068610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644770b3b60 con 0x564476d0c390 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x5644770ca7f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1d24012a10 tx=0x7f1d24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44e77b0a5f0a03f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d17fff640 1 -- 172.21.15.78:0/1730068610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2401a070 con 0x564476d0c390 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d17fff640 1 -- 172.21.15.78:0/1730068610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d24011ca0 con 0x564476d0c390 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.319+0000 7f1d17fff640 1 -- 172.21.15.78:0/1730068610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24021660 con 0x564476d0c390 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/1730068610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 msgr2=0x5644770ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x5644770ca7f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1d24012a10 tx=0x7f1d24009840 comp rx=0 tx=0).stop 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/1730068610 shutdown_connections 2024-04-25T14:05:44.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x5644770ca7f0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/1730068610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/1730068610 >> 172.21.15.78:0/1730068610 conn(0x564476d004e0 msgr2=0x5644770b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/1730068610 shutdown_connections 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/1730068610 wait complete. 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- start start 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 Processor -- start 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770be540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x56447712c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1a9940 con 0x564476d0c390 2024-04-25T14:05:44.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1aadb0 con 0x564476ceb6b0 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1aa010 con 0x564476ce34d0 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d3b3bb740 10 client.0 _ll_register_callbacks cb 0x7f1d28009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d32aa3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x56447712c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d32aa3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x56447712c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55370/0 (socket says 172.21.15.78:55370) 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d32aa3640 1 -- 172.21.15.78:0/2077373977 learned_addr learned my addr 172.21.15.78:0/2077373977 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d322a2640 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770be540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.320+0000 7f1d31aa1640 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d32aa3640 1 -- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 msgr2=0x5644770be540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770be540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d32aa3640 1 -- 172.21.15.78:0/2077373977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 msgr2=0x5644770bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d32aa3640 1 -- 172.21.15.78:0/2077373977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644770bfb50 con 0x564476d0c390 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d32aa3640 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x56447712c3a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1d24005f40 tx=0x7f1d24009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1157397aa656b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d15ffb640 1 -- 172.21.15.78:0/2077373977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24038060 con 0x564476d0c390 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d15ffb640 1 -- 172.21.15.78:0/2077373977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1d24011aa0 con 0x564476d0c390 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d24011ca0 con 0x564476d0c390 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d15ffb640 1 -- 172.21.15.78:0/2077373977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1d2401a040 con 0x564476d0c390 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 10 client.17250 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56447712d840 con 0x564476d0c390 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d15ffb640 1 -- 172.21.15.78:0/2077373977 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d24011ca0 con 0x564476d0c390 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d15ffb640 1 -- 172.21.15.78:0/2077373977 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1d2400f040 con 0x564476d0c390 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 10 client.17250 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 10 client.17250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 20 client.17250 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ca03a0 0x5644771301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 4 client.17250 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.321+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564476f95610 con 0x564476ca03a0 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.322+0000 7f1d322a2640 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ca03a0 0x5644771301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.322+0000 7f1d15ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.322+0000 7f1d322a2640 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ca03a0 0x5644771301f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c98a1c936e8f1c5b server_cookie=37f9d9f508c3233 in_seq=0 out_seq=0 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.322+0000 7f1d15ffb640 1 -- 172.21.15.78:0/2077373977 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564476f95610 con 0x564476ca03a0 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.322+0000 7f1d15ffb640 10 client.17250 handle_command_reply: tid=0 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.322+0000 7f1d3b3bb740 1 client.17250 shutdown 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 msgr2=0x56447712c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x56447712c3a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1d24005f40 tx=0x7f1d24009d50 comp rx=0 tx=0).stop 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 shutdown_connections 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564476ca03a0 0x5644771301f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564476d0c390 0x56447712c3a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564476ceb6b0 0x5644770bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d15ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 --2- 172.21.15.78:0/2077373977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564476ce34d0 0x5644770be540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 >> 172.21.15.78:0/2077373977 conn(0x564476d004e0 msgr2=0x5644770ace20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 shutdown_connections 2024-04-25T14:05:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 1 -- 172.21.15.78:0/2077373977 wait complete. 2024-04-25T14:05:44.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d3b3bb740 20 client.17250 trim_cache size 0 max 16384 2024-04-25T14:05:44.330 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d180afaf0 msgr2=0x7f1d180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d180afaf0 0x7f1d180b2000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x5644770af860 tx=0x7f1d24004ab0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 msgr2=0x7f1d2c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c19aa70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c006010 tx=0x7f1d1c004340 comp rx=0 tx=0).stop 2024-04-25T14:05:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 shutdown_connections 2024-04-25T14:05:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1d2c108920 0x7f1d2c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1d180afaf0 0x7f1d180b2000 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d2c103c20 0x7f1d2c19aa70 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 --2- 172.21.15.78:0/3334379839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2c1029e0 0x7f1d2c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 >> 172.21.15.78:0/3334379839 conn(0x7f1d2c0fdbd0 msgr2=0x7f1d2c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 shutdown_connections 2024-04-25T14:05:44.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.323+0000 7f1d34506640 1 -- 172.21.15.78:0/3334379839 wait complete. 2024-04-25T14:05:44.334 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:05:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/3551172677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8108940 msgr2=0x7fc1f810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/3551172677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8108940 0x7fc1f810ad60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009ca0 tx=0x7fc1f002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/3551172677 shutdown_connections 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/3551172677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8108940 0x7fc1f810ad60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/3551172677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1f8103c40 0x7fc1f8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/3551172677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f8102a00 0x7fc1f8103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/3551172677 >> 172.21.15.78:0/3551172677 conn(0x7fc1f80fdbf0 msgr2=0x7fc1f81000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/3551172677 shutdown_connections 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/3551172677 wait complete. 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.444+0000 7fc1ffdc7640 1 Processor -- start 2024-04-25T14:05:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 -- start start 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8102a00 0x7fc1f81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f8103c40 0x7fc1f8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1f8108940 0x7fc1f81a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810e510 con 0x7fc1f8102a00 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810e390 con 0x7fc1f8103c40 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810e690 con 0x7fc1f8108940 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8102a00 0x7fc1f81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8102a00 0x7fc1f81990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55376/0 (socket says 172.21.15.78:55376) 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 -- 172.21.15.78:0/1090852332 learned_addr learned my addr 172.21.15.78:0/1090852332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fd362640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f8103c40 0x7fc1f8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fe364640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1f8108940 0x7fc1f81a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 -- 172.21.15.78:0/1090852332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1f8108940 msgr2=0x7fc1f81a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1f8108940 0x7fc1f81a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 -- 172.21.15.78:0/1090852332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f8103c40 msgr2=0x7fc1f8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f8103c40 0x7fc1f8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 -- 172.21.15.78:0/1090852332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f81a0b80 con 0x7fc1f8102a00 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8102a00 0x7fc1f81990f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005fd0 tx=0x7fc1e800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1eeffd640 1 -- 172.21.15.78:0/1090852332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8017070 con 0x7fc1f8102a00 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1eeffd640 1 -- 172.21.15.78:0/1090852332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e800de50 con 0x7fc1f8102a00 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f81a0e70 con 0x7fc1f8102a00 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.445+0000 7fc1eeffd640 1 -- 172.21.15.78:0/1090852332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8003c50 con 0x7fc1f8102a00 2024-04-25T14:05:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.446+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81a12d0 con 0x7fc1f8102a00 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.448+0000 7fc206c73740 1 Processor -- start 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.448+0000 7fc1eeffd640 1 -- 172.21.15.78:0/1090852332 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc1e8003df0 con 0x7fc1f8102a00 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.448+0000 7fc206c73740 1 -- start start 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc206c73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022f5390 0x55c302697810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc206c73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c302697d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc206c73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c302698a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc206c73740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c3025fbd80 con 0x55c3022d46b0 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc206c73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c3025b5a20 con 0x55c3022f5390 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc206c73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c3025f7d40 con 0x55c3022cc4d0 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c302698a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c302698a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55382/0 (socket says 172.21.15.78:55382) 2024-04-25T14:05:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 -- 172.21.15.78:0/3514098434 learned_addr learned my addr 172.21.15.78:0/3514098434 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fe364640 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022f5390 0x55c302697810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c302697d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 -- 172.21.15.78:0/3514098434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 msgr2=0x55c302697d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c302697d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 -- 172.21.15.78:0/3514098434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022f5390 msgr2=0x55c302697810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022f5390 0x55c302697810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 -- 172.21.15.78:0/3514098434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3025c0c30 con 0x55c3022d46b0 2024-04-25T14:05:44.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1eeffd640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc1e40afaf0 0x7fc1e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fe364640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc1e40afaf0 0x7fc1e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1fd362640 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c302698a20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400ac00 tx=0x7fc1f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b05303e5731411e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1eeffd640 1 -- 172.21.15.78:0/1090852332 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc1e80de8e0 con 0x7fc1f8102a00 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.449+0000 7fc1e3fff640 1 -- 172.21.15.78:0/3514098434 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f401d070 con 0x55c3022d46b0 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.450+0000 7fc1e3fff640 1 -- 172.21.15.78:0/3514098434 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4018af0 con 0x55c3022d46b0 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.450+0000 7fc1e3fff640 1 -- 172.21.15.78:0/3514098434 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4010ab0 con 0x55c3022d46b0 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.450+0000 7fc206c73740 1 -- 172.21.15.78:0/3514098434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 msgr2=0x55c302698a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.450+0000 7fc1fe364640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc1e40afaf0 0x7fc1e40b2000 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1f002f500 tx=0x7fc1f002f310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.450+0000 7fc206c73740 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c302698a20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400ac00 tx=0x7fc1f400f790 comp rx=0 tx=0).stop 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 -- 172.21.15.78:0/3514098434 shutdown_connections 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c302698a20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c302697d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 --2- 172.21.15.78:0/3514098434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022f5390 0x55c302697810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 -- 172.21.15.78:0/3514098434 >> 172.21.15.78:0/3514098434 conn(0x55c3022e94e0 msgr2=0x55c302693020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 -- 172.21.15.78:0/3514098434 shutdown_connections 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 -- 172.21.15.78:0/3514098434 wait complete. 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 -- start start 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 Processor -- start 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022cc4d0 0x55c3026a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.451+0000 7fc206c73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022f5390 0x55c3026a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c302699530 con 0x55c3022cc4d0 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c302582dc0 con 0x55c3022d46b0 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c3026996b0 con 0x55c3022f5390 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 10 client.0 _ll_register_callbacks cb 0x55c30259e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022cc4d0 0x55c3026a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022cc4d0 0x55c3026a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55384/0 (socket says 172.21.15.78:55384) 2024-04-25T14:05:44.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 -- 172.21.15.78:0/327135372 learned_addr learned my addr 172.21.15.78:0/327135372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fd362640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fe364640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022f5390 0x55c3026a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 -- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022f5390 msgr2=0x55c3026a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022f5390 0x55c3026a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 -- 172.21.15.78:0/327135372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 msgr2=0x55c3026a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 -- 172.21.15.78:0/327135372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c30230d3f0 con 0x55c3022cc4d0 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fd362640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022cc4d0 0x55c3026a1570 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80058c0 tx=0x7fc1e8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719f84abf9021a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1e17fa640 1 -- 172.21.15.78:0/327135372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80a6db0 con 0x55c3022cc4d0 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1e17fa640 1 -- 172.21.15.78:0/327135372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc1e80a6010 con 0x55c3022cc4d0 2024-04-25T14:05:44.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f4018af0 con 0x55c3022cc4d0 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc1e17fa640 1 -- 172.21.15.78:0/327135372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc1e80ab160 con 0x55c3022cc4d0 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 10 client.17262 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.452+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c3023d77d0 con 0x55c3022cc4d0 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc1e17fa640 1 -- 172.21.15.78:0/327135372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4018af0 con 0x55c3022cc4d0 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc1e17fa640 1 -- 172.21.15.78:0/327135372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc1e80a27b0 con 0x55c3022cc4d0 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc206c73740 10 client.17262 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc206c73740 10 client.17262 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc206c73740 20 client.17262 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc206c73740 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022b3f60 0x55c3026af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc206c73740 4 client.17262 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c302609990 con 0x55c3022b3f60 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.453+0000 7fc1fd362640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022b3f60 0x55c3026af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.454+0000 7fc1e17fa640 10 client.17262 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.454+0000 7fc1fd362640 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022b3f60 0x55c3026af240 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4710166bc65872d server_cookie=f381535111085453 in_seq=0 out_seq=0 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.454+0000 7fc1e17fa640 1 -- 172.21.15.78:0/327135372 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c302609990 con 0x55c3022b3f60 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.454+0000 7fc1e17fa640 10 client.17262 handle_command_reply: tid=0 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.454+0000 7fc206c73740 1 client.17262 shutdown 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.454+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022cc4d0 msgr2=0x55c3026a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.454+0000 7fc206c73740 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022cc4d0 0x55c3026a1570 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80058c0 tx=0x7fc1e8009f60 comp rx=0 tx=0).stop 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 shutdown_connections 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022b3f60 0x55c3026af240 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022f5390 0x55c3026a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc1e17fa640 0 client.17262 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 --2- 172.21.15.78:0/327135372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022cc4d0 0x55c3026a1570 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 >> 172.21.15.78:0/327135372 conn(0x55c3022e94e0 msgr2=0x55c302695f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 shutdown_connections 2024-04-25T14:05:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 1 -- 172.21.15.78:0/327135372 wait complete. 2024-04-25T14:05:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.455+0000 7fc206c73740 20 client.17262 trim_cache size 0 max 16384 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 Processor -- start 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 -- start start 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022b3f60 0x55c3026b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022cc4d0 0x55c3026a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c3026b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81ab840 con 0x55c3022d46b0 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81a9110 con 0x55c3022cc4d0 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc206c73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81a6f60 con 0x55c3022b3f60 2024-04-25T14:05:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc1fe364640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c3026b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc1fe364640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c3026b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55386/0 (socket says 172.21.15.78:55386) 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.468+0000 7fc1fe364640 1 -- 172.21.15.78:0/788767576 learned_addr learned my addr 172.21.15.78:0/788767576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022b3f60 0x55c3026b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fe364640 1 -- 172.21.15.78:0/788767576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022b3f60 msgr2=0x55c3026b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fe364640 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022b3f60 0x55c3026b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fd362640 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022cc4d0 0x55c3026a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fe364640 1 -- 172.21.15.78:0/788767576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022cc4d0 msgr2=0x55c3026a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fe364640 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022cc4d0 0x55c3026a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fe364640 1 -- 172.21.15.78:0/788767576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c30269f660 con 0x55c3022d46b0 2024-04-25T14:05:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fd362640 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022cc4d0 0x55c3026a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:44.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:44 smithi078 ceph-mon[29163]: pgmap v534: 97 pgs: 97 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T14:05:44.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:44 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2195653244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:44.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1fe364640 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c3026b4f10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0035000 tx=0x7fc1f0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c27469f7c89d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1e37fe640 1 -- 172.21.15.78:0/788767576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00317a0 con 0x55c3022d46b0 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1e37fe640 1 -- 172.21.15.78:0/788767576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f0036040 con 0x55c3022d46b0 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.469+0000 7fc1e37fe640 1 -- 172.21.15.78:0/788767576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0004440 con 0x55c3022d46b0 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 -- 172.21.15.78:0/788767576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 msgr2=0x55c3026b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c3026b4f10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0035000 tx=0x7fc1f0040000 comp rx=0 tx=0).stop 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 -- 172.21.15.78:0/788767576 shutdown_connections 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022d46b0 0x55c3026b4f10 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022cc4d0 0x55c3026a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 --2- 172.21.15.78:0/788767576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022b3f60 0x55c3026b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 -- 172.21.15.78:0/788767576 >> 172.21.15.78:0/788767576 conn(0x55c3022e94e0 msgr2=0x55c30269c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 -- 172.21.15.78:0/788767576 shutdown_connections 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 -- 172.21.15.78:0/788767576 wait complete. 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.470+0000 7fc206c73740 1 -- start start 2024-04-25T14:05:44.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 1 Processor -- start 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022b3f60 0x55c3026a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c3026a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81a75d0 con 0x55c3022b3f60 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81a8a40 con 0x55c3022d46b0 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f81a7ca0 con 0x55c3022cc4d0 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc206c73740 10 client.0 _ll_register_callbacks cb 0x7fc1e8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022b3f60 0x55c3026a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022b3f60 0x55c3026a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55390/0 (socket says 172.21.15.78:55390) 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 -- 172.21.15.78:0/2434515015 learned_addr learned my addr 172.21.15.78:0/2434515015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fd362640 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c3026a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 -- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 msgr2=0x55c3026a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c3026a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fe364640 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 -- 172.21.15.78:0/2434515015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 msgr2=0x55c3026a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 -- 172.21.15.78:0/2434515015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c302715570 con 0x55c3022b3f60 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1fdb63640 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022b3f60 0x55c3026a7220 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8000c00 tx=0x7fc1e80a5b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742ac50777318a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.471+0000 7fc1e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc1e17fa640 1 -- 172.21.15.78:0/2434515015 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80a6b80 con 0x55c3022b3f60 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc1e17fa640 1 -- 172.21.15.78:0/2434515015 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc1e80a6010 con 0x55c3022b3f60 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f0036040 con 0x55c3022b3f60 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc1e17fa640 1 -- 172.21.15.78:0/2434515015 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc1e80a2930 con 0x55c3022b3f60 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 10 client.17268 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c3027167d0 con 0x55c3022b3f60 2024-04-25T14:05:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc1e17fa640 1 -- 172.21.15.78:0/2434515015 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f0036040 con 0x55c3022b3f60 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc1e17fa640 1 -- 172.21.15.78:0/2434515015 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc1e80040e0 con 0x55c3022b3f60 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 10 client.17268 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 10 client.17268 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 20 client.17268 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022f5390 0x55c302719000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 4 client.17268 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c302603800 con 0x55c3022f5390 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.472+0000 7fc1fd362640 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022f5390 0x55c302719000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc1fd362640 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022f5390 0x55c302719000 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a8b2d14a44e8d04 server_cookie=c8aab689dcc47e91 in_seq=0 out_seq=0 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc1e17fa640 10 client.17268 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc1e17fa640 1 -- 172.21.15.78:0/2434515015 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c302603800 con 0x55c3022f5390 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc1e17fa640 10 client.17268 handle_command_reply: tid=0 2024-04-25T14:05:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 client.17268 shutdown 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022b3f60 msgr2=0x55c3026a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022b3f60 0x55c3026a7220 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8000c00 tx=0x7fc1e80a5b40 comp rx=0 tx=0).stop 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 shutdown_connections 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c3022d46b0 0x55c3026a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c3022f5390 0x55c302719000 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3022cc4d0 0x55c3026a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 --2- 172.21.15.78:0/2434515015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3022b3f60 0x55c3026a7220 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc1e17fa640 0 client.17268 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 >> 172.21.15.78:0/2434515015 conn(0x55c3022e94e0 msgr2=0x55c302695ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 shutdown_connections 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 1 -- 172.21.15.78:0/2434515015 wait complete. 2024-04-25T14:05:44.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.473+0000 7fc206c73740 20 client.17268 trim_cache size 0 max 16384 2024-04-25T14:05:44.481 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc1e40afaf0 msgr2=0x7fc1e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc1e40afaf0 0x7fc1e40b2000 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1f002f500 tx=0x7fc1f002f310 comp rx=0 tx=0).stop 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8102a00 msgr2=0x7fc1f81990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8102a00 0x7fc1f81990f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005fd0 tx=0x7fc1e800dd20 comp rx=0 tx=0).stop 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 shutdown_connections 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1f8108940 0x7fc1f81a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1f8103c40 0x7fc1f8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc1e40afaf0 0x7fc1e40b2000 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 --2- 172.21.15.78:0/1090852332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f8102a00 0x7fc1f81990f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 >> 172.21.15.78:0/1090852332 conn(0x7fc1f80fdbf0 msgr2=0x7fc1f8104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 shutdown_connections 2024-04-25T14:05:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.474+0000 7fc1ffdc7640 1 -- 172.21.15.78:0/1090852332 wait complete. 2024-04-25T14:05:44.485 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:05:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:44 smithi112 ceph-mon[32617]: pgmap v534: 97 pgs: 97 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T14:05:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:44 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2195653244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/910155962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8104d30 msgr2=0x7f3cb8105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/910155962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8104d30 0x7f3cb8105140 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009900 tx=0x7f3ca002a410 comp rx=0 tx=0).stop 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/910155962 shutdown_connections 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/910155962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8109a30 0x7f3cb810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/910155962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8104d30 0x7f3cb8105140 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/910155962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb8103af0 0x7f3cb81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/910155962 >> 172.21.15.78:0/910155962 conn(0x7f3cb80fece0 msgr2=0x7f3cb8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/910155962 shutdown_connections 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.594+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/910155962 wait complete. 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 Processor -- start 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 -- start start 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8103af0 0x7f3cb819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb8104d30 0x7f3cb819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8109a30 0x7f3cb81a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb810f610 con 0x7f3cb8103af0 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb810f490 con 0x7f3cb8109a30 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.595+0000 7f3cbc9c5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb810f790 con 0x7f3cb8104d30 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8103af0 0x7f3cb819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8103af0 0x7f3cb819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55420/0 (socket says 172.21.15.78:55420) 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 -- 172.21.15.78:0/2441074327 learned_addr learned my addr 172.21.15.78:0/2441074327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8109a30 0x7f3cb81a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3caffff640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb8104d30 0x7f3cb819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 -- 172.21.15.78:0/2441074327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb8104d30 msgr2=0x7f3cb819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb8104d30 0x7f3cb819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 -- 172.21.15.78:0/2441074327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8109a30 msgr2=0x7f3cb81a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8109a30 0x7f3cb81a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 -- 172.21.15.78:0/2441074327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb81a3fe0 con 0x7f3cb8103af0 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8109a30 0x7f3cb81a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3caffff640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb8104d30 0x7f3cb819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8103af0 0x7f3cb819c4a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8005d00 tx=0x7f3ca800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3caf7fe640 1 -- 172.21.15.78:0/2441074327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8017070 con 0x7f3cb8103af0 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3caf7fe640 1 -- 172.21.15.78:0/2441074327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca8009c30 con 0x7f3cb8103af0 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb81a42d0 con 0x7f3cb8103af0 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3caf7fe640 1 -- 172.21.15.78:0/2441074327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800f620 con 0x7f3cb8103af0 2024-04-25T14:05:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.596+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb81a47d0 con 0x7f3cb8103af0 2024-04-25T14:05:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3caf7fe640 1 -- 172.21.15.78:0/2441074327 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3ca801a070 con 0x7f3cb8103af0 2024-04-25T14:05:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 Processor -- start 2024-04-25T14:05:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 -- start start 2024-04-25T14:05:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f6f390 0x558878311810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f464d0 0x558878311d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x558878312a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558878275d80 con 0x558877f4e6b0 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55887822fa20 con 0x558877f6f390 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cc3879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558878271d40 con 0x558877f464d0 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x558878312a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3caffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x558878312a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55428/0 (socket says 172.21.15.78:55428) 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3caffff640 1 -- 172.21.15.78:0/2518436550 learned_addr learned my addr 172.21.15.78:0/2518436550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f6f390 0x558878311810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f464d0 0x558878311d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.599+0000 7f3caf7fe640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3ca40afaf0 0x7f3ca40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3caffff640 1 -- 172.21.15.78:0/2518436550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f464d0 msgr2=0x558878311d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3caffff640 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f464d0 0x558878311d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3caffff640 1 -- 172.21.15.78:0/2518436550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f6f390 msgr2=0x558878311810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3caffff640 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f6f390 0x558878311810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3caffff640 1 -- 172.21.15.78:0/2518436550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887823ac30 con 0x558877f4e6b0 2024-04-25T14:05:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f6f390 0x558878311810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3ca40afaf0 0x7f3ca40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3caf7fe640 1 -- 172.21.15.78:0/2441074327 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3ca80de740 con 0x7f3cb8103af0 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3caffff640 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x558878312a20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0031440 tx=0x7f3ca0032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af611623606cb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3cacff9640 1 -- 172.21.15.78:0/2518436550 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca003f070 con 0x558877f4e6b0 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3ca40afaf0 0x7f3ca40b2000 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x558878312860 tx=0x7f3cb0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3cacff9640 1 -- 172.21.15.78:0/2518436550 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca00317a0 con 0x558877f4e6b0 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.600+0000 7f3cacff9640 1 -- 172.21.15.78:0/2518436550 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0035070 con 0x558877f4e6b0 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 -- 172.21.15.78:0/2518436550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 msgr2=0x558878312a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x558878312a20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0031440 tx=0x7f3ca0032c70 comp rx=0 tx=0).stop 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 -- 172.21.15.78:0/2518436550 shutdown_connections 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x558878312a20 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f464d0 0x558878311d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2518436550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f6f390 0x558878311810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 -- 172.21.15.78:0/2518436550 >> 172.21.15.78:0/2518436550 conn(0x558877f634e0 msgr2=0x55887830d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 -- 172.21.15.78:0/2518436550 shutdown_connections 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 -- 172.21.15.78:0/2518436550 wait complete. 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.601+0000 7f3cc3879740 1 -- start start 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 1 Processor -- start 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f464d0 0x55887831b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f4e6b0 0x55887831bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f6f390 0x558878322a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588783134d0 con 0x558877f464d0 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5588781fcdc0 con 0x558877f4e6b0 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558878313650 con 0x558877f6f390 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cc3879740 10 client.0 _ll_register_callbacks cb 0x558878218600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f464d0 0x55887831b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f464d0 0x55887831b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55442/0 (socket says 172.21.15.78:55442) 2024-04-25T14:05:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 -- 172.21.15.78:0/2650949788 learned_addr learned my addr 172.21.15.78:0/2650949788 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 -- 172.21.15.78:0/2650949788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f6f390 msgr2=0x558878322a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f6f390 0x558878322a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 -- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f4e6b0 msgr2=0x55887831bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3caffff640 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f4e6b0 0x55887831bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f4e6b0 0x55887831bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 -- 172.21.15.78:0/2650949788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558877f873f0 con 0x558877f464d0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3caffff640 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f4e6b0 0x55887831bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f464d0 0x55887831b590 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80a3e80 tx=0x7f3ca80a6aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71176c93ad7e9a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.602+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2650949788 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8021070 con 0x558877f464d0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2650949788 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3ca80af7e0 con 0x558877f464d0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca00317a0 con 0x558877f464d0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2650949788 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3ca80af3b0 con 0x558877f464d0 2024-04-25T14:05:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 10 client.17280 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5588780517d0 con 0x558877f464d0 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2650949788 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca00317a0 con 0x558877f464d0 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2650949788 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3ca80a2580 con 0x558877f464d0 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 10 client.17280 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 20 client.17280 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f2df60 0x5588783292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 4 client.17280 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558878283990 con 0x558877f2df60 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.603+0000 7f3caffff640 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f2df60 0x5588783292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.604+0000 7f3c9dffb640 10 client.17280 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.604+0000 7f3caffff640 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f2df60 0x5588783292b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90677e26337a48a6 server_cookie=d7028f637620ca4c in_seq=0 out_seq=0 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2650949788 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558878283990 con 0x558877f2df60 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3c9dffb640 10 client.17280 handle_command_reply: tid=0 2024-04-25T14:05:44.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 client.17280 shutdown 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f464d0 msgr2=0x55887831b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f464d0 0x55887831b590 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80a3e80 tx=0x7f3ca80a6aa0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 shutdown_connections 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f2df60 0x5588783292b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f6f390 0x558878322a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f4e6b0 0x55887831bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3c9dffb640 0 client.17280 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2650949788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f464d0 0x55887831b590 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 >> 172.21.15.78:0/2650949788 conn(0x558877f634e0 msgr2=0x55887830ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 shutdown_connections 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 1 -- 172.21.15.78:0/2650949788 wait complete. 2024-04-25T14:05:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.605+0000 7f3cc3879740 20 client.17280 trim_cache size 0 max 16384 2024-04-25T14:05:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 Processor -- start 2024-04-25T14:05:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 -- start start 2024-04-25T14:05:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f2df60 0x558878310530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x55887832ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f4e6b0 0x55887832efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb81afd10 con 0x558877f2df60 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb81ad5e0 con 0x558877f464d0 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.618+0000 7f3cc3879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb81ab430 con 0x558877f4e6b0 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f2df60 0x558878310530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f2df60 0x558878310530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55446/0 (socket says 172.21.15.78:55446) 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 -- 172.21.15.78:0/3473192237 learned_addr learned my addr 172.21.15.78:0/3473192237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3caffff640 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x55887832ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 -- 172.21.15.78:0/3473192237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f4e6b0 msgr2=0x55887832efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f4e6b0 0x55887832efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 -- 172.21.15.78:0/3473192237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 msgr2=0x55887832ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x55887832ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 -- 172.21.15.78:0/3473192237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558878310a70 con 0x558877f2df60 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3caffff640 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x55887832ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3cb659c640 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f2df60 0x558878310530 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80debc0 tx=0x7f3ca80a2a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be32267fa4357149 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3c9ffff640 1 -- 172.21.15.78:0/3473192237 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8021050 con 0x558877f2df60 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3c9ffff640 1 -- 172.21.15.78:0/3473192237 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca8009de0 con 0x558877f2df60 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.619+0000 7f3c9ffff640 1 -- 172.21.15.78:0/3473192237 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80af5e0 con 0x558877f2df60 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 -- 172.21.15.78:0/3473192237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f2df60 msgr2=0x558878310530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f2df60 0x558878310530 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80debc0 tx=0x7f3ca80a2a30 comp rx=0 tx=0).stop 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 -- 172.21.15.78:0/3473192237 shutdown_connections 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f4e6b0 0x55887832efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x55887832ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 --2- 172.21.15.78:0/3473192237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f2df60 0x558878310530 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 -- 172.21.15.78:0/3473192237 >> 172.21.15.78:0/3473192237 conn(0x558877f634e0 msgr2=0x55887830fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 -- 172.21.15.78:0/3473192237 shutdown_connections 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 -- 172.21.15.78:0/3473192237 wait complete. 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 -- start start 2024-04-25T14:05:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 Processor -- start 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.620+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f2df60 0x558878320a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x558878320f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x5588783214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb81abaa0 con 0x558877f4e6b0 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb81acf10 con 0x558877f464d0 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb81ac170 con 0x558877f2df60 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 10 client.0 _ll_register_callbacks cb 0x7f3ca8005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x5588783214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x5588783214b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55458/0 (socket says 172.21.15.78:55458) 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 -- 172.21.15.78:0/2039236251 learned_addr learned my addr 172.21.15.78:0/2039236251 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3caffff640 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x558878320f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f2df60 0x558878320a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 -- 172.21.15.78:0/2039236251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f2df60 msgr2=0x558878320a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f2df60 0x558878320a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 -- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 msgr2=0x558878320f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x558878320f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 -- 172.21.15.78:0/2039236251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887838f690 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cb6d9d640 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x5588783214b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0012960 tx=0x7f3cb0012d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb75892bf5bb97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2039236251 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb001dcf0 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca8009de0 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2039236251 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3cb0027470 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2039236251 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3cb001de90 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 10 client.17286 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.621+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5588783909a0 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2039236251 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca8009de0 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2039236251 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3cb001e980 con 0x558877f4e6b0 2024-04-25T14:05:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3cc3879740 10 client.17286 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3cc3879740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3cc3879740 20 client.17286 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f6f390 0x558878393370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3cc3879740 4 client.17286 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55887827d800 con 0x558877f6f390 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.622+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f6f390 0x558878393370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3c9dffb640 10 client.17286 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cb659c640 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f6f390 0x558878393370 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3986f5e0b0d9083 server_cookie=fafb924748983cb in_seq=0 out_seq=0 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3c9dffb640 1 -- 172.21.15.78:0/2039236251 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55887827d800 con 0x558877f6f390 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3c9dffb640 10 client.17286 handle_command_reply: tid=0 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 client.17286 shutdown 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 msgr2=0x5588783214b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x5588783214b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0012960 tx=0x7f3cb0012d60 comp rx=0 tx=0).stop 2024-04-25T14:05:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 shutdown_connections 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558877f4e6b0 0x5588783214b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558877f464d0 0x558878320f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558877f6f390 0x558878393370 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 --2- 172.21.15.78:0/2039236251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558877f2df60 0x558878320a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3c9dffb640 0 client.17286 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 >> 172.21.15.78:0/2039236251 conn(0x558877f634e0 msgr2=0x558878319cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 shutdown_connections 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 1 -- 172.21.15.78:0/2039236251 wait complete. 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.623+0000 7f3cc3879740 20 client.17286 trim_cache size 0 max 16384 2024-04-25T14:05:44.630 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:44.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3ca40afaf0 msgr2=0x7f3ca40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3ca40afaf0 0x7f3ca40b2000 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x558878312860 tx=0x7f3cb0004a40 comp rx=0 tx=0).stop 2024-04-25T14:05:44.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8103af0 msgr2=0x7f3cb819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:44.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8103af0 0x7f3cb819c4a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8005d00 tx=0x7f3ca800dde0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 shutdown_connections 2024-04-25T14:05:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cb8109a30 0x7f3cb81a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb8104d30 0x7f3cb819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3ca40afaf0 0x7f3ca40b2000 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 --2- 172.21.15.78:0/2441074327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8103af0 0x7f3cb819c4a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 >> 172.21.15.78:0/2441074327 conn(0x7f3cb80fece0 msgr2=0x7f3cb80ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 shutdown_connections 2024-04-25T14:05:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:44.624+0000 7f3cbc9c5640 1 -- 172.21.15.78:0/2441074327 wait complete. 2024-04-25T14:05:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:44 smithi129 ceph-mon[32569]: pgmap v534: 97 pgs: 97 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-25T14:05:44.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:44 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2195653244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:45.635 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:05:45.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:45.635 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:05:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/2856998875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 msgr2=0x7ff2cc10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/2856998875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc10abb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009ca0 tx=0x7ff2c402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/2856998875 shutdown_connections 2024-04-25T14:05:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/2856998875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc10abb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/2856998875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2cc103930 0x7ff2cc103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/2856998875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2cc102f50 0x7ff2cc103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/2856998875 >> 172.21.15.78:0/2856998875 conn(0x7ff2cc0fd350 msgr2=0x7ff2cc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/2856998875 shutdown_connections 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.764+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/2856998875 wait complete. 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 Processor -- start 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 -- start start 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2cc102f50 0x7ff2cc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2cc103930 0x7ff2cc19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10f650 con 0x7ff2cc104470 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10f4d0 con 0x7ff2cc103930 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d2cd1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10f7d0 con 0x7ff2cc102f50 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d126e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d126e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc1a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55464/0 (socket says 172.21.15.78:55464) 2024-04-25T14:05:45.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d126e640 1 -- 172.21.15.78:0/135260909 learned_addr learned my addr 172.21.15.78:0/135260909 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2cc103930 0x7ff2cc19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.765+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2cc102f50 0x7ff2cc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d126e640 1 -- 172.21.15.78:0/135260909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2cc102f50 msgr2=0x7ff2cc19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d126e640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2cc102f50 0x7ff2cc19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d126e640 1 -- 172.21.15.78:0/135260909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2cc103930 msgr2=0x7ff2cc19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d126e640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2cc103930 0x7ff2cc19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d126e640 1 -- 172.21.15.78:0/135260909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc1a1d30 con 0x7ff2cc104470 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d126e640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc1a16c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097b0 tx=0x7ff2c4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2c9ffb640 1 -- 172.21.15.78:0/135260909 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4003bf0 con 0x7ff2cc104470 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2c9ffb640 1 -- 172.21.15.78:0/135260909 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c4003d90 con 0x7ff2cc104470 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc1a1fc0 con 0x7ff2cc104470 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2c9ffb640 1 -- 172.21.15.78:0/135260909 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c402fc30 con 0x7ff2cc104470 2024-04-25T14:05:45.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.766+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc1a2480 con 0x7ff2cc104470 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 Processor -- start 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2c9ffb640 1 -- 172.21.15.78:0/135260909 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff2c4004040 con 0x7ff2cc104470 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 -- start start 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa4b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa4ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa52920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f9b0b00 con 0x55ee2f689730 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f94c920 con 0x55ee2f6aa410 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f9acac0 con 0x55ee2f681550 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa52920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa52920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55480/0 (socket says 172.21.15.78:55480) 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2cbfff640 1 -- 172.21.15.78:0/923004501 learned_addr learned my addr 172.21.15.78:0/923004501 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa4ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa4b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2c9ffb640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff2bc0afb70 0x7ff2bc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2cbfff640 1 -- 172.21.15.78:0/923004501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 msgr2=0x55ee2fa4ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa4ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.769+0000 7ff2cbfff640 1 -- 172.21.15.78:0/923004501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 msgr2=0x55ee2fa4b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa4b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2cbfff640 1 -- 172.21.15.78:0/923004501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2f78a7e0 con 0x55ee2f689730 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa4ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2d126e640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff2bc0afb70 0x7ff2bc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa52920 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80058a0 tx=0x7ff2b800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733fa7bde33e7562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2c9ffb640 1 -- 172.21.15.78:0/135260909 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff2c4100320 con 0x7ff2cc104470 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2b6ffd640 1 -- 172.21.15.78:0/923004501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8012d00 con 0x55ee2f689730 2024-04-25T14:05:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2b6ffd640 1 -- 172.21.15.78:0/923004501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8010c20 con 0x55ee2f689730 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.770+0000 7ff2b6ffd640 1 -- 172.21.15.78:0/923004501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b801e9a0 con 0x55ee2f689730 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/923004501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 msgr2=0x55ee2fa52920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d126e640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff2bc0afb70 0x7ff2bc0b2080 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40a2e60 tx=0x7ff2c40f6ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa52920 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80058a0 tx=0x7ff2b800b7c0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/923004501 shutdown_connections 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa52920 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa4ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/923004501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa4b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/923004501 >> 172.21.15.78:0/923004501 conn(0x55ee2f69e560 msgr2=0x55ee2fa46e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/923004501 shutdown_connections 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/923004501 wait complete. 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 -- start start 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.771+0000 7ff2d9b7f740 1 Processor -- start 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee2fa53370 con 0x55ee2f689730 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f94a5d0 con 0x55ee2f6aa410 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ee2fa534f0 con 0x55ee2f681550 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d9b7f740 10 client.0 _ll_register_callbacks cb 0x55ee2f938080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55496/0 (socket says 172.21.15.78:55496) 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 -- 172.21.15.78:0/2311408836 learned_addr learned my addr 172.21.15.78:0/2311408836 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 -- 172.21.15.78:0/2311408836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 msgr2=0x55ee2fa552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 -- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 msgr2=0x55ee2fa5c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 -- 172.21.15.78:0/2311408836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2f7eaf70 con 0x55ee2f689730 2024-04-25T14:05:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa557e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0012700 tx=0x7ff2c0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42e54d74ad29966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.772+0000 7ff2abfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2abfff640 1 -- 172.21.15.78:0/2311408836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0011bb0 con 0x55ee2f689730 2024-04-25T14:05:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2abfff640 1 -- 172.21.15.78:0/2311408836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff2c0011d50 con 0x55ee2f689730 2024-04-25T14:05:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b8010c20 con 0x55ee2f689730 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2abfff640 1 -- 172.21.15.78:0/2311408836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff2c0004030 con 0x55ee2f689730 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 10 client.17298 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ee2f682170 con 0x55ee2f689730 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2abfff640 1 -- 172.21.15.78:0/2311408836 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8010c20 con 0x55ee2f689730 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2abfff640 1 -- 172.21.15.78:0/2311408836 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff2c001c460 con 0x55ee2f689730 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 10 client.17298 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 20 client.17298 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f6691f0 0x55ee2fa62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 4 client.17298 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee2f9be710 con 0x55ee2f6691f0 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.773+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f6691f0 0x55ee2fa62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.774+0000 7ff2abfff640 10 client.17298 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.774+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f6691f0 0x55ee2fa62cc0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427cdff59eb4a843 server_cookie=e1a6199b39217166 in_seq=0 out_seq=0 2024-04-25T14:05:45.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2abfff640 1 -- 172.21.15.78:0/2311408836 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee2f9be710 con 0x55ee2f6691f0 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2abfff640 10 client.17298 handle_command_reply: tid=0 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 client.17298 shutdown 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 msgr2=0x55ee2fa557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa557e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0012700 tx=0x7ff2c0012bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 shutdown_connections 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f689730 0x55ee2fa557e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f6691f0 0x55ee2fa62cc0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2311408836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 >> 172.21.15.78:0/2311408836 conn(0x55ee2f69e560 msgr2=0x55ee2fa45ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2abfff640 0 client.17298 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 shutdown_connections 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2311408836 wait complete. 2024-04-25T14:05:45.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.775+0000 7ff2d9b7f740 20 client.17298 trim_cache size 0 max 16384 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 Processor -- start 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 -- start start 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa53fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f689730 0x55ee2fa67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1adb20 con 0x55ee2f6aa410 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1ab3f0 con 0x55ee2f689730 2024-04-25T14:05:45.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1a9240 con 0x55ee2f681550 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55498/0 (socket says 172.21.15.78:55498) 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 -- 172.21.15.78:0/4259342162 learned_addr learned my addr 172.21.15.78:0/4259342162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa53fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f689730 0x55ee2fa67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 -- 172.21.15.78:0/4259342162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 msgr2=0x55ee2fa53fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa53fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 -- 172.21.15.78:0/4259342162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f689730 msgr2=0x55ee2fa67470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f689730 0x55ee2fa67470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 -- 172.21.15.78:0/4259342162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2fa50120 con 0x55ee2f6aa410 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f689730 0x55ee2fa67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.793+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa679b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8001eb0 tx=0x7ff2b800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a39cd7f88e8b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2b67fc640 1 -- 172.21.15.78:0/4259342162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8032070 con 0x55ee2f6aa410 2024-04-25T14:05:45.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2b67fc640 1 -- 172.21.15.78:0/4259342162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8016070 con 0x55ee2f6aa410 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2b67fc640 1 -- 172.21.15.78:0/4259342162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b802c450 con 0x55ee2f6aa410 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/4259342162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 msgr2=0x55ee2fa679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa679b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8001eb0 tx=0x7ff2b800bc20 comp rx=0 tx=0).stop 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/4259342162 shutdown_connections 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa679b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f689730 0x55ee2fa67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/4259342162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f681550 0x55ee2fa53fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/4259342162 >> 172.21.15.78:0/4259342162 conn(0x55ee2f69e560 msgr2=0x55ee2fa538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/4259342162 shutdown_connections 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.794+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/4259342162 wait complete. 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 -- start start 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 Processor -- start 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f681550 0x55ee2fa597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f689730 0x55ee2fa59d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1a98b0 con 0x55ee2f6aa410 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1aad20 con 0x55ee2f681550 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1a9f80 con 0x55ee2f689730 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d9b7f740 10 client.0 _ll_register_callbacks cb 0x7ff2c0009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55504/0 (socket says 172.21.15.78:55504) 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2cbfff640 1 -- 172.21.15.78:0/2888882775 learned_addr learned my addr 172.21.15.78:0/2888882775 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.795+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f689730 0x55ee2fa59d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f681550 0x55ee2fa597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2cbfff640 1 -- 172.21.15.78:0/2888882775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f689730 msgr2=0x55ee2fa59d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f689730 0x55ee2fa59d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2cbfff640 1 -- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f681550 msgr2=0x55ee2fa597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f681550 0x55ee2fa597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2cbfff640 1 -- 172.21.15.78:0/2888882775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2fa5b180 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2cbfff640 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5abc0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800bbf0 tx=0x7ff2b800bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455f4af6d0e91c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2abfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2abfff640 1 -- 172.21.15.78:0/2888882775 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8032070 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2abfff640 1 -- 172.21.15.78:0/2888882775 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff2b802c650 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b8016070 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2abfff640 1 -- 172.21.15.78:0/2888882775 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff2b802c850 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2d9b7f740 10 client.17304 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ee2facb120 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.796+0000 7ff2abfff640 1 -- 172.21.15.78:0/2888882775 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b8016070 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2abfff640 1 -- 172.21.15.78:0/2888882775 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff2b80049a0 con 0x55ee2f6aa410 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d9b7f740 10 client.17304 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:45.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d9b7f740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d9b7f740 20 client.17304 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f63e630 0x55ee2facd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d9b7f740 4 client.17304 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee2f9b8580 con 0x55ee2f63e630 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f63e630 0x55ee2facd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2abfff640 10 client.17304 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.797+0000 7ff2d0a6d640 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f63e630 0x55ee2facd8d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a812f0ec46c521cf server_cookie=8b3f6e7035db96c0 in_seq=0 out_seq=0 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2abfff640 1 -- 172.21.15.78:0/2888882775 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee2f9b8580 con 0x55ee2f63e630 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2abfff640 10 client.17304 handle_command_reply: tid=0 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 client.17304 shutdown 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 msgr2=0x55ee2fa5abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5abc0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800bbf0 tx=0x7ff2b800bad0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 shutdown_connections 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee2f6aa410 0x55ee2fa5abc0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ee2f63e630 0x55ee2facd8d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ee2f689730 0x55ee2fa59d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 --2- 172.21.15.78:0/2888882775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ee2f681550 0x55ee2fa597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 >> 172.21.15.78:0/2888882775 conn(0x55ee2f69e560 msgr2=0x55ee2fa49c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.798+0000 7ff2abfff640 0 client.17304 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 shutdown_connections 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d9b7f740 1 -- 172.21.15.78:0/2888882775 wait complete. 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d9b7f740 20 client.17304 trim_cache size 0 max 16384 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:05:45.805 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e8c67d6a-8118-4a01-899a-5d3bb635e3c5", 2024-04-25T14:05:45.806 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:05:45.806 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:45.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff2bc0afb70 msgr2=0x7ff2bc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff2bc0afb70 0x7ff2bc0b2080 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40a2e60 tx=0x7ff2c40f6ee0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 msgr2=0x7ff2cc1a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc1a16c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097b0 tx=0x7ff2c4031840 comp rx=0 tx=0).stop 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 shutdown_connections 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff2bc0afb70 0x7ff2bc0b2080 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2cc104470 0x7ff2cc1a16c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2cc103930 0x7ff2cc19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 --2- 172.21.15.78:0/135260909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2cc102f50 0x7ff2cc19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 >> 172.21.15.78:0/135260909 conn(0x7ff2cc0fd350 msgr2=0x7ff2cc106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 shutdown_connections 2024-04-25T14:05:45.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.799+0000 7ff2d2cd1640 1 -- 172.21.15.78:0/135260909 wait complete. 2024-04-25T14:05:45.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:45.810 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 -- 172.21.15.78:0/2311807960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0103b00 msgr2=0x7fa9a0104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 --2- 172.21.15.78:0/2311807960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0103b00 0x7fa9a0104800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa990009900 tx=0x7fa99002a410 comp rx=0 tx=0).stop 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 -- 172.21.15.78:0/2311807960 shutdown_connections 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 --2- 172.21.15.78:0/2311807960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9a0109a40 0x7fa9a010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 --2- 172.21.15.78:0/2311807960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0104d40 0x7fa9a0105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 --2- 172.21.15.78:0/2311807960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0103b00 0x7fa9a0104800 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 -- 172.21.15.78:0/2311807960 >> 172.21.15.78:0/2311807960 conn(0x7fa9a00fecf0 msgr2=0x7fa9a01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 -- 172.21.15.78:0/2311807960 shutdown_connections 2024-04-25T14:05:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 -- 172.21.15.78:0/2311807960 wait complete. 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 Processor -- start 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.921+0000 7fa9a6449640 1 -- start start 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a6449640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9a0103b00 0x7fa9a019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a6449640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0104d40 0x7fa9a019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a6449640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0109a40 0x7fa9a01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a6449640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a010f600 con 0x7fa9a0104d40 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a6449640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a010f480 con 0x7fa9a0109a40 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a6449640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a010f780 con 0x7fa9a0103b00 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0104d40 0x7fa9a019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0104d40 0x7fa9a019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55522/0 (socket says 172.21.15.78:55522) 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 -- 172.21.15.78:0/385699304 learned_addr learned my addr 172.21.15.78:0/385699304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0109a40 0x7fa9a01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99ffff640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9a0103b00 0x7fa9a019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 -- 172.21.15.78:0/385699304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9a0103b00 msgr2=0x7fa9a019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9a0103b00 0x7fa9a019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 -- 172.21.15.78:0/385699304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0109a40 msgr2=0x7fa9a01a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0109a40 0x7fa9a01a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 -- 172.21.15.78:0/385699304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a01a1d50 con 0x7fa9a0104d40 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99ffff640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9a0103b00 0x7fa9a019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0104d40 0x7fa9a019a750 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d950 tx=0x7fa99400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99d7fa640 1 -- 172.21.15.78:0/385699304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994017030 con 0x7fa9a0104d40 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99d7fa640 1 -- 172.21.15.78:0/385699304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa994004020 con 0x7fa9a0104d40 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa99d7fa640 1 -- 172.21.15.78:0/385699304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940047a0 con 0x7fa9a0104d40 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.922+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01a1fe0 con 0x7fa9a0104d40 2024-04-25T14:05:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.923+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a01a2520 con 0x7fa9a0104d40 2024-04-25T14:05:45.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 Processor -- start 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 -- start start 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa99d7fa640 1 -- 172.21.15.78:0/385699304 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa994009a30 con 0x7fa9a0104d40 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca6766b0 0x55d5caa39c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d5ca99dd80 con 0x55d5ca66e4d0 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d5ca9578b0 con 0x55d5ca697390 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d5ca999d40 con 0x55d5ca6766b0 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.925+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa38fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55528/0 (socket says 172.21.15.78:55528) 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 -- 172.21.15.78:0/3531418144 learned_addr learned my addr 172.21.15.78:0/3531418144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca6766b0 0x55d5caa39c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 -- 172.21.15.78:0/3531418144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca6766b0 msgr2=0x55d5caa39c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca6766b0 0x55d5caa39c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 -- 172.21.15.78:0/3531418144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 msgr2=0x55d5caa38a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa38a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 -- 172.21.15.78:0/3531418144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ca928a40 con 0x55d5ca66e4d0 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99d7fa640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9880afb70 0x7fa9880b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9880afb70 0x7fa9880b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa38fe0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa990033000 tx=0x7fa990032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1060ddb74d754fff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa99d7fa640 1 -- 172.21.15.78:0/385699304 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa9940de700 con 0x7fa9a0104d40 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa9867fc640 1 -- 172.21.15.78:0/3531418144 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99003f030 con 0x55d5ca66e4d0 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa9867fc640 1 -- 172.21.15.78:0/3531418144 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa990032db0 con 0x55d5ca66e4d0 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.926+0000 7fa9867fc640 1 -- 172.21.15.78:0/3531418144 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990035070 con 0x55d5ca66e4d0 2024-04-25T14:05:45.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3531418144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 msgr2=0x55d5caa38fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9880afb70 0x7fa9880b2080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa998009fd0 tx=0x7fa998009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa38fe0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa990033000 tx=0x7fa990032b90 comp rx=0 tx=0).stop 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3531418144 shutdown_connections 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca6766b0 0x55d5caa39c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa38fe0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3531418144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3531418144 >> 172.21.15.78:0/3531418144 conn(0x55d5ca68b4e0 msgr2=0x55d5caa34270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3531418144 shutdown_connections 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.927+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3531418144 wait complete. 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 -- start start 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 Processor -- start 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 0x55d5caa42920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa42e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa49dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d5caa3a720 con 0x55d5ca6766b0 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d5ca924d10 con 0x55d5ca697390 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d5caa3a8a0 con 0x55d5ca66e4d0 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9ad3d1740 10 client.0 _ll_register_callbacks cb 0x55d5ca951210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa42e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa42e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55532/0 (socket says 172.21.15.78:55532) 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3082159553 learned_addr learned my addr 172.21.15.78:0/3082159553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa49dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.928+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 0x55d5caa42920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3082159553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 msgr2=0x55d5caa42920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 0x55d5caa42920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 msgr2=0x55d5caa49dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa49dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3082159553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ca7797d0 con 0x55d5ca6766b0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa49dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa42e60 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa9940a3de0 tx=0x7fa9940a62e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fbb13efc752272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa97f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3082159553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940a6b70 con 0x55d5ca6766b0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3082159553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa9940041d0 con 0x55d5ca6766b0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa990032db0 con 0x55d5ca6766b0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3082159553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa9940aaf20 con 0x55d5ca6766b0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9ad3d1740 10 client.17316 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d5ca6af3f0 con 0x55d5ca6766b0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3082159553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa990032db0 con 0x55d5ca6766b0 2024-04-25T14:05:45.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3082159553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa9940e0c50 con 0x55d5ca6766b0 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9ad3d1740 10 client.17316 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9ad3d1740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9ad3d1740 20 client.17316 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.929+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca655f60 0x55d5caa50370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.930+0000 7fa9ad3d1740 4 client.17316 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.930+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ca9ab990 con 0x55d5ca655f60 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.930+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca655f60 0x55d5caa50370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.930+0000 7fa97f7fe640 10 client.17316 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.930+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca655f60 0x55d5caa50370 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21463aee8ea2171 server_cookie=54c21d7555310201 in_seq=0 out_seq=0 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3082159553 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d5ca9ab990 con 0x55d5ca655f60 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa97f7fe640 10 client.17316 handle_command_reply: tid=0 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 client.17316 shutdown 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 msgr2=0x55d5caa42e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa42e60 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa9940a3de0 tx=0x7fa9940a62e0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 shutdown_connections 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa49dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa42e60 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca655f60 0x55d5caa50370 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3082159553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 0x55d5caa42920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa97f7fe640 0 client.17316 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 >> 172.21.15.78:0/3082159553 conn(0x55d5ca68b4e0 msgr2=0x55d5caa37180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 shutdown_connections 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3082159553 wait complete. 2024-04-25T14:05:45.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.931+0000 7fa9ad3d1740 20 client.17316 trim_cache size 0 max 16384 2024-04-25T14:05:45.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 Processor -- start 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 -- start start 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 0x55d5caa41610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa49ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa4a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a01adc00 con 0x55d5ca6766b0 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a01ab4d0 con 0x55d5ca697390 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a01a9320 con 0x55d5ca66e4d0 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa49ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa49ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55538/0 (socket says 172.21.15.78:55538) 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.949+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3794570886 learned_addr learned my addr 172.21.15.78:0/3794570886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3794570886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 msgr2=0x55d5caa41610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3794570886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 0x55d5caa41610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3794570886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 msgr2=0x55d5caa4a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3794570886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa4a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa99f7fe640 1 -- 172.21.15.78:0/3794570886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ca66f0f0 con 0x55d5ca6766b0 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3794570886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa49ce0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa994000c00 tx=0x7fa9940a6410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325065c42f516d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa985ffb640 1 -- 172.21.15.78:0/3794570886 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940a6b70 con 0x55d5ca6766b0 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa985ffb640 1 -- 172.21.15.78:0/3794570886 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9940a6680 con 0x55d5ca6766b0 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.950+0000 7fa985ffb640 1 -- 172.21.15.78:0/3794570886 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940a2e00 con 0x55d5ca6766b0 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3794570886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 msgr2=0x55d5caa49ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3794570886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa49ce0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa994000c00 tx=0x7fa9940a6410 comp rx=0 tx=0).stop 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3794570886 shutdown_connections 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3794570886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca697390 0x55d5caa4a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3794570886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca6766b0 0x55d5caa49ce0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3794570886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca66e4d0 0x55d5caa41610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3794570886 >> 172.21.15.78:0/3794570886 conn(0x55d5ca68b4e0 msgr2=0x55d5caa40f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3794570886 shutdown_connections 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3794570886 wait complete. 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- start start 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 Processor -- start 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca6766b0 0x55d5caa46ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca697390 0x55d5caa47410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a01a9990 con 0x55d5ca66e4d0 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a01aae00 con 0x55d5ca6766b0 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a01aa060 con 0x55d5ca697390 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa9ad3d1740 10 client.0 _ll_register_callbacks cb 0x7fa9940a6940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:45.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa4a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55542/0 (socket says 172.21.15.78:55542) 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.951+0000 7fa99ffff640 1 -- 172.21.15.78:0/3379801826 learned_addr learned my addr 172.21.15.78:0/3379801826 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca6766b0 0x55d5caa46ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca697390 0x55d5caa47410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa99ffff640 1 -- 172.21.15.78:0/3379801826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca697390 msgr2=0x55d5caa47410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca697390 0x55d5caa47410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa99ffff640 1 -- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca6766b0 msgr2=0x55d5caa46ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca6766b0 0x55d5caa46ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa99ffff640 1 -- 172.21.15.78:0/3379801826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ca95e4b0 con 0x55d5ca66e4d0 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa9a49e6640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca697390 0x55d5caa47410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa99ffff640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa4a2e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa990004650 tx=0x7fa9900058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0450fcc2b0ee84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa97f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:45.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3379801826 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99003f040 con 0x55d5ca66e4d0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3379801826 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa990032040 con 0x55d5ca66e4d0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9940a6680 con 0x55d5ca66e4d0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3379801826 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa990035040 con 0x55d5ca66e4d0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa9ad3d1740 10 client.17322 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d5caab75e0 con 0x55d5ca66e4d0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.952+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3379801826 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9940a6680 con 0x55d5ca66e4d0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3379801826 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa99003b460 con 0x55d5ca66e4d0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa9ad3d1740 10 client.17322 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa9ad3d1740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa9ad3d1740 20 client.17322 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca62b3a0 0x55d5caab9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa9ad3d1740 4 client.17322 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d5ca9a5800 con 0x55d5ca62b3a0 2024-04-25T14:05:45.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca62b3a0 0x55d5caab9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa97f7fe640 10 client.17322 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa99f7fe640 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca62b3a0 0x55d5caab9c70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1a4076ce965589 server_cookie=e4cefa590df3a590 in_seq=0 out_seq=0 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa97f7fe640 1 -- 172.21.15.78:0/3379801826 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d5ca9a5800 con 0x55d5ca62b3a0 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa97f7fe640 10 client.17322 handle_command_reply: tid=0 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.953+0000 7fa9ad3d1740 1 client.17322 shutdown 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 msgr2=0x55d5caa4a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa4a2e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa990004650 tx=0x7fa9900058f0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 shutdown_connections 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d5ca62b3a0 0x55d5caab9c70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5ca697390 0x55d5caa47410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d5ca6766b0 0x55d5caa46ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa97f7fe640 0 client.17322 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 --2- 172.21.15.78:0/3379801826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d5ca66e4d0 0x55d5caa4a2e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 >> 172.21.15.78:0/3379801826 conn(0x55d5ca68b4e0 msgr2=0x55d5caa33c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 shutdown_connections 2024-04-25T14:05:45.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 1 -- 172.21.15.78:0/3379801826 wait complete. 2024-04-25T14:05:45.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9ad3d1740 20 client.17322 trim_cache size 0 max 16384 2024-04-25T14:05:45.961 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:45.961 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:05:45.961 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:05:45.961 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9880afb70 msgr2=0x7fa9880b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9880afb70 0x7fa9880b2080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa998009fd0 tx=0x7fa998009500 comp rx=0 tx=0).stop 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0104d40 msgr2=0x7fa9a019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0104d40 0x7fa9a019a750 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d950 tx=0x7fa99400de10 comp rx=0 tx=0).stop 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 shutdown_connections 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a0109a40 0x7fa9a01a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa9880afb70 0x7fa9880b2080 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9a0104d40 0x7fa9a019a750 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 --2- 172.21.15.78:0/385699304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9a0103b00 0x7fa9a019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.954+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 >> 172.21.15.78:0/385699304 conn(0x7fa9a00fecf0 msgr2=0x7fa9a0105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.955+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 shutdown_connections 2024-04-25T14:05:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:45.955+0000 7fa9a6449640 1 -- 172.21.15.78:0/385699304 wait complete. 2024-04-25T14:05:45.965 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:05:45.965 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 -- 172.21.15.78:0/3269821852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220106f30 msgr2=0x7fe220109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 --2- 172.21.15.78:0/3269821852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220106f30 0x7fe220109350 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe214009900 tx=0x7fe21402a410 comp rx=0 tx=0).stop 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 -- 172.21.15.78:0/3269821852 shutdown_connections 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 --2- 172.21.15.78:0/3269821852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe220109890 0x7fe22010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 --2- 172.21.15.78:0/3269821852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220106f30 0x7fe220109350 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 --2- 172.21.15.78:0/3269821852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2201045d0 0x7fe2201069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 -- 172.21.15.78:0/3269821852 >> 172.21.15.78:0/3269821852 conn(0x7fe2200fdbf0 msgr2=0x7fe220100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 -- 172.21.15.78:0/3269821852 shutdown_connections 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.073+0000 7fe226896640 1 -- 172.21.15.78:0/3269821852 wait complete. 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 Processor -- start 2024-04-25T14:05:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 -- start start 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2201045d0 0x7fe22019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe220106f30 0x7fe22019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220109890 0x7fe2201a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010f5f0 con 0x7fe220109890 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010f470 con 0x7fe220106f30 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe226896640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010f770 con 0x7fe2201045d0 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220109890 0x7fe2201a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220109890 0x7fe2201a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55562/0 (socket says 172.21.15.78:55562) 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 -- 172.21.15.78:0/3150762928 learned_addr learned my addr 172.21.15.78:0/3150762928 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe21ffff640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2201045d0 0x7fe22019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 -- 172.21.15.78:0/3150762928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2201045d0 msgr2=0x7fe22019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2201045d0 0x7fe22019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 -- 172.21.15.78:0/3150762928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe220106f30 msgr2=0x7fe22019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe21f7fe640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe220106f30 0x7fe22019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe220106f30 0x7fe22019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.074+0000 7fe224e33640 1 -- 172.21.15.78:0/3150762928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2201a1dc0 con 0x7fe220109890 2024-04-25T14:05:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.075+0000 7fe224e33640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220109890 0x7fe2201a1750 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe21800e9d0 tx=0x7fe21800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.075+0000 7fe21d7fa640 1 -- 172.21.15.78:0/3150762928 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21800cd80 con 0x7fe220109890 2024-04-25T14:05:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.075+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2201a20b0 con 0x7fe220109890 2024-04-25T14:05:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.075+0000 7fe21d7fa640 1 -- 172.21.15.78:0/3150762928 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe218005020 con 0x7fe220109890 2024-04-25T14:05:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.075+0000 7fe21d7fa640 1 -- 172.21.15.78:0/3150762928 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180116d0 con 0x7fe220109890 2024-04-25T14:05:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.075+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a2600 con 0x7fe220109890 2024-04-25T14:05:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.076+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2200ff710 con 0x7fe220109890 2024-04-25T14:05:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.077+0000 7fe21d7fa640 1 -- 172.21.15.78:0/3150762928 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe2180051d0 con 0x7fe220109890 2024-04-25T14:05:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.078+0000 7fe21d7fa640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe2080afaf0 0x7fe2080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.078+0000 7fe21ffff640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe2080afaf0 0x7fe2080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.078+0000 7fe21d7fa640 1 -- 172.21.15.78:0/3150762928 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe2180dee50 con 0x7fe220109890 2024-04-25T14:05:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.078+0000 7fe21ffff640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe2080afaf0 0x7fe2080b2000 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe2200757f0 tx=0x7fe210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.082+0000 7fe21d7fa640 1 -- 172.21.15.78:0/3150762928 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe218016030 con 0x7fe220109890 2024-04-25T14:05:46.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.298+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2200637a0 con 0x7fe220109890 2024-04-25T14:05:46.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.298+0000 7fe21d7fa640 1 -- 172.21.15.78:0/3150762928 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fe2180a7a20 con 0x7fe220109890 2024-04-25T14:05:46.298 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:05:46.298 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:05:46.299 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:05:46.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe2080afaf0 msgr2=0x7fe2080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe2080afaf0 0x7fe2080b2000 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe2200757f0 tx=0x7fe210005f90 comp rx=0 tx=0).stop 2024-04-25T14:05:46.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220109890 msgr2=0x7fe2201a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220109890 0x7fe2201a1750 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe21800e9d0 tx=0x7fe21800ee90 comp rx=0 tx=0).stop 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 shutdown_connections 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe2080afaf0 0x7fe2080b2000 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe220109890 0x7fe2201a1750 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe220106f30 0x7fe22019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.300+0000 7fe226896640 1 --2- 172.21.15.78:0/3150762928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2201045d0 0x7fe22019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.301+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 >> 172.21.15.78:0/3150762928 conn(0x7fe2200fdbf0 msgr2=0x7fe220105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.301+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 shutdown_connections 2024-04-25T14:05:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.301+0000 7fe226896640 1 -- 172.21.15.78:0/3150762928 wait complete. 2024-04-25T14:05:46.312 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 -- 172.21.15.78:0/1837548453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 msgr2=0x7f19c810bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/1837548453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c810bca0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009ca0 tx=0x7f19c002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 -- 172.21.15.78:0/1837548453 shutdown_connections 2024-04-25T14:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/1837548453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c810bca0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/1837548453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c8102370 0x7f19c8109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/1837548453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c8101e70 0x7f19c8106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 -- 172.21.15.78:0/1837548453 >> 172.21.15.78:0/1837548453 conn(0x7f19c80fd380 msgr2=0x7f19c80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 -- 172.21.15.78:0/1837548453 shutdown_connections 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.422+0000 7f19ccaf9640 1 -- 172.21.15.78:0/1837548453 wait complete. 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 Processor -- start 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 -- start start 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c8101e70 0x7f19c819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c8102370 0x7f19c819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c81a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19c810f600 con 0x7f19c8109790 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19c810f480 con 0x7f19c8102370 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19ccaf9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19c810f780 con 0x7f19c8101e70 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19c6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c81a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19c6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c81a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55590/0 (socket says 172.21.15.78:55590) 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19c6d9d640 1 -- 172.21.15.78:0/3315650411 learned_addr learned my addr 172.21.15.78:0/3315650411 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19c659c640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c8101e70 0x7f19c819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.423+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c8102370 0x7f19c819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19c6d9d640 1 -- 172.21.15.78:0/3315650411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c8101e70 msgr2=0x7f19c819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19c6d9d640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c8101e70 0x7f19c819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19c6d9d640 1 -- 172.21.15.78:0/3315650411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c8102370 msgr2=0x7f19c819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19c6d9d640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c8102370 0x7f19c819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19c6d9d640 1 -- 172.21.15.78:0/3315650411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c81a1be0 con 0x7f19c8109790 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19c6d9d640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c81a1620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f19c00097b0 tx=0x7f19c0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19b77fe640 1 -- 172.21.15.78:0/3315650411 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0004240 con 0x7f19c8109790 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19b77fe640 1 -- 172.21.15.78:0/3315650411 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19c002f380 con 0x7f19c8109790 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c81a1e70 con 0x7f19c8109790 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19b77fe640 1 -- 172.21.15.78:0/3315650411 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c002fc30 con 0x7f19c8109790 2024-04-25T14:05:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.424+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c81a22d0 con 0x7f19c8109790 2024-04-25T14:05:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.427+0000 7f19d39a9740 1 Processor -- start 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.427+0000 7f19b77fe640 1 -- 172.21.15.78:0/3315650411 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f19c00305c0 con 0x7f19c8109790 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.427+0000 7f19d39a9740 1 -- start start 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b71f390 0x56036bac1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6f64d0 0x56036bac1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bac2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19d39a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56036ba25d80 con 0x56036b71f390 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19d39a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56036b9dfa20 con 0x56036b6fe6b0 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19d39a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56036ba21d40 con 0x56036b6f64d0 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b71f390 0x56036bac1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b71f390 0x56036bac1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55602/0 (socket says 172.21.15.78:55602) 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 -- 172.21.15.78:0/375161766 learned_addr learned my addr 172.21.15.78:0/375161766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c659c640 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6f64d0 0x56036bac1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19b77fe640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b00afaf0 0x7f19b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bac2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c6d9d640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b00afaf0 0x7f19b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 -- 172.21.15.78:0/375161766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6f64d0 msgr2=0x56036bac1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6f64d0 0x56036bac1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 -- 172.21.15.78:0/375161766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 msgr2=0x56036bac2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bac2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 -- 172.21.15.78:0/375161766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036b9eac30 con 0x56036b71f390 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19b77fe640 1 -- 172.21.15.78:0/3315650411 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f19c0100400 con 0x7f19c8109790 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.428+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b71f390 0x56036bac1810 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0135b0 tx=0x7f19bc00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be192d95d5a0d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19b4ff9640 1 -- 172.21.15.78:0/375161766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc019030 con 0x56036b71f390 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19b4ff9640 1 -- 172.21.15.78:0/375161766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19bc011970 con 0x56036b71f390 2024-04-25T14:05:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19b4ff9640 1 -- 172.21.15.78:0/375161766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00f740 con 0x56036b71f390 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19d39a9740 1 -- 172.21.15.78:0/375161766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b71f390 msgr2=0x56036bac1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19d39a9740 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b71f390 0x56036bac1810 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0135b0 tx=0x7f19bc00b690 comp rx=0 tx=0).stop 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19c6d9d640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b00afaf0 0x7f19b00b2000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f19c00c8860 tx=0x7f19c0097390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19d39a9740 1 -- 172.21.15.78:0/375161766 shutdown_connections 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19d39a9740 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bac2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19d39a9740 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6f64d0 0x56036bac1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19d39a9740 1 --2- 172.21.15.78:0/375161766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b71f390 0x56036bac1810 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.429+0000 7f19d39a9740 1 -- 172.21.15.78:0/375161766 >> 172.21.15.78:0/375161766 conn(0x56036b7134e0 msgr2=0x56036babd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 -- 172.21.15.78:0/375161766 shutdown_connections 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 -- 172.21.15.78:0/375161766 wait complete. 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 -- start start 2024-04-25T14:05:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 Processor -- start 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036bacb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bacba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b71f390 0x56036bad29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56036bac3530 con 0x56036b6f64d0 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56036b9acdc0 con 0x56036b6fe6b0 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56036bac36b0 con 0x56036b71f390 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19d39a9740 10 client.0 _ll_register_callbacks cb 0x56036b9c8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036bacb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036bacb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55604/0 (socket says 172.21.15.78:55604) 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19c5d9b640 1 -- 172.21.15.78:0/2734261572 learned_addr learned my addr 172.21.15.78:0/2734261572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.430+0000 7f19c659c640 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bacba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b71f390 0x56036bad29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19c5d9b640 1 -- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b71f390 msgr2=0x56036bad29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b71f390 0x56036bad29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19c5d9b640 1 -- 172.21.15.78:0/2734261572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 msgr2=0x56036bacba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bacba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19c5d9b640 1 -- 172.21.15.78:0/2734261572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56036b7373f0 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036bacb530 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f19bc011b90 tx=0x7f19bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afe594e153a708a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19adffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19adffb640 1 -- 172.21.15.78:0/2734261572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc019040 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19adffb640 1 -- 172.21.15.78:0/2734261572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f19bc005040 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19bc011970 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19adffb640 1 -- 172.21.15.78:0/2734261572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f19bc005240 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19d39a9740 10 client.17340 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56036b8017d0 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.431+0000 7f19adffb640 1 -- 172.21.15.78:0/2734261572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19bc011970 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19adffb640 1 -- 172.21.15.78:0/2734261572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f19bc00f4c0 con 0x56036b6f64d0 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19d39a9740 10 client.17340 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19d39a9740 10 client.17340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19d39a9740 20 client.17340 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b6ddf60 0x56036bad9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19d39a9740 4 client.17340 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56036ba33990 con 0x56036b6ddf60 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19c659c640 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b6ddf60 0x56036bad9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19c659c640 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b6ddf60 0x56036bad9010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b0cd96a24f91828 server_cookie=f4bd019076733272 in_seq=0 out_seq=0 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.432+0000 7f19adffb640 10 client.17340 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.433+0000 7f19adffb640 1 -- 172.21.15.78:0/2734261572 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56036ba33990 con 0x56036b6ddf60 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.433+0000 7f19adffb640 10 client.17340 handle_command_reply: tid=0 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.433+0000 7f19d39a9740 1 client.17340 shutdown 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.433+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 msgr2=0x56036bacb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.433+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036bacb530 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f19bc011b90 tx=0x7f19bc00b8d0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 shutdown_connections 2024-04-25T14:05:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b6ddf60 0x56036bad9010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b71f390 0x56036bad29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6fe6b0 0x56036bacba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2734261572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036bacb530 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19adffb640 0 client.17340 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 >> 172.21.15.78:0/2734261572 conn(0x56036b7134e0 msgr2=0x56036babff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 shutdown_connections 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 1 -- 172.21.15.78:0/2734261572 wait complete. 2024-04-25T14:05:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.434+0000 7f19d39a9740 20 client.17340 trim_cache size 0 max 16384 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 Processor -- start 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 -- start start 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b7134e0 0x56036babca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036babcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 0x56036bacb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81ad960 con 0x56036b6f64d0 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81ab230 con 0x56036b7134e0 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19d39a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81a9080 con 0x56036b6fe6b0 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036babcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036babcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55610/0 (socket says 172.21.15.78:55610) 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 -- 172.21.15.78:0/2263741454 learned_addr learned my addr 172.21.15.78:0/2263741454 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b7134e0 0x56036babca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 -- 172.21.15.78:0/2263741454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 msgr2=0x56036bacb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 0x56036bacb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 -- 172.21.15.78:0/2263741454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b7134e0 msgr2=0x56036babca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b7134e0 0x56036babca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c659c640 1 -- 172.21.15.78:0/2263741454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036bac9610 con 0x56036b6f64d0 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.447+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b7134e0 0x56036babca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.448+0000 7f19c659c640 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036babcf90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x56036bac1530 tx=0x7f19b800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f75314b501c725 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.448+0000 7f19affff640 1 -- 172.21.15.78:0/2263741454 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8003c40 con 0x56036b6f64d0 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.448+0000 7f19affff640 1 -- 172.21.15.78:0/2263741454 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b8021070 con 0x56036b6f64d0 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.448+0000 7f19affff640 1 -- 172.21.15.78:0/2263741454 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8010990 con 0x56036b6f64d0 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.448+0000 7f19d39a9740 1 -- 172.21.15.78:0/2263741454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 msgr2=0x56036babcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.448+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036babcf90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x56036bac1530 tx=0x7f19b800cd00 comp rx=0 tx=0).stop 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- 172.21.15.78:0/2263741454 shutdown_connections 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 0x56036bacb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b6f64d0 0x56036babcf90 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 --2- 172.21.15.78:0/2263741454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b7134e0 0x56036babca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- 172.21.15.78:0/2263741454 >> 172.21.15.78:0/2263741454 conn(0x56036bad9340 msgr2=0x56036bac6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- 172.21.15.78:0/2263741454 shutdown_connections 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- 172.21.15.78:0/2263741454 wait complete. 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- start start 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 Processor -- start 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6f64d0 0x56036bacf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 0x56036bacfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b7134e0 0x56036bad09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81a96f0 con 0x56036b7134e0 2024-04-25T14:05:46.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81aab60 con 0x56036b6f64d0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19c81a9dc0 con 0x56036b6fe6b0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19d39a9740 10 client.0 _ll_register_callbacks cb 0x7f19b800c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b7134e0 0x56036bad09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19c5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b7134e0 0x56036bad09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55614/0 (socket says 172.21.15.78:55614) 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19c5d9b640 1 -- 172.21.15.78:0/438614575 learned_addr learned my addr 172.21.15.78:0/438614575 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6f64d0 0x56036bacf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19c5d9b640 1 -- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 msgr2=0x56036bacfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19c659c640 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 0x56036bacfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.449+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 0x56036bacfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19c5d9b640 1 -- 172.21.15.78:0/438614575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6f64d0 msgr2=0x56036bacf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6f64d0 0x56036bacf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19c5d9b640 1 -- 172.21.15.78:0/438614575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56036bad0fb0 con 0x56036b7134e0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19c5d9b640 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b7134e0 0x56036bad09f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00fad0 tx=0x7f19bc00b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac14b234d52f6006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19adffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19adffb640 1 -- 172.21.15.78:0/438614575 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc03e060 con 0x56036b7134e0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b8021070 con 0x56036b7134e0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19adffb640 1 -- 172.21.15.78:0/438614575 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f19bc011dc0 con 0x56036b7134e0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19adffb640 1 -- 172.21.15.78:0/438614575 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f19bc00f3c0 con 0x56036b7134e0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 10 client.17346 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56036bad21a0 con 0x56036b7134e0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19adffb640 1 -- 172.21.15.78:0/438614575 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b8021070 con 0x56036b7134e0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19adffb640 1 -- 172.21.15.78:0/438614575 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f19bc02e630 con 0x56036b7134e0 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 10 client.17346 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 20 client.17346 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b71f390 0x56036bb439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 4 client.17346 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.450+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56036ba2d800 con 0x56036b71f390 2024-04-25T14:05:46.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.451+0000 7f19c659c640 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b71f390 0x56036bb439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.451+0000 7f19c659c640 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b71f390 0x56036bb439b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f79b80fc25fb627 server_cookie=c6e120c6b0deb84 in_seq=0 out_seq=0 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.451+0000 7f19adffb640 10 client.17346 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.451+0000 7f19adffb640 1 -- 172.21.15.78:0/438614575 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56036ba2d800 con 0x56036b71f390 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.451+0000 7f19adffb640 10 client.17346 handle_command_reply: tid=0 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.451+0000 7f19d39a9740 1 client.17346 shutdown 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b7134e0 msgr2=0x56036bad09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b7134e0 0x56036bad09f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00fad0 tx=0x7f19bc00b5f0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 shutdown_connections 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x56036b71f390 0x56036bb439b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56036b7134e0 0x56036bad09f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19adffb640 0 client.17346 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56036b6fe6b0 0x56036bacfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 --2- 172.21.15.78:0/438614575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56036b6f64d0 0x56036bacf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 >> 172.21.15.78:0/438614575 conn(0x56036bad9340 msgr2=0x56036babc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 shutdown_connections 2024-04-25T14:05:46.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 1 -- 172.21.15.78:0/438614575 wait complete. 2024-04-25T14:05:46.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19d39a9740 20 client.17346 trim_cache size 0 max 16384 2024-04-25T14:05:46.458 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b00afaf0 msgr2=0x7f19b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b00afaf0 0x7f19b00b2000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f19c00c8860 tx=0x7f19c0097390 comp rx=0 tx=0).stop 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 msgr2=0x7f19c81a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c81a1620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f19c00097b0 tx=0x7f19c0003c20 comp rx=0 tx=0).stop 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 shutdown_connections 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19b00afaf0 0x7f19b00b2000 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c8109790 0x7f19c81a1620 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c8102370 0x7f19c819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 --2- 172.21.15.78:0/3315650411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19c8101e70 0x7f19c819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 >> 172.21.15.78:0/3315650411 conn(0x7f19c80fd380 msgr2=0x7f19c81053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 shutdown_connections 2024-04-25T14:05:46.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.452+0000 7f19ccaf9640 1 -- 172.21.15.78:0/3315650411 wait complete. 2024-04-25T14:05:46.463 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:05:46.482 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:46 smithi078 ceph-mon[29163]: pgmap v535: 97 pgs: 97 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:05:46.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:46 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:05:46.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:46 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:46.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:46 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:05:46.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:46 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:46.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:46 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:05:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:46 smithi112 ceph-mon[32617]: pgmap v535: 97 pgs: 97 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:05:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:46 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:05:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:46 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:46 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:05:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:46 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:46 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 -- 172.21.15.78:0/1148208736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c1043d0 msgr2=0x7f0f0c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 --2- 172.21.15.78:0/1148208736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c1043d0 0x7f0f0c102dd0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009900 tx=0x7f0f0002a410 comp rx=0 tx=0).stop 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 -- 172.21.15.78:0/1148208736 shutdown_connections 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 --2- 172.21.15.78:0/1148208736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f0c108a20 0x7f0f0c10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 --2- 172.21.15.78:0/1148208736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c1043d0 0x7f0f0c102dd0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 --2- 172.21.15.78:0/1148208736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f0c103ef0 0x7f0f0c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 -- 172.21.15.78:0/1148208736 >> 172.21.15.78:0/1148208736 conn(0x7f0f0c0fd350 msgr2=0x7f0f0c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 -- 172.21.15.78:0/1148208736 shutdown_connections 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.572+0000 7f0f12a42640 1 -- 172.21.15.78:0/1148208736 wait complete. 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 Processor -- start 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 -- start start 2024-04-25T14:05:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c103ef0 0x7f0f0c199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f0c108a20 0x7f0f0c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f0c19f280 0x7f0f0c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c101c30 con 0x7f0f0c103ef0 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c101ab0 con 0x7f0f0c108a20 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f12a42640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c101db0 con 0x7f0f0c19f280 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c103ef0 0x7f0f0c199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c103ef0 0x7f0f0c199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50056/0 (socket says 172.21.15.78:50056) 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.573+0000 7f0f0bfff640 1 -- 172.21.15.78:0/117328443 learned_addr learned my addr 172.21.15.78:0/117328443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f0c19f280 0x7f0f0c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0b7fe640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f0c108a20 0x7f0f0c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0bfff640 1 -- 172.21.15.78:0/117328443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f0c19f280 msgr2=0x7f0f0c1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f0c19f280 0x7f0f0c1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0bfff640 1 -- 172.21.15.78:0/117328443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f0c108a20 msgr2=0x7f0f0c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f0c108a20 0x7f0f0c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0bfff640 1 -- 172.21.15.78:0/117328443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c1a1c90 con 0x7f0f0c103ef0 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f0c19f280 0x7f0f0c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0b7fe640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f0c108a20 0x7f0f0c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c103ef0 0x7f0f0c199e00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c970 tx=0x7f0efc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f097fa640 1 -- 172.21.15.78:0/117328443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc014070 con 0x7f0f0c103ef0 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f097fa640 1 -- 172.21.15.78:0/117328443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc004440 con 0x7f0f0c103ef0 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f097fa640 1 -- 172.21.15.78:0/117328443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc003e50 con 0x7f0f0c103ef0 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c1a1f80 con 0x7f0f0c103ef0 2024-04-25T14:05:46.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.574+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1a23e0 con 0x7f0f0c103ef0 2024-04-25T14:05:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.576+0000 7f0f199c7740 1 Processor -- start 2024-04-25T14:05:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.576+0000 7f0f199c7740 1 -- start start 2024-04-25T14:05:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f097fa640 1 -- 172.21.15.78:0/117328443 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0efc017070 con 0x7f0f0c103ef0 2024-04-25T14:05:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033d1c390 0x55d0340be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033cf34d0 0x55d0340bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f199c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d034022d80 con 0x55d033cf34d0 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f199c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d033fdca20 con 0x55d033d1c390 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f199c7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d03401ed40 con 0x55d033cfb6b0 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033cf34d0 0x55d0340bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033cf34d0 0x55d0340bed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50068/0 (socket says 172.21.15.78:50068) 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 -- 172.21.15.78:0/1512646622 learned_addr learned my addr 172.21.15.78:0/1512646622 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0b7fe640 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033d1c390 0x55d0340be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 -- 172.21.15.78:0/1512646622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 msgr2=0x55d0340bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340bfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 -- 172.21.15.78:0/1512646622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033d1c390 msgr2=0x55d0340be810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033d1c390 0x55d0340be810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 -- 172.21.15.78:0/1512646622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d033fe7c30 con 0x55d033cf34d0 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033d1c390 0x55d0340be810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f097fa640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0ef40afc90 0x7f0ef40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033cf34d0 0x55d0340bed90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0e2c60 tx=0x7f0efc0e7590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c5de57b7fc1308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.577+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0ef40afc90 0x7f0ef40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0ef27fc640 1 -- 172.21.15.78:0/1512646622 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0a2e50 con 0x55d033cf34d0 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0f097fa640 1 -- 172.21.15.78:0/117328443 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0efc0deac0 con 0x7f0f0c103ef0 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0ef27fc640 1 -- 172.21.15.78:0/1512646622 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc0a6aa0 con 0x55d033cf34d0 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0ef27fc640 1 -- 172.21.15.78:0/1512646622 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0a6700 con 0x55d033cf34d0 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0f199c7740 1 -- 172.21.15.78:0/1512646622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033cf34d0 msgr2=0x55d0340bed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033cf34d0 0x55d0340bed90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0e2c60 tx=0x7f0efc0e7590 comp rx=0 tx=0).stop 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0ef40afc90 0x7f0ef40b21a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005fd0 tx=0x7f0f040093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0f199c7740 1 -- 172.21.15.78:0/1512646622 shutdown_connections 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.578+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033cf34d0 0x55d0340bed90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1512646622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033d1c390 0x55d0340be810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 -- 172.21.15.78:0/1512646622 >> 172.21.15.78:0/1512646622 conn(0x55d033d104e0 msgr2=0x55d0340ba020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 -- 172.21.15.78:0/1512646622 shutdown_connections 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 -- 172.21.15.78:0/1512646622 wait complete. 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 -- start start 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 Processor -- start 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cf34d0 0x55d0340c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cfb6b0 0x55d0340c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d0340c04d0 con 0x55d033d1c390 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d033fa9dc0 con 0x55d033cfb6b0 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d0340c0650 con 0x55d033cf34d0 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f199c7740 10 client.0 _ll_register_callbacks cb 0x55d033fc5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.579+0000 7f0f10fdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50072/0 (socket says 172.21.15.78:50072) 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 -- 172.21.15.78:0/1872348255 learned_addr learned my addr 172.21.15.78:0/1872348255 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cf34d0 0x55d0340c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f0b7fe640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cfb6b0 0x55d0340c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 -- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cf34d0 msgr2=0x55d0340c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cf34d0 0x55d0340c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 -- 172.21.15.78:0/1872348255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cfb6b0 msgr2=0x55d0340c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cfb6b0 0x55d0340c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 -- 172.21.15.78:0/1872348255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d033d343f0 con 0x55d033d1c390 2024-04-25T14:05:46.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f0b7fe640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cfb6b0 0x55d0340c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340cf9b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400fe20 tx=0x7f0f04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d899a9203df9f390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0eeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/1872348255 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0401c620 con 0x55d033d1c390 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/1872348255 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0f0401c7c0 con 0x55d033d1c390 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0efc0a6aa0 con 0x55d033d1c390 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/1872348255 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0f0400ee90 con 0x55d033d1c390 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f199c7740 10 client.17358 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d033dfe7d0 con 0x55d033d1c390 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.580+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/1872348255 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc0a6aa0 con 0x55d033d1c390 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/1872348255 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0f0402faa0 con 0x55d033d1c390 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0f199c7740 10 client.17358 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0f199c7740 10 client.17358 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0f199c7740 20 client.17358 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:46.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cdaf60 0x55d0340d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0f199c7740 4 client.17358 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d034030990 con 0x55d033cdaf60 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.581+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cdaf60 0x55d0340d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.582+0000 7f0eeb7fe640 10 client.17358 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.582+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cdaf60 0x55d0340d6270 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f74f90bf3439093 server_cookie=a55fb3de46b74efe in_seq=0 out_seq=0 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.582+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/1872348255 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d034030990 con 0x55d033cdaf60 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.582+0000 7f0eeb7fe640 10 client.17358 handle_command_reply: tid=0 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.582+0000 7f0f199c7740 1 client.17358 shutdown 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.582+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 msgr2=0x55d0340cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.582+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340cf9b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400fe20 tx=0x7f0f04004a40 comp rx=0 tx=0).stop 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 shutdown_connections 2024-04-25T14:05:46.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340cf9b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cfb6b0 0x55d0340c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cdaf60 0x55d0340d6270 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 --2- 172.21.15.78:0/1872348255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cf34d0 0x55d0340c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0eeb7fe640 0 client.17358 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 >> 172.21.15.78:0/1872348255 conn(0x55d033d104e0 msgr2=0x55d0340bcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 shutdown_connections 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 1 -- 172.21.15.78:0/1872348255 wait complete. 2024-04-25T14:05:46.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.583+0000 7f0f199c7740 20 client.17358 trim_cache size 0 max 16384 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 Processor -- start 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 -- start start 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1adb30 con 0x55d033d1c390 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1ab400 con 0x55d033cf34d0 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f199c7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1a9250 con 0x55d033cfb6b0 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f10fdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f10fdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340b8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50086/0 (socket says 172.21.15.78:50086) 2024-04-25T14:05:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.596+0000 7f0f10fdf640 1 -- 172.21.15.78:0/2788157897 learned_addr learned my addr 172.21.15.78:0/2788157897 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0f10fdf640 1 -- 172.21.15.78:0/2788157897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 msgr2=0x55d0340cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0f10fdf640 1 -- 172.21.15.78:0/2788157897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 msgr2=0x55d0340ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340ba150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0f10fdf640 1 -- 172.21.15.78:0/2788157897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0340bf730 con 0x55d033d1c390 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340b8d20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005fa0 tx=0x7f0f04009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92644ab4233a15bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0ef1ffb640 1 -- 172.21.15.78:0/2788157897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04034040 con 0x55d033d1c390 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0ef1ffb640 1 -- 172.21.15.78:0/2788157897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f04024df0 con 0x55d033d1c390 2024-04-25T14:05:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.597+0000 7f0ef1ffb640 1 -- 172.21.15.78:0/2788157897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0401cd40 con 0x55d033d1c390 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- 172.21.15.78:0/2788157897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 msgr2=0x55d0340b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340b8d20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005fa0 tx=0x7f0f04009440 comp rx=0 tx=0).stop 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- 172.21.15.78:0/2788157897 shutdown_connections 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d0340b8d20 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 --2- 172.21.15.78:0/2788157897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- 172.21.15.78:0/2788157897 >> 172.21.15.78:0/2788157897 conn(0x55d033d104e0 msgr2=0x55d0340b8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- 172.21.15.78:0/2788157897 shutdown_connections 2024-04-25T14:05:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- 172.21.15.78:0/2788157897 wait complete. 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- start start 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 Processor -- start 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d03413c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1a98c0 con 0x55d033d1c390 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1aad30 con 0x55d033cf34d0 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.598+0000 7f0f199c7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1a9f90 con 0x55d033cfb6b0 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f199c7740 10 client.0 _ll_register_callbacks cb 0x7f0f04009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d03413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d03413c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50090/0 (socket says 172.21.15.78:50090) 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 -- 172.21.15.78:0/3554754903 learned_addr learned my addr 172.21.15.78:0/3554754903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 -- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 msgr2=0x55d0340ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f0b7fe640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 -- 172.21.15.78:0/3554754903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 msgr2=0x55d0340cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 -- 172.21.15.78:0/3554754903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0340cfe10 con 0x55d033d1c390 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f0b7fe640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f10fdf640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d03413c610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0f04024420 tx=0x7f0f0401df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8556999fdbd1ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0eeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/3554754903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04047060 con 0x55d033d1c390 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/3554754903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0f0401c420 con 0x55d033d1c390 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f04024df0 con 0x55d033d1c390 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/3554754903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0f04034040 con 0x55d033d1c390 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f199c7740 10 client.17364 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:46.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.599+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d03413d9e0 con 0x55d033d1c390 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/3554754903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f04024df0 con 0x55d033d1c390 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/3554754903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0f04041070 con 0x55d033d1c390 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f199c7740 10 client.17364 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f199c7740 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f199c7740 20 client.17364 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f199c7740 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cb03a0 0x55d0341403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f199c7740 4 client.17364 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d03402a800 con 0x55d033cb03a0 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cb03a0 0x55d0341403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0eeb7fe640 10 client.17364 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.600+0000 7f0f0bfff640 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cb03a0 0x55d0341403e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=940e937ac1b0f5f8 server_cookie=ce7e6799df9124d1 in_seq=0 out_seq=0 2024-04-25T14:05:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0eeb7fe640 1 -- 172.21.15.78:0/3554754903 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d03402a800 con 0x55d033cb03a0 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0eeb7fe640 10 client.17364 handle_command_reply: tid=0 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 client.17364 shutdown 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 msgr2=0x55d03413c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d03413c610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0f04024420 tx=0x7f0f0401df20 comp rx=0 tx=0).stop 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 shutdown_connections 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d033d1c390 0x55d03413c610 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55d033cb03a0 0x55d0341403e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d033cfb6b0 0x55d0340ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 --2- 172.21.15.78:0/3554754903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d033cf34d0 0x55d0340cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0eeb7fe640 0 client.17364 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 >> 172.21.15.78:0/3554754903 conn(0x55d033d104e0 msgr2=0x55d0340bd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 shutdown_connections 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 1 -- 172.21.15.78:0/3554754903 wait complete. 2024-04-25T14:05:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f199c7740 20 client.17364 trim_cache size 0 max 16384 2024-04-25T14:05:46.608 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0ef40afc90 msgr2=0x7f0ef40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f12a42640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0ef40afc90 0x7f0ef40b21a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005fd0 tx=0x7f0f040093d0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c103ef0 msgr2=0x7f0f0c199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.601+0000 7f0f12a42640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c103ef0 0x7f0f0c199e00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00c970 tx=0x7f0efc00ce30 comp rx=0 tx=0).stop 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 shutdown_connections 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f0c19f280 0x7f0f0c1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f0c108a20 0x7f0f0c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0ef40afc90 0x7f0ef40b21a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 --2- 172.21.15.78:0/117328443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f0c103ef0 0x7f0f0c199e00 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 >> 172.21.15.78:0/117328443 conn(0x7f0f0c0fd350 msgr2=0x7f0f0c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 shutdown_connections 2024-04-25T14:05:46.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.602+0000 7f0f12a42640 1 -- 172.21.15.78:0/117328443 wait complete. 2024-04-25T14:05:46.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:05:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:46 smithi129 ceph-mon[32569]: pgmap v535: 97 pgs: 97 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:05:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:46 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:05:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:46 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:46 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:05:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:46 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:46 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:05:46.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 -- 172.21.15.78:0/1970084238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 msgr2=0x7f0ce810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 --2- 172.21.15.78:0/1970084238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce810ad40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009ca0 tx=0x7f0ce002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 -- 172.21.15.78:0/1970084238 shutdown_connections 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 --2- 172.21.15.78:0/1970084238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce810ad40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 --2- 172.21.15.78:0/1970084238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce8103c20 0x7f0ce8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 --2- 172.21.15.78:0/1970084238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce81029e0 0x7f0ce81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 -- 172.21.15.78:0/1970084238 >> 172.21.15.78:0/1970084238 conn(0x7f0ce80fdbd0 msgr2=0x7f0ce8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.762+0000 7f0ced51f640 1 -- 172.21.15.78:0/1970084238 shutdown_connections 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 -- 172.21.15.78:0/1970084238 wait complete. 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 Processor -- start 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 -- start start 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce81029e0 0x7f0ce8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce8103c20 0x7f0ce8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810e4f0 con 0x7f0ce8108920 2024-04-25T14:05:46.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810e370 con 0x7f0ce8103c20 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ced51f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce810e670 con 0x7f0ce81029e0 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ce77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.763+0000 7f0ce77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce81a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50100/0 (socket says 172.21.15.78:50100) 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce77fe640 1 -- 172.21.15.78:0/201193407 learned_addr learned my addr 172.21.15.78:0/201193407 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0cdffff640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce8103c20 0x7f0ce8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce81029e0 0x7f0ce8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce77fe640 1 -- 172.21.15.78:0/201193407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce81029e0 msgr2=0x7f0ce8199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce77fe640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce81029e0 0x7f0ce8199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce77fe640 1 -- 172.21.15.78:0/201193407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce8103c20 msgr2=0x7f0ce8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce77fe640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce8103c20 0x7f0ce8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce77fe640 1 -- 172.21.15.78:0/201193407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce81a0c70 con 0x7f0ce8108920 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce77fe640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce81a0600 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005e30 tx=0x7f0ce002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce4ff9640 1 -- 172.21.15.78:0/201193407 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0003bf0 con 0x7f0ce8108920 2024-04-25T14:05:46.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce4ff9640 1 -- 172.21.15.78:0/201193407 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce0003d90 con 0x7f0ce8108920 2024-04-25T14:05:46.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce81a0f00 con 0x7f0ce8108920 2024-04-25T14:05:46.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ce4ff9640 1 -- 172.21.15.78:0/201193407 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce002fb90 con 0x7f0ce8108920 2024-04-25T14:05:46.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.764+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce81a1360 con 0x7f0ce8108920 2024-04-25T14:05:46.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0ce4ff9640 1 -- 172.21.15.78:0/201193407 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0ce002fd30 con 0x7f0ce8108920 2024-04-25T14:05:46.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 Processor -- start 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 -- start start 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 0x558ca9995810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95ca4d0 0x558ca9995d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95d26b0 0x558ca9996a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ca98f9d80 con 0x558ca95ca4d0 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558ca98b3a20 con 0x558ca95d26b0 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558ca98f5d40 con 0x558ca95f3390 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0ce6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95ca4d0 0x558ca9995d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.767+0000 7f0ce6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95ca4d0 0x558ca9995d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50102/0 (socket says 172.21.15.78:50102) 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/2518228315 learned_addr learned my addr 172.21.15.78:0/2518228315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce4ff9640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0cdffff640 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95d26b0 0x558ca9996a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0cdffff640 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 0x558ca9995810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce77fe640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/2518228315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 msgr2=0x558ca9995810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 0x558ca9995810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/2518228315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95d26b0 msgr2=0x558ca9996a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95d26b0 0x558ca9996a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/2518228315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca98bec30 con 0x558ca95ca4d0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0cdffff640 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95d26b0 0x558ca9996a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce4ff9640 1 -- 172.21.15.78:0/201193407 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0ce0100360 con 0x7f0ce8108920 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95ca4d0 0x558ca9995d90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009bf0 tx=0x7f0cd80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728b898e83bf039 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0cdd7fa640 1 -- 172.21.15.78:0/2518228315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8004030 con 0x558ca95ca4d0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0cdd7fa640 1 -- 172.21.15.78:0/2518228315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd80035f0 con 0x558ca95ca4d0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.768+0000 7f0cdd7fa640 1 -- 172.21.15.78:0/2518228315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80126b0 con 0x558ca95ca4d0 2024-04-25T14:05:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2518228315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95ca4d0 msgr2=0x558ca9995d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0ce77fe640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00fdea0 tx=0x7f0ce00bd170 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95ca4d0 0x558ca9995d90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009bf0 tx=0x7f0cd80117b0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2518228315 shutdown_connections 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95d26b0 0x558ca9996a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95ca4d0 0x558ca9995d90 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2518228315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 0x558ca9995810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2518228315 >> 172.21.15.78:0/2518228315 conn(0x558ca95e74e0 msgr2=0x558ca9991020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2518228315 shutdown_connections 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.769+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2518228315 wait complete. 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 -- start start 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 Processor -- start 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca999f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95d26b0 0x558ca999faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95f3390 0x558ca99a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ca9997530 con 0x558ca95f3390 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558ca9880dc0 con 0x558ca95ca4d0 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558ca99976b0 con 0x558ca95d26b0 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cf43d1740 10 client.0 _ll_register_callbacks cb 0x558ca989c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95f3390 0x558ca99a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95f3390 0x558ca99a6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50114/0 (socket says 172.21.15.78:50114) 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 -- 172.21.15.78:0/2205451473 learned_addr learned my addr 172.21.15.78:0/2205451473 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca999f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95d26b0 0x558ca999faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 -- 172.21.15.78:0/2205451473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95d26b0 msgr2=0x558ca999faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95d26b0 0x558ca999faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 -- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 msgr2=0x558ca999f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca999f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 -- 172.21.15.78:0/2205451473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca960b3f0 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cdffff640 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95f3390 0x558ca99a6990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00129b0 tx=0x7f0cd0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31deca06ebc8b17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.770+0000 7f0cce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cce7fc640 1 -- 172.21.15.78:0/2205451473 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000fa70 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cce7fc640 1 -- 172.21.15.78:0/2205451473 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0cd000fc10 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd80035f0 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cce7fc640 1 -- 172.21.15.78:0/2205451473 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0cd000fe10 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 10 client.17376 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558ca96d57d0 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cce7fc640 1 -- 172.21.15.78:0/2205451473 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd80035f0 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cce7fc640 1 -- 172.21.15.78:0/2205451473 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0cd0010440 con 0x558ca95f3390 2024-04-25T14:05:46.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 10 client.17376 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 20 client.17376 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95b1f60 0x558ca99ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 4 client.17376 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.771+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca9907990 con 0x558ca95b1f60 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.772+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95b1f60 0x558ca99ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.772+0000 7f0cce7fc640 10 client.17376 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.772+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95b1f60 0x558ca99ad220 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8aeec6de58f6c6d server_cookie=9a1ac8c1c07bb7b2 in_seq=0 out_seq=0 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cce7fc640 1 -- 172.21.15.78:0/2205451473 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ca9907990 con 0x558ca95b1f60 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cce7fc640 10 client.17376 handle_command_reply: tid=0 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 client.17376 shutdown 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95f3390 msgr2=0x558ca99a6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95f3390 0x558ca99a6990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00129b0 tx=0x7f0cd0014040 comp rx=0 tx=0).stop 2024-04-25T14:05:46.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 shutdown_connections 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95f3390 0x558ca99a6990 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95b1f60 0x558ca99ad220 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95d26b0 0x558ca999faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cce7fc640 0 client.17376 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/2205451473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca999f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 >> 172.21.15.78:0/2205451473 conn(0x558ca95e74e0 msgr2=0x558ca9993f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 shutdown_connections 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 1 -- 172.21.15.78:0/2205451473 wait complete. 2024-04-25T14:05:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.773+0000 7f0cf43d1740 20 client.17376 trim_cache size 0 max 16384 2024-04-25T14:05:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.786+0000 7f0cf43d1740 1 Processor -- start 2024-04-25T14:05:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.786+0000 7f0cf43d1740 1 -- start start 2024-04-25T14:05:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95ca4d0 0x558ca9990a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca9990f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95f3390 0x558ca999dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81aca70 con 0x558ca95d26b0 2024-04-25T14:05:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81aa340 con 0x558ca95f3390 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a8190 con 0x558ca95ca4d0 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca9990f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca9990f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50126/0 (socket says 172.21.15.78:50126) 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/661518790 learned_addr learned my addr 172.21.15.78:0/661518790 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cdffff640 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95f3390 0x558ca999dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/661518790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95ca4d0 msgr2=0x558ca9990a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95ca4d0 0x558ca9990a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/661518790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95f3390 msgr2=0x558ca999dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95f3390 0x558ca999dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/661518790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca9995360 con 0x558ca95d26b0 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cdffff640 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95f3390 0x558ca999dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca9990f90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005fd0 tx=0x7f0cd8005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291df4a5b6da15d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cdcff9640 1 -- 172.21.15.78:0/661518790 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd801f040 con 0x558ca95d26b0 2024-04-25T14:05:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cdcff9640 1 -- 172.21.15.78:0/661518790 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd802e070 con 0x558ca95d26b0 2024-04-25T14:05:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.787+0000 7f0cdcff9640 1 -- 172.21.15.78:0/661518790 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8029370 con 0x558ca95d26b0 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 -- 172.21.15.78:0/661518790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 msgr2=0x558ca9990f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca9990f90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005fd0 tx=0x7f0cd8005900 comp rx=0 tx=0).stop 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 -- 172.21.15.78:0/661518790 shutdown_connections 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95f3390 0x558ca999dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca9990f90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/661518790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95ca4d0 0x558ca9990a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 -- 172.21.15.78:0/661518790 >> 172.21.15.78:0/661518790 conn(0x558ca95e74e0 msgr2=0x558ca999a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 -- 172.21.15.78:0/661518790 shutdown_connections 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 -- 172.21.15.78:0/661518790 wait complete. 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.788+0000 7f0cf43d1740 1 -- start start 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 Processor -- start 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca99a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca99a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 0x558ca9a156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a8800 con 0x558ca95d26b0 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a9c70 con 0x558ca95ca4d0 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce81a8ed0 con 0x558ca95f3390 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 10 client.0 _ll_register_callbacks cb 0x7f0cd000b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca99a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca99a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50140/0 (socket says 172.21.15.78:50140) 2024-04-25T14:05:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/1504125908 learned_addr learned my addr 172.21.15.78:0/1504125908 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cdffff640 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca99a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/1504125908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 msgr2=0x558ca9a156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 0x558ca9a156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 msgr2=0x558ca99a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca99a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 -- 172.21.15.78:0/1504125908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca9a15cb0 con 0x558ca95d26b0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cdffff640 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca99a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0ce6ffd640 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca99a1c30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8011cb0 tx=0x7f0cd8011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca4ac408bc23ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cce7fc640 1 -- 172.21.15.78:0/1504125908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80052d0 con 0x558ca95d26b0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cce7fc640 1 -- 172.21.15.78:0/1504125908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0cd8003820 con 0x558ca95d26b0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.789+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd802e070 con 0x558ca95d26b0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cce7fc640 1 -- 172.21.15.78:0/1504125908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0cd8003a20 con 0x558ca95d26b0 2024-04-25T14:05:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 10 client.17382 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558ca9a16ea0 con 0x558ca95d26b0 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cce7fc640 1 -- 172.21.15.78:0/1504125908 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd802e070 con 0x558ca95d26b0 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cce7fc640 1 -- 172.21.15.78:0/1504125908 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0cd8025040 con 0x558ca95d26b0 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 10 client.17382 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 20 client.17382 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95873a0 0x558ca9a19740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 4 client.17382 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ca9901800 con 0x558ca95873a0 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.790+0000 7f0cdffff640 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95873a0 0x558ca9a19740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cdffff640 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95873a0 0x558ca9a19740 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fd4828a124e49f server_cookie=a9389a5d615d324c in_seq=0 out_seq=0 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cce7fc640 10 client.17382 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cce7fc640 1 -- 172.21.15.78:0/1504125908 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ca9901800 con 0x558ca95873a0 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cce7fc640 10 client.17382 handle_command_reply: tid=0 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 client.17382 shutdown 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 msgr2=0x558ca99a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca99a1c30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8011cb0 tx=0x7f0cd8011e30 comp rx=0 tx=0).stop 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 shutdown_connections 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558ca95873a0 0x558ca9a19740 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ca95f3390 0x558ca9a156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ca95d26b0 0x558ca99a1c30 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cce7fc640 0 client.17382 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 --2- 172.21.15.78:0/1504125908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558ca95ca4d0 0x558ca99a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 >> 172.21.15.78:0/1504125908 conn(0x558ca95e74e0 msgr2=0x558ca99909e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 shutdown_connections 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 1 -- 172.21.15.78:0/1504125908 wait complete. 2024-04-25T14:05:46.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.791+0000 7f0cf43d1740 20 client.17382 trim_cache size 0 max 16384 2024-04-25T14:05:46.798 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 msgr2=0x7f0cd40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00fdea0 tx=0x7f0ce00bd170 comp rx=0 tx=0).stop 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 msgr2=0x7f0ce81a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce81a0600 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005e30 tx=0x7f0ce002f380 comp rx=0 tx=0).stop 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 shutdown_connections 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0cd40afaf0 0x7f0cd40b2000 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ce8108920 0x7f0ce81a0600 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ce8103c20 0x7f0ce8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 --2- 172.21.15.78:0/201193407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce81029e0 0x7f0ce8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 >> 172.21.15.78:0/201193407 conn(0x7f0ce80fdbd0 msgr2=0x7f0ce8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 shutdown_connections 2024-04-25T14:05:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:46.792+0000 7f0ced51f640 1 -- 172.21.15.78:0/201193407 wait complete. 2024-04-25T14:05:47.518 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:47 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3150762928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:47.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:47 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3150762928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:47.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:47 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3150762928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:05:47.803 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:05:47.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:05:47.804 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:05:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 -- 172.21.15.78:0/3871440495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 msgr2=0x7f38181036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 --2- 172.21.15.78:0/3871440495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f38181036f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3808009900 tx=0x7f380802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 -- 172.21.15.78:0/3871440495 shutdown_connections 2024-04-25T14:05:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 --2- 172.21.15.78:0/3871440495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818108930 0x7f381810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 --2- 172.21.15.78:0/3871440495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3818103c30 0x7f3818104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 --2- 172.21.15.78:0/3871440495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f38181036f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 -- 172.21.15.78:0/3871440495 >> 172.21.15.78:0/3871440495 conn(0x7f38180fdbc0 msgr2=0x7f3818100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 -- 172.21.15.78:0/3871440495 shutdown_connections 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.925+0000 7f381e508640 1 -- 172.21.15.78:0/3871440495 wait complete. 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 Processor -- start 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 -- start start 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f3818199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3818103c30 0x7f38181996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818108930 0x7f38181a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f381810e4d0 con 0x7f38181029f0 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f381810e350 con 0x7f3818108930 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381e508640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f381810e650 con 0x7f3818103c30 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f3817fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f3818199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f3817fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f3818199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50158/0 (socket says 172.21.15.78:50158) 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f3817fff640 1 -- 172.21.15.78:0/631653686 learned_addr learned my addr 172.21.15.78:0/631653686 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f381caa5640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818108930 0x7f38181a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f38177fe640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3818103c30 0x7f38181996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.926+0000 7f3817fff640 1 -- 172.21.15.78:0/631653686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3818103c30 msgr2=0x7f38181996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f3817fff640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3818103c30 0x7f38181996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f3817fff640 1 -- 172.21.15.78:0/631653686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818108930 msgr2=0x7f38181a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f3817fff640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818108930 0x7f38181a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f3817fff640 1 -- 172.21.15.78:0/631653686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38181a0c40 con 0x7f38181029f0 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f38177fe640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3818103c30 0x7f38181996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f381caa5640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818108930 0x7f38181a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f3817fff640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f3818199170 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3808006010 tx=0x7f3808031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f38157fa640 1 -- 172.21.15.78:0/631653686 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380803f030 con 0x7f38181029f0 2024-04-25T14:05:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38181a0ed0 con 0x7f38181029f0 2024-04-25T14:05:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f38157fa640 1 -- 172.21.15.78:0/631653686 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3808003aa0 con 0x7f38181029f0 2024-04-25T14:05:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f38157fa640 1 -- 172.21.15.78:0/631653686 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808035070 con 0x7f38181029f0 2024-04-25T14:05:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.927+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38181a1330 con 0x7f38181029f0 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.929+0000 7f38253b6740 1 Processor -- start 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.929+0000 7f38253b6740 1 -- start start 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.929+0000 7f38157fa640 1 -- 172.21.15.78:0/631653686 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3808042030 con 0x7f38181029f0 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38253b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cde4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38253b6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cdea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38253b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3ce5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38253b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584d3c43b00 con 0x5584d393d410 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38253b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5584d3bdf920 con 0x5584d391c730 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38253b6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5584d3c3fac0 con 0x5584d3914550 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cde4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cde4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50174/0 (socket says 172.21.15.78:50174) 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 -- 172.21.15.78:0/138188536 learned_addr learned my addr 172.21.15.78:0/138188536 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38177fe640 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3ce5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f3817fff640 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cdea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 -- 172.21.15.78:0/138188536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 msgr2=0x5584d3cdea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cdea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 -- 172.21.15.78:0/138188536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 msgr2=0x5584d3ce5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3ce5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 -- 172.21.15.78:0/138188536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d3a1d7e0 con 0x5584d393d410 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f38157fa640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38000afaf0 0x7f38000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f381caa5640 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cde4a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3810009fb0 tx=0x7f3810010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182358851083ffdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.930+0000 7f3817fff640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38000afaf0 0x7f38000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f37fe7fc640 1 -- 172.21.15.78:0/138188536 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810016070 con 0x5584d393d410 2024-04-25T14:05:47.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38157fa640 1 -- 172.21.15.78:0/631653686 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f38080ff270 con 0x7f38181029f0 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f37fe7fc640 1 -- 172.21.15.78:0/138188536 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810012bc0 con 0x5584d393d410 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f37fe7fc640 1 -- 172.21.15.78:0/138188536 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381001bc70 con 0x5584d393d410 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38253b6740 1 -- 172.21.15.78:0/138188536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 msgr2=0x5584d3cde4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f3817fff640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38000afaf0 0x7f38000b2000 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f38080c49b0 tx=0x7f38080b0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38253b6740 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cde4a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3810009fb0 tx=0x7f3810010800 comp rx=0 tx=0).stop 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38253b6740 1 -- 172.21.15.78:0/138188536 shutdown_connections 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38253b6740 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3ce5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38253b6740 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cdea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38253b6740 1 --2- 172.21.15.78:0/138188536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cde4a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.931+0000 7f38253b6740 1 -- 172.21.15.78:0/138188536 >> 172.21.15.78:0/138188536 conn(0x5584d3931560 msgr2=0x5584d3cd9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 -- 172.21.15.78:0/138188536 shutdown_connections 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 -- 172.21.15.78:0/138188536 wait complete. 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 -- start start 2024-04-25T14:05:47.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 Processor -- start 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 0x5584d3cf3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584d3ce62c0 con 0x5584d3914550 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5584d3bdd5d0 con 0x5584d393d410 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5584d3ce6440 con 0x5584d391c730 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f38253b6740 10 client.0 _ll_register_callbacks cb 0x5584d3bcb080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.932+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cec8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50182/0 (socket says 172.21.15.78:50182) 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 -- 172.21.15.78:0/1625063807 learned_addr learned my addr 172.21.15.78:0/1625063807 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f38177fe640 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 0x5584d3cf3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 -- 172.21.15.78:0/1625063807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 msgr2=0x5584d3cecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 -- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 msgr2=0x5584d3cf3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 0x5584d3cf3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 -- 172.21.15.78:0/1625063807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d3a7df70 con 0x5584d3914550 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f381caa5640 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cec8a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f381001b770 tx=0x7f381001b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fcf68b5549b7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f37f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f37f77fe640 1 -- 172.21.15.78:0/1625063807 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810016050 con 0x5584d3914550 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f37f77fe640 1 -- 172.21.15.78:0/1625063807 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f381001b9f0 con 0x5584d3914550 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3810012bc0 con 0x5584d3914550 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f37f77fe640 1 -- 172.21.15.78:0/1625063807 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f381001bbf0 con 0x5584d3914550 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f38253b6740 10 client.17394 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5584d3915170 con 0x5584d3914550 2024-04-25T14:05:47.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.933+0000 7f37f77fe640 1 -- 172.21.15.78:0/1625063807 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810012bc0 con 0x5584d3914550 2024-04-25T14:05:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f37f77fe640 1 -- 172.21.15.78:0/1625063807 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3810028020 con 0x5584d3914550 2024-04-25T14:05:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38253b6740 10 client.17394 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38253b6740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38253b6740 20 client.17394 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38253b6740 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38fc1f0 0x5584d3cf9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38253b6740 4 client.17394 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584d3c51710 con 0x5584d38fc1f0 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38177fe640 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38fc1f0 0x5584d3cf9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f38177fe640 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38fc1f0 0x5584d3cf9dd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc01f877faed62e server_cookie=a44425da5b1434b3 in_seq=0 out_seq=0 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.934+0000 7f37f77fe640 10 client.17394 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f37f77fe640 1 -- 172.21.15.78:0/1625063807 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584d3c51710 con 0x5584d38fc1f0 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f37f77fe640 10 client.17394 handle_command_reply: tid=0 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 client.17394 shutdown 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 msgr2=0x5584d3cec8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cec8a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f381001b770 tx=0x7f381001b460 comp rx=0 tx=0).stop 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 shutdown_connections 2024-04-25T14:05:47.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 0x5584d3cf3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38fc1f0 0x5584d3cf9dd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 --2- 172.21.15.78:0/1625063807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cec8a0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f37f77fe640 0 client.17394 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 >> 172.21.15.78:0/1625063807 conn(0x5584d3931560 msgr2=0x5584d3cd8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 shutdown_connections 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 1 -- 172.21.15.78:0/1625063807 wait complete. 2024-04-25T14:05:47.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.935+0000 7f38253b6740 20 client.17394 trim_cache size 0 max 16384 2024-04-25T14:05:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 Processor -- start 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 -- start start 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cdd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cdd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 0x5584d3cf3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38181ab9b0 con 0x5584d3914550 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38181a9280 con 0x5584d393d410 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38253b6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38181a70d0 con 0x5584d391c730 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cdd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cdd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50192/0 (socket says 172.21.15.78:50192) 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 -- 172.21.15.78:0/2071272998 learned_addr learned my addr 172.21.15.78:0/2071272998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38177fe640 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cdd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 -- 172.21.15.78:0/2071272998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 msgr2=0x5584d3cdd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cdd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 -- 172.21.15.78:0/2071272998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 msgr2=0x5584d3cf3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 0x5584d3cf3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f381caa5640 1 -- 172.21.15.78:0/2071272998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d3cda0c0 con 0x5584d3914550 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.953+0000 7f38177fe640 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cdd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.954+0000 7f381caa5640 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cdd340 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3810005500 tx=0x7f3810009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb15fc615484ac5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.954+0000 7f37fdffb640 1 -- 172.21.15.78:0/2071272998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810016040 con 0x5584d3914550 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.954+0000 7f37fdffb640 1 -- 172.21.15.78:0/2071272998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810039070 con 0x5584d3914550 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.954+0000 7f37fdffb640 1 -- 172.21.15.78:0/2071272998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381001b7a0 con 0x5584d3914550 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.954+0000 7f38253b6740 1 -- 172.21.15.78:0/2071272998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 msgr2=0x5584d3cdd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.954+0000 7f38253b6740 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cdd340 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3810005500 tx=0x7f3810009e00 comp rx=0 tx=0).stop 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- 172.21.15.78:0/2071272998 shutdown_connections 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d393d410 0x5584d3cf3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d391c730 0x5584d3cdd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 --2- 172.21.15.78:0/2071272998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d3914550 0x5584d3cdd340 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- 172.21.15.78:0/2071272998 >> 172.21.15.78:0/2071272998 conn(0x5584d3931560 msgr2=0x5584d3cdcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- 172.21.15.78:0/2071272998 shutdown_connections 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- 172.21.15.78:0/2071272998 wait complete. 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- start start 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 Processor -- start 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cef930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3cefe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cf2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38181a7740 con 0x5584d393d410 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38181a8bb0 con 0x5584d391c730 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38181a7e10 con 0x5584d3914550 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f38253b6740 10 client.0 _ll_register_callbacks cb 0x7f3810009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cf2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f381caa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cf2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50194/0 (socket says 172.21.15.78:50194) 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.955+0000 7f381caa5640 1 -- 172.21.15.78:0/1017972710 learned_addr learned my addr 172.21.15.78:0/1017972710 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f38177fe640 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3cefe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f381caa5640 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f381caa5640 1 -- 172.21.15.78:0/1017972710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 msgr2=0x5584d3cef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f381caa5640 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f381caa5640 1 -- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 msgr2=0x5584d3cefe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f381caa5640 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3cefe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f381caa5640 1 -- 172.21.15.78:0/1017972710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d3cf3320 con 0x5584d393d410 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f381caa5640 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cf2d60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3810009a30 tx=0x7f3810005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78127da42bd7025 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f37f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f37f77fe640 1 -- 172.21.15.78:0/1017972710 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810012510 con 0x5584d393d410 2024-04-25T14:05:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f37f77fe640 1 -- 172.21.15.78:0/1017972710 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f38100126b0 con 0x5584d393d410 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3810039070 con 0x5584d393d410 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f37f77fe640 1 -- 172.21.15.78:0/1017972710 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f38100128b0 con 0x5584d393d410 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f38253b6740 10 client.17400 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5584d3cf4550 con 0x5584d393d410 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.956+0000 7f37f77fe640 1 -- 172.21.15.78:0/1017972710 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810039070 con 0x5584d393d410 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f37f77fe640 1 -- 172.21.15.78:0/1017972710 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3810028070 con 0x5584d393d410 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38253b6740 10 client.17400 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38253b6740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38253b6740 20 client.17400 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38253b6740 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38d1630 0x5584d3d646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38253b6740 4 client.17400 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5584d3c4b580 con 0x5584d38d1630 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38177fe640 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38d1630 0x5584d3d646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f37f77fe640 10 client.17400 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:47.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.957+0000 7f38177fe640 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38d1630 0x5584d3d646e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591648a0a79cc385 server_cookie=d50614ceb06b5bd6 in_seq=0 out_seq=0 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f37f77fe640 1 -- 172.21.15.78:0/1017972710 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5584d3c4b580 con 0x5584d38d1630 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f37f77fe640 10 client.17400 handle_command_reply: tid=0 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 client.17400 shutdown 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 msgr2=0x5584d3cf2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cf2d60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f3810009a30 tx=0x7f3810005600 comp rx=0 tx=0).stop 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 shutdown_connections 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d393d410 0x5584d3cf2d60 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5584d391c730 0x5584d3cefe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5584d38d1630 0x5584d3d646e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 --2- 172.21.15.78:0/1017972710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584d3914550 0x5584d3cef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 >> 172.21.15.78:0/1017972710 conn(0x5584d3931560 msgr2=0x5584d3cdd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.958+0000 7f37f77fe640 0 client.17400 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 shutdown_connections 2024-04-25T14:05:47.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f38253b6740 1 -- 172.21.15.78:0/1017972710 wait complete. 2024-04-25T14:05:47.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f38253b6740 20 client.17400 trim_cache size 0 max 16384 2024-04-25T14:05:47.965 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:05:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "967d8799-2269-424e-81f5-ce790c6001f2", 2024-04-25T14:05:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:05:47.965 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38000afaf0 msgr2=0x7f38000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38000afaf0 0x7f38000b2000 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f38080c49b0 tx=0x7f38080b0040 comp rx=0 tx=0).stop 2024-04-25T14:05:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 msgr2=0x7f3818199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f3818199170 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3808006010 tx=0x7f3808031a50 comp rx=0 tx=0).stop 2024-04-25T14:05:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 shutdown_connections 2024-04-25T14:05:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3818108930 0x7f38181a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3818103c30 0x7f38181996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f38000afaf0 0x7f38000b2000 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 --2- 172.21.15.78:0/631653686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38181029f0 0x7f3818199170 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:47.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 >> 172.21.15.78:0/631653686 conn(0x7f38180fdbc0 msgr2=0x7f3818104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:47.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 shutdown_connections 2024-04-25T14:05:47.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:47.959+0000 7f381e508640 1 -- 172.21.15.78:0/631653686 wait complete. 2024-04-25T14:05:47.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:05:47.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:05:48.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 -- 172.21.15.78:0/3253308500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c106f40 msgr2=0x7fc87c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 --2- 172.21.15.78:0/3253308500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c106f40 0x7fc87c109360 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009900 tx=0x7fc86c02a410 comp rx=0 tx=0).stop 2024-04-25T14:05:48.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 -- 172.21.15.78:0/3253308500 shutdown_connections 2024-04-25T14:05:48.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 --2- 172.21.15.78:0/3253308500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc87c1098a0 0x7fc87c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 --2- 172.21.15.78:0/3253308500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c106f40 0x7fc87c109360 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 --2- 172.21.15.78:0/3253308500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc87c1045e0 0x7fc87c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 -- 172.21.15.78:0/3253308500 >> 172.21.15.78:0/3253308500 conn(0x7fc87c0fdbc0 msgr2=0x7fc87c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 -- 172.21.15.78:0/3253308500 shutdown_connections 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.079+0000 7fc884338640 1 -- 172.21.15.78:0/3253308500 wait complete. 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 Processor -- start 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 -- start start 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc87c1045e0 0x7fc87c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc87c106f40 0x7fc87c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c1098a0 0x7fc87c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c10f620 con 0x7fc87c1098a0 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c10f4a0 con 0x7fc87c106f40 2024-04-25T14:05:48.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc884338640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c10f7a0 con 0x7fc87c1045e0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc8828d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c1098a0 0x7fc87c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc8828d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c1098a0 0x7fc87c1a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50200/0 (socket says 172.21.15.78:50200) 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.080+0000 7fc8828d5640 1 -- 172.21.15.78:0/2738205709 learned_addr learned my addr 172.21.15.78:0/2738205709 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8818d3640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc87c106f40 0x7fc87c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8820d4640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc87c1045e0 0x7fc87c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8828d5640 1 -- 172.21.15.78:0/2738205709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc87c1045e0 msgr2=0x7fc87c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8828d5640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc87c1045e0 0x7fc87c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8828d5640 1 -- 172.21.15.78:0/2738205709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc87c106f40 msgr2=0x7fc87c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8828d5640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc87c106f40 0x7fc87c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8828d5640 1 -- 172.21.15.78:0/2738205709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c1a1dc0 con 0x7fc87c1098a0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8828d5640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c1098a0 0x7fc87c1a1750 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e9d0 tx=0x7fc87400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8737fe640 1 -- 172.21.15.78:0/2738205709 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87400cd80 con 0x7fc87c1098a0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8737fe640 1 -- 172.21.15.78:0/2738205709 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc874005020 con 0x7fc87c1098a0 2024-04-25T14:05:48.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c1a20b0 con 0x7fc87c1098a0 2024-04-25T14:05:48.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc8737fe640 1 -- 172.21.15.78:0/2738205709 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740116d0 con 0x7fc87c1098a0 2024-04-25T14:05:48.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.081+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1a25b0 con 0x7fc87c1098a0 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.083+0000 7fc8737fe640 1 -- 172.21.15.78:0/2738205709 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc8740051d0 con 0x7fc87c1098a0 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 Processor -- start 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 -- start start 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39a2390 0x5612e3d43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d43fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39816b0 0x5612e3d44c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612e3ca8d80 con 0x5612e39a2390 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5612e3c628b0 con 0x5612e39816b0 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5612e3ca4d40 con 0x5612e39794d0 2024-04-25T14:05:48.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc8737fe640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8680afaf0 0x7fc8680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.084+0000 7fc8818d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39a2390 0x5612e3d43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39a2390 0x5612e3d43a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50212/0 (socket says 172.21.15.78:50212) 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 -- 172.21.15.78:0/3755157868 learned_addr learned my addr 172.21.15.78:0/3755157868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8828d5640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8680afaf0 0x7fc8680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8737fe640 1 -- 172.21.15.78:0/2738205709 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc8740dee50 con 0x7fc87c1098a0 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39816b0 0x5612e3d44c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d43fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 -- 172.21.15.78:0/3755157868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 msgr2=0x5612e3d43fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d43fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 -- 172.21.15.78:0/3755157868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39816b0 msgr2=0x5612e3d44c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39816b0 0x5612e3d44c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 -- 172.21.15.78:0/3755157868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e3c33a40 con 0x5612e39a2390 2024-04-25T14:05:48.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d43fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8828d5640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8680afaf0 0x7fc8680b2000 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8740a2c20 tx=0x7fc874005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39a2390 0x5612e3d43a60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0096c0 tx=0x7fc86c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d88389dde70fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc870bfe640 1 -- 172.21.15.78:0/3755157868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c035050 con 0x5612e39a2390 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc870bfe640 1 -- 172.21.15.78:0/3755157868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c031650 con 0x5612e39a2390 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.085+0000 7fc870bfe640 1 -- 172.21.15.78:0/3755157868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c003930 con 0x5612e39a2390 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3755157868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39a2390 msgr2=0x5612e3d43a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39a2390 0x5612e3d43a60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0096c0 tx=0x7fc86c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3755157868 shutdown_connections 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39816b0 0x5612e3d44c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d43fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3755157868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39a2390 0x5612e3d43a60 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3755157868 >> 172.21.15.78:0/3755157868 conn(0x5612e39964e0 msgr2=0x5612e3d3f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3755157868 shutdown_connections 2024-04-25T14:05:48.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.086+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3755157868 wait complete. 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 -- start start 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 Processor -- start 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d4dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d54db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612e3d456c0 con 0x5612e39816b0 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5612e3c2fd10 con 0x5612e39794d0 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5612e3d45840 con 0x5612e39a2390 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc88b1ee740 10 client.0 _ll_register_callbacks cb 0x5612e3c5c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d4dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d4dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50226/0 (socket says 172.21.15.78:50226) 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 -- 172.21.15.78:0/3074542700 learned_addr learned my addr 172.21.15.78:0/3074542700 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:48.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 -- 172.21.15.78:0/3074542700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 msgr2=0x5612e3d54db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d54db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 -- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 msgr2=0x5612e3d4d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d4d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8820d4640 1 -- 172.21.15.78:0/3074542700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e3a847d0 con 0x5612e39816b0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.087+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d4dec0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc878013790 tx=0x7fc878013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf734f0878c9973d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc865ffb640 1 -- 172.21.15.78:0/3074542700 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878011dd0 con 0x5612e39816b0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc865ffb640 1 -- 172.21.15.78:0/3074542700 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc878004030 con 0x5612e39816b0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc86c031650 con 0x5612e39816b0 2024-04-25T14:05:48.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc865ffb640 1 -- 172.21.15.78:0/3074542700 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc878004230 con 0x5612e39816b0 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 10 client.17412 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5612e39ba3f0 con 0x5612e39816b0 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc865ffb640 1 -- 172.21.15.78:0/3074542700 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c031650 con 0x5612e39816b0 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc865ffb640 1 -- 172.21.15.78:0/3074542700 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc878024050 con 0x5612e39816b0 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 10 client.17412 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 20 client.17412 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e3960f60 0x5612e3d5b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 4 client.17412 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.088+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612e3cb6990 con 0x5612e3960f60 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.089+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e3960f60 0x5612e3d5b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.089+0000 7fc865ffb640 10 client.17412 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.089+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e3960f60 0x5612e3d5b2a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95adcb57334cf2df server_cookie=eca855a80c817f73 in_seq=0 out_seq=0 2024-04-25T14:05:48.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc865ffb640 1 -- 172.21.15.78:0/3074542700 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612e3cb6990 con 0x5612e3960f60 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc865ffb640 10 client.17412 handle_command_reply: tid=0 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 client.17412 shutdown 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 msgr2=0x5612e3d4dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d4dec0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc878013790 tx=0x7fc878013c50 comp rx=0 tx=0).stop 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 shutdown_connections 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e3960f60 0x5612e3d5b2a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d54db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc865ffb640 0 client.17412 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d4dec0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3074542700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 >> 172.21.15.78:0/3074542700 conn(0x5612e39964e0 msgr2=0x5612e3d42180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 shutdown_connections 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3074542700 wait complete. 2024-04-25T14:05:48.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.090+0000 7fc88b1ee740 20 client.17412 trim_cache size 0 max 16384 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.107+0000 7fc88b1ee740 1 Processor -- start 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.107+0000 7fc88b1ee740 1 -- start start 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d3ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d54cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1adb10 con 0x5612e39816b0 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1ab3e0 con 0x5612e39794d0 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1a9230 con 0x5612e39a2390 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d54cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d54cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50242/0 (socket says 172.21.15.78:50242) 2024-04-25T14:05:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 -- 172.21.15.78:0/2981025883 learned_addr learned my addr 172.21.15.78:0/2981025883 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8818d3640 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d3ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8818d3640 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 -- 172.21.15.78:0/2981025883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 msgr2=0x5612e3d3f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d3f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 -- 172.21.15.78:0/2981025883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 msgr2=0x5612e3d3ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d3ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 -- 172.21.15.78:0/2981025883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e397a0f0 con 0x5612e39816b0 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8818d3640 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d3f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc8820d4640 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d54cd0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc878000c00 tx=0x7fc87800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af18167af07a3159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc867fff640 1 -- 172.21.15.78:0/2981025883 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878011dd0 con 0x5612e39816b0 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc867fff640 1 -- 172.21.15.78:0/2981025883 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc878005450 con 0x5612e39816b0 2024-04-25T14:05:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.108+0000 7fc867fff640 1 -- 172.21.15.78:0/2981025883 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87802ec30 con 0x5612e39816b0 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 -- 172.21.15.78:0/2981025883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 msgr2=0x5612e3d54cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d54cd0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc878000c00 tx=0x7fc87800f870 comp rx=0 tx=0).stop 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 -- 172.21.15.78:0/2981025883 shutdown_connections 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39a2390 0x5612e3d3f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d54cd0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/2981025883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39794d0 0x5612e3d3ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 -- 172.21.15.78:0/2981025883 >> 172.21.15.78:0/2981025883 conn(0x5612e39964e0 msgr2=0x5612e3d48780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 -- 172.21.15.78:0/2981025883 shutdown_connections 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 -- 172.21.15.78:0/2981025883 wait complete. 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 -- start start 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.109+0000 7fc88b1ee740 1 Processor -- start 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d51b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39a2390 0x5612e3dc1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1a98a0 con 0x5612e39816b0 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1aad10 con 0x5612e39a2390 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1a9f70 con 0x5612e39794d0 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 10 client.0 _ll_register_callbacks cb 0x7fc878009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d52050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50256/0 (socket says 172.21.15.78:50256) 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 -- 172.21.15.78:0/3139338901 learned_addr learned my addr 172.21.15.78:0/3139338901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d51b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 -- 172.21.15.78:0/3139338901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 msgr2=0x5612e3d51b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d51b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 -- 172.21.15.78:0/3139338901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39a2390 msgr2=0x5612e3dc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39a2390 0x5612e3dc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 -- 172.21.15.78:0/3139338901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e3c694b0 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc8820d4640 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d52050 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc87800f6a0 tx=0x7fc878009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4a5e8a99e0d6ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc865ffb640 1 -- 172.21.15.78:0/3139338901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87802e510 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc865ffb640 1 -- 172.21.15.78:0/3139338901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc87802e930 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.110+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc878005450 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc865ffb640 1 -- 172.21.15.78:0/3139338901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc878041050 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 10 client.17418 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5612e3dc27f0 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc865ffb640 1 -- 172.21.15.78:0/3139338901 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc878005450 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc865ffb640 1 -- 172.21.15.78:0/3139338901 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc878024040 con 0x5612e39816b0 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 10 client.17418 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 20 client.17418 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e39363a0 0x5612e3dc5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 4 client.17418 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612e3cb0800 con 0x5612e39363a0 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.111+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e39363a0 0x5612e3dc5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc865ffb640 10 client.17418 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc8818d3640 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e39363a0 0x5612e3dc5150 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7da65890719d217 server_cookie=36198a91354b3d5a in_seq=0 out_seq=0 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc865ffb640 1 -- 172.21.15.78:0/3139338901 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5612e3cb0800 con 0x5612e39363a0 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc865ffb640 10 client.17418 handle_command_reply: tid=0 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 client.17418 shutdown 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 msgr2=0x5612e3d52050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d52050 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc87800f6a0 tx=0x7fc878009d90 comp rx=0 tx=0).stop 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 shutdown_connections 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5612e39a2390 0x5612e3dc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612e39816b0 0x5612e3d52050 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5612e39363a0 0x5612e3dc5150 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 --2- 172.21.15.78:0/3139338901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612e39794d0 0x5612e3d51b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc865ffb640 0 client.17418 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 >> 172.21.15.78:0/3139338901 conn(0x5612e39964e0 msgr2=0x5612e3d3ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 shutdown_connections 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 1 -- 172.21.15.78:0/3139338901 wait complete. 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc88b1ee740 20 client.17418 trim_cache size 0 max 16384 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:05:48.119 INFO:teuthology.orchestra.run.smithi078.stdout: "967d8799-2269-424e-81f5-ce790c6001f2": { 2024-04-25T14:05:48.120 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:05:48.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "967d8799-2269-424e-81f5-ce790c6001f2", 2024-04-25T14:05:48.120 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:05:48.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:05:48.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:05:48.120 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8680afaf0 msgr2=0x7fc8680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.112+0000 7fc884338640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8680afaf0 0x7fc8680b2000 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8740a2c20 tx=0x7fc874005b40 comp rx=0 tx=0).stop 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c1098a0 msgr2=0x7fc87c1a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c1098a0 0x7fc87c1a1750 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e9d0 tx=0x7fc87400ee90 comp rx=0 tx=0).stop 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 shutdown_connections 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc8680afaf0 0x7fc8680b2000 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c1098a0 0x7fc87c1a1750 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc87c106f40 0x7fc87c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 --2- 172.21.15.78:0/2738205709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc87c1045e0 0x7fc87c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 >> 172.21.15.78:0/2738205709 conn(0x7fc87c0fdbc0 msgr2=0x7fc87c1065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 shutdown_connections 2024-04-25T14:05:48.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:05:48.113+0000 7fc884338640 1 -- 172.21.15.78:0/2738205709 wait complete. 2024-04-25T14:05:48.123 INFO:tasks.cephfs.filesystem:scrub status for tag:967d8799-2269-424e-81f5-ce790c6001f2 - {'path': '/', 'tag': '967d8799-2269-424e-81f5-ce790c6001f2', 'options': 'recursive,force'} 2024-04-25T14:05:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:48 smithi078 ceph-mon[29163]: pgmap v536: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:05:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:48 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:05:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:48 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:48 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:05:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:48 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:48 smithi112 ceph-mon[32617]: pgmap v536: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:05:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:48 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:05:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:48 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:48 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:05:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:48 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:48 smithi129 ceph-mon[32569]: pgmap v536: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:05:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:48 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:05:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:48 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:48 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:05:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:48 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:05:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:50 smithi112 ceph-mon[32617]: pgmap v537: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:05:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:50 smithi129 ceph-mon[32569]: pgmap v537: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:05:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:50 smithi078 ceph-mon[29163]: pgmap v537: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:05:51.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:51 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:05:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:51 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:05:51.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:51 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:05:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.071+0000 7f090337d640 1 -- 172.21.15.78:0/4109418394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 msgr2=0x7f08fc106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.071+0000 7f090337d640 1 --2- 172.21.15.78:0/4109418394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc106a50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009900 tx=0x7f08f802a410 comp rx=0 tx=0).stop 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.071+0000 7f090337d640 1 -- 172.21.15.78:0/4109418394 shutdown_connections 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.071+0000 7f090337d640 1 --2- 172.21.15.78:0/4109418394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc1098f0 0x7f08fc10bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.071+0000 7f090337d640 1 --2- 172.21.15.78:0/4109418394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08fc106f90 0x7f08fc1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.071+0000 7f090337d640 1 --2- 172.21.15.78:0/4109418394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc106a50 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.071+0000 7f090337d640 1 -- 172.21.15.78:0/4109418394 >> 172.21.15.78:0/4109418394 conn(0x7f08fc0fdc30 msgr2=0x7f08fc1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 -- 172.21.15.78:0/4109418394 shutdown_connections 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 -- 172.21.15.78:0/4109418394 wait complete. 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 Processor -- start 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 -- start start 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc106f90 0x7f08fc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08fc1098f0 0x7f08fc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc10f640 con 0x7f08fc104630 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc10f4c0 con 0x7f08fc106f90 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f090337d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc10f7c0 con 0x7f08fc1098f0 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f0901119640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.072+0000 7f0901119640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50270/0 (socket says 172.21.15.78:50270) 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0901119640 1 -- 172.21.15.78:0/1262126438 learned_addr learned my addr 172.21.15.78:0/1262126438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0900918640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc106f90 0x7f08fc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f090191a640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08fc1098f0 0x7f08fc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0901119640 1 -- 172.21.15.78:0/1262126438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08fc1098f0 msgr2=0x7f08fc1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0901119640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08fc1098f0 0x7f08fc1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0901119640 1 -- 172.21.15.78:0/1262126438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc106f90 msgr2=0x7f08fc19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0901119640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc106f90 0x7f08fc19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0901119640 1 -- 172.21.15.78:0/1262126438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc1a1d80 con 0x7f08fc104630 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f0901119640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc19a240 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006010 tx=0x7f08f8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f08f27fc640 1 -- 172.21.15.78:0/1262126438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f803f030 con 0x7f08fc104630 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f08f27fc640 1 -- 172.21.15.78:0/1262126438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f8003aa0 con 0x7f08fc104630 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc1a2010 con 0x7f08fc104630 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f08f27fc640 1 -- 172.21.15.78:0/1262126438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8035070 con 0x7f08fc104630 2024-04-25T14:05:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.073+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc1a2550 con 0x7f08fc104630 2024-04-25T14:05:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.076+0000 7f08f27fc640 1 -- 172.21.15.78:0/1262126438 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f08f8042030 con 0x7f08fc104630 2024-04-25T14:05:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.076+0000 7f090a22e740 1 Processor -- start 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.076+0000 7f090a22e740 1 -- start start 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.076+0000 7f090a22e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607ea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.076+0000 7f090a22e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090a22e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090a22e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555560750460 con 0x55556044a1b0 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090a22e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55556070a590 con 0x5555604212f0 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090a22e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55556074c420 con 0x5555604294d0 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607ea970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50272/0 (socket says 172.21.15.78:50272) 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 -- 172.21.15.78:0/3381684821 learned_addr learned my addr 172.21.15.78:0/3381684821 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f0900918640 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f08f27fc640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08e80afb60 0x7f08e80b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f0901119640 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 -- 172.21.15.78:0/3381684821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 msgr2=0x5555607f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 -- 172.21.15.78:0/3381684821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 msgr2=0x5555607eaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607eaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 -- 172.21.15.78:0/3381684821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555607170d0 con 0x55556044a1b0 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f0900918640 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08e80afb60 0x7f08e80b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f08f27fc640 1 -- 172.21.15.78:0/1262126438 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f08f80ff1a0 con 0x7f08fc104630 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f090191a640 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607ea970 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f08f40103a0 tx=0x7f08f4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1a433d6400ab47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f08e77fe640 1 -- 172.21.15.78:0/3381684821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4021070 con 0x55556044a1b0 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f08e77fe640 1 -- 172.21.15.78:0/3381684821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f4013be0 con 0x55556044a1b0 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.077+0000 7f08e77fe640 1 -- 172.21.15.78:0/3381684821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f401b680 con 0x55556044a1b0 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 -- 172.21.15.78:0/3381684821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 msgr2=0x5555607ea970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090191a640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08e80afb60 0x7f08e80b2070 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f08f401e000 tx=0x7f08f401bdb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607ea970 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f08f40103a0 tx=0x7f08f4010860 comp rx=0 tx=0).stop 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 -- 172.21.15.78:0/3381684821 shutdown_connections 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607eaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 --2- 172.21.15.78:0/3381684821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607ea970 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 -- 172.21.15.78:0/3381684821 >> 172.21.15.78:0/3381684821 conn(0x55556043e300 msgr2=0x5555607e6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 -- 172.21.15.78:0/3381684821 shutdown_connections 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.078+0000 7f090a22e740 1 -- 172.21.15.78:0/3381684821 wait complete. 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 -- start start 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 Processor -- start 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5555607f2840 con 0x55556044a1b0 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5555606d7b60 con 0x5555604212f0 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5555607f29c0 con 0x5555604294d0 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090a22e740 10 client.0 _ll_register_callbacks cb 0x5555606fc2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50278/0 (socket says 172.21.15.78:50278) 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.079+0000 7f090191a640 1 -- 172.21.15.78:0/3530620124 learned_addr learned my addr 172.21.15.78:0/3530620124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f0900918640 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f0901119640 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090191a640 1 -- 172.21.15.78:0/3530620124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 msgr2=0x5555607f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090191a640 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090191a640 1 -- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 msgr2=0x5555607f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090191a640 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090191a640 1 -- 172.21.15.78:0/3530620124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555560462210 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090191a640 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fbb10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f08f4010370 tx=0x7f08f400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d7d3e39fd45ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f08e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f08e4ff9640 1 -- 172.21.15.78:0/3530620124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4021040 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f08e4ff9640 1 -- 172.21.15.78:0/3530620124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f08f401b3c0 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f4013be0 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f08e4ff9640 1 -- 172.21.15.78:0/3530620124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f08f401b5c0 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090a22e740 10 client.17430 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55556052e620 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.080+0000 7f08e4ff9640 1 -- 172.21.15.78:0/3530620124 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f4013be0 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f08e4ff9640 1 -- 172.21.15.78:0/3530620124 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f08f400f040 con 0x55556044a1b0 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f090a22e740 10 client.17430 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f090a22e740 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f090a22e740 20 client.17430 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f090a22e740 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555560408d80 0x555560802240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f090a22e740 4 client.17430 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55556075e070 con 0x555560408d80 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f0901119640 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555560408d80 0x555560802240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f08e4ff9640 10 client.17430 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.081+0000 7f0901119640 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555560408d80 0x555560802240 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7660a7d5ae05c27 server_cookie=47e42cd70ae84b7d in_seq=0 out_seq=0 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f08e4ff9640 1 -- 172.21.15.78:0/3530620124 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55556075e070 con 0x555560408d80 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f08e4ff9640 10 client.17430 handle_command_reply: tid=0 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 client.17430 shutdown 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 msgr2=0x5555607fbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fbb10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f08f4010370 tx=0x7f08f400e000 comp rx=0 tx=0).stop 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 shutdown_connections 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fbb10 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x555560408d80 0x555560802240 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604294d0 0x5555607f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f08e4ff9640 0 client.17430 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 --2- 172.21.15.78:0/3530620124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604212f0 0x5555607f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 >> 172.21.15.78:0/3530620124 conn(0x55556043e300 msgr2=0x5555607e9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 shutdown_connections 2024-04-25T14:05:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.082+0000 7f090a22e740 1 -- 172.21.15.78:0/3530620124 wait complete. 2024-04-25T14:05:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.083+0000 7f090a22e740 20 client.17430 trim_cache size 0 max 16384 2024-04-25T14:05:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 Processor -- start 2024-04-25T14:05:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 -- start start 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607fba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1adb30 con 0x55556044a1b0 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1ab400 con 0x5555604294d0 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090a22e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1a9250 con 0x5555604212f0 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50280/0 (socket says 172.21.15.78:50280) 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 -- 172.21.15.78:0/2836540026 learned_addr learned my addr 172.21.15.78:0/2836540026 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f0901119640 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f0900918640 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 -- 172.21.15.78:0/2836540026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 msgr2=0x5555607f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 -- 172.21.15.78:0/2836540026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 msgr2=0x5555607fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607fba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 -- 172.21.15.78:0/2836540026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555607fbf70 con 0x55556044a1b0 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f0900918640 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f090191a640 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607f32a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f08f40096c0 tx=0x7f08f4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5602cb20cac1f95d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f08e6ffd640 1 -- 172.21.15.78:0/2836540026 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4021070 con 0x55556044a1b0 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f08e6ffd640 1 -- 172.21.15.78:0/2836540026 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f4033070 con 0x55556044a1b0 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.098+0000 7f08e6ffd640 1 -- 172.21.15.78:0/2836540026 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f401ca70 con 0x55556044a1b0 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 -- 172.21.15.78:0/2836540026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 msgr2=0x5555607f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607f32a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f08f40096c0 tx=0x7f08f4010860 comp rx=0 tx=0).stop 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 -- 172.21.15.78:0/2836540026 shutdown_connections 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607f32a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 --2- 172.21.15.78:0/2836540026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 -- 172.21.15.78:0/2836540026 >> 172.21.15.78:0/2836540026 conn(0x55556043e300 msgr2=0x5555607e5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 -- 172.21.15.78:0/2836540026 shutdown_connections 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.099+0000 7f090a22e740 1 -- 172.21.15.78:0/2836540026 wait complete. 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 -- start start 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 Processor -- start 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1a98c0 con 0x55556044a1b0 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1aad30 con 0x5555604294d0 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1a9f90 con 0x5555604212f0 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090a22e740 10 client.0 _ll_register_callbacks cb 0x7f08f401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:05:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50294/0 (socket says 172.21.15.78:50294) 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 -- 172.21.15.78:0/1522103567 learned_addr learned my addr 172.21.15.78:0/1522103567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f0901119640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f0900918640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 -- 172.21.15.78:0/1522103567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 msgr2=0x5555607f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 -- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 msgr2=0x5555607f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.100+0000 7f090191a640 1 -- 172.21.15.78:0/1522103567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555607fa730 con 0x55556044a1b0 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f0900918640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090191a640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fa170 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f08f400a010 tx=0x7f08f401abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd1b8eae640769d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:05:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f08e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f08e4ff9640 1 -- 172.21.15.78:0/1522103567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4050060 con 0x55556044a1b0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f08e4ff9640 1 -- 172.21.15.78:0/1522103567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f08f401b760 con 0x55556044a1b0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f4033070 con 0x55556044a1b0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f08e4ff9640 1 -- 172.21.15.78:0/1522103567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f08f4021040 con 0x55556044a1b0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 10 client.17436 fetch_fsmap learned FSMap version 22 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5555607fb930 con 0x55556044a1b0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f08e4ff9640 1 -- 172.21.15.78:0/1522103567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f4033070 con 0x55556044a1b0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f08e4ff9640 1 -- 172.21.15.78:0/1522103567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f08f404b570 con 0x55556044a1b0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 10 client.17436 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 10 client.17436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 20 client.17436 populate_metadata read hostname 'smithi078' 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5555603de1c0 0x55556086c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 4 client.17436 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.101+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555560757ee0 con 0x5555603de1c0 2024-04-25T14:05:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.102+0000 7f0901119640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5555603de1c0 0x55556086c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:05:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.102+0000 7f08e4ff9640 10 client.17436 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.102+0000 7f0901119640 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5555603de1c0 0x55556086c9b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e26e937c09889c6 server_cookie=73050652d1b20b38 in_seq=0 out_seq=0 2024-04-25T14:05:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f08e4ff9640 1 -- 172.21.15.78:0/1522103567 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555560757ee0 con 0x5555603de1c0 2024-04-25T14:05:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f08e4ff9640 10 client.17436 handle_command_reply: tid=0 2024-04-25T14:05:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 client.17436 shutdown 2024-04-25T14:05:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 msgr2=0x5555607fa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fa170 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f08f400a010 tx=0x7f08f401abb0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 shutdown_connections 2024-04-25T14:05:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55556044a1b0 0x5555607fa170 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5555604294d0 0x5555607f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5555603de1c0 0x55556086c9b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 --2- 172.21.15.78:0/1522103567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5555604212f0 0x5555607f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f08e4ff9640 0 client.17436 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 >> 172.21.15.78:0/1522103567 conn(0x55556043e300 msgr2=0x5555607f2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 shutdown_connections 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 1 -- 172.21.15.78:0/1522103567 wait complete. 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090a22e740 20 client.17436 trim_cache size 0 max 16384 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:05:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:05:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:05:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:05:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08e80afb60 msgr2=0x7f08e80b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08e80afb60 0x7f08e80b2070 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f08f401e000 tx=0x7f08f401bdb0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 msgr2=0x7f08fc19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc19a240 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006010 tx=0x7f08f8031a50 comp rx=0 tx=0).stop 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 shutdown_connections 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08fc1098f0 0x7f08fc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc106f90 0x7f08fc19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08e80afb60 0x7f08e80b2070 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 --2- 172.21.15.78:0/1262126438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08fc104630 0x7f08fc19a240 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 >> 172.21.15.78:0/1262126438 conn(0x7f08fc0fdc30 msgr2=0x7f08fc105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 shutdown_connections 2024-04-25T14:05:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:05:52.118+0000 7f090337d640 1 -- 172.21.15.78:0/1262126438 wait complete. 2024-04-25T14:05:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:52 smithi112 ceph-mon[32617]: pgmap v538: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:05:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:52 smithi129 ceph-mon[32569]: pgmap v538: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:05:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:52 smithi078 ceph-mon[29163]: pgmap v538: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:05:54.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:54 smithi112 ceph-mon[32617]: pgmap v539: 97 pgs: 97 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:05:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:54 smithi129 ceph-mon[32569]: pgmap v539: 97 pgs: 97 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:05:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:54 smithi078 ceph-mon[29163]: pgmap v539: 97 pgs: 97 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:05:55.956 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.956 INFO:tasks.workunit.client.0.smithi078.stdout:251/600: open fd = 3 2024-04-25T14:05:55.956 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.956 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:252/600: open fd = 3 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:253/600: open fd = 3 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:254/600: open fd = 3 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.957 INFO:tasks.workunit.client.0.smithi078.stdout:255/600: open fd = 3 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:256/600: open fd = 3 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:257/600: open fd = 3 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:258/600: open fd = 3 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.958 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:259/600: open fd = 3 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:260/600: open fd = 3 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:261/600: open fd = 3 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.959 INFO:tasks.workunit.client.0.smithi078.stdout:262/600: open fd = 3 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:263/600: open fd = 3 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:264/600: open fd = 3 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:265/600: open fd = 3 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.960 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:266/600: open fd = 3 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:267/600: open fd = 3 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:268/600: open fd = 3 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:269/600: open fd = 3 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.961 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:270/600: open fd = 3 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:271/600: open fd = 3 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:272/600: open fd = 3 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.962 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:273/600: open fd = 3 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:274/600: open fd = 3 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:275/600: open fd = 3 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:276/600: open fd = 3 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.963 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:277/600: open fd = 3 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:278/600: open fd = 3 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:279/600: open fd = 3 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.964 INFO:tasks.workunit.client.0.smithi078.stdout:280/600: open fd = 3 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:281/600: open fd = 3 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:282/600: open fd = 3 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:283/600: open fd = 3 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.965 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:284/600: open fd = 3 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:285/600: open fd = 3 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:286/600: open fd = 3 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.966 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:287/600: open fd = 3 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:288/600: open fd = 3 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:289/600: open fd = 3 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:290/600: open fd = 3 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.967 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:291/600: open fd = 3 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:292/600: open fd = 3 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:293/600: open fd = 3 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:294/600: open fd = 3 2024-04-25T14:05:55.968 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:295/600: open fd = 3 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:296/600: open fd = 3 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:297/600: open fd = 3 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.969 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:298/600: open fd = 3 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:299/600: open fd = 3 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:300/600: open fd = 3 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:301/600: open fd = 3 2024-04-25T14:05:55.970 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:302/600: open fd = 3 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:303/600: open fd = 3 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:304/600: open fd = 3 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.971 INFO:tasks.workunit.client.0.smithi078.stdout:305/600: open fd = 3 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:306/600: open fd = 3 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:307/600: open fd = 3 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:308/600: open fd = 3 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.972 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:309/600: open fd = 3 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:310/600: open fd = 3 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:311/600: open fd = 3 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:312/600: open fd = 3 2024-04-25T14:05:55.973 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:55.974 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:05:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:56 smithi112 ceph-mon[32617]: pgmap v540: 97 pgs: 97 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:05:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:56 smithi129 ceph-mon[32569]: pgmap v540: 97 pgs: 97 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:05:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:56 smithi078 ceph-mon[29163]: pgmap v540: 97 pgs: 97 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:05:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:05:58 smithi112 ceph-mon[32617]: pgmap v541: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:05:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:05:58 smithi129 ceph-mon[32569]: pgmap v541: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:05:58.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:05:58 smithi078 ceph-mon[29163]: pgmap v541: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-25T14:06:00.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:00 smithi112 ceph-mon[32617]: pgmap v542: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-25T14:06:00.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:00 smithi078 ceph-mon[29163]: pgmap v542: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-25T14:06:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:00 smithi129 ceph-mon[32569]: pgmap v542: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-25T14:06:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:02 smithi112 ceph-mon[32617]: pgmap v543: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:02 smithi129 ceph-mon[32569]: pgmap v543: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:02 smithi078 ceph-mon[29163]: pgmap v543: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:04 smithi129 ceph-mon[32569]: pgmap v544: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:06:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:04 smithi078 ceph-mon[29163]: pgmap v544: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:06:04.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:04 smithi112 ceph-mon[32617]: pgmap v544: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:06:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 -- 172.21.15.78:0/2445685938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 msgr2=0x7f87f810bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/2445685938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f810bcd0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009ca0 tx=0x7f87f002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 -- 172.21.15.78:0/2445685938 shutdown_connections 2024-04-25T14:06:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/2445685938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f810bcd0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/2445685938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f8106c50 0x7f87f8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/2445685938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87f8101e70 0x7f87f8102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 -- 172.21.15.78:0/2445685938 >> 172.21.15.78:0/2445685938 conn(0x7f87f80fd3a0 msgr2=0x7f87f80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 -- 172.21.15.78:0/2445685938 shutdown_connections 2024-04-25T14:06:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.223+0000 7f87fd0b6640 1 -- 172.21.15.78:0/2445685938 wait complete. 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.224+0000 7f87fd0b6640 1 Processor -- start 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.224+0000 7f87fd0b6640 1 -- start start 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.224+0000 7f87fd0b6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87f8101e70 0x7f87f819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87fd0b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f8106c50 0x7f87f819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87fd0b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f81a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87fd0b6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810f630 con 0x7f87f81097c0 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87fd0b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810f4b0 con 0x7f87f8106c50 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87fd0b6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87f810f7b0 con 0x7f87f8101e70 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f81a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f81a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49046/0 (socket says 172.21.15.78:49046) 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 -- 172.21.15.78:0/4092266662 learned_addr learned my addr 172.21.15.78:0/4092266662 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87f8101e70 0x7f87f819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f659c640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f8106c50 0x7f87f819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 -- 172.21.15.78:0/4092266662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87f8101e70 msgr2=0x7f87f819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87f8101e70 0x7f87f819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 -- 172.21.15.78:0/4092266662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f8106c50 msgr2=0x7f87f819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f8106c50 0x7f87f819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 -- 172.21.15.78:0/4092266662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f81a1cf0 con 0x7f87f81097c0 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87f759e640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f81a1680 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f87f00097b0 tx=0x7f87f0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87dffff640 1 -- 172.21.15.78:0/4092266662 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0004240 con 0x7f87f81097c0 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81a1f80 con 0x7f87f81097c0 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87dffff640 1 -- 172.21.15.78:0/4092266662 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f002f380 con 0x7f87f81097c0 2024-04-25T14:06:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87dffff640 1 -- 172.21.15.78:0/4092266662 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f002fc30 con 0x7f87f81097c0 2024-04-25T14:06:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.225+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81a23e0 con 0x7f87f81097c0 2024-04-25T14:06:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.228+0000 7f8803f69740 1 Processor -- start 2024-04-25T14:06:05.234 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.228+0000 7f87dffff640 1 -- 172.21.15.78:0/4092266662 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f87f00043e0 con 0x7f87f81097c0 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.228+0000 7f8803f69740 1 -- start start 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f8803f69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5538970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f8803f69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee516f2f0 0x559ee5538ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f8803f69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee51774d0 0x559ee553fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f8803f69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ee549e460 con 0x559ee516f2f0 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f8803f69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559ee5458590 con 0x559ee51774d0 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f8803f69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ee549a420 con 0x559ee51981b0 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee516f2f0 0x559ee5538ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee516f2f0 0x559ee5538ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49058/0 (socket says 172.21.15.78:49058) 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 -- 172.21.15.78:0/2593625161 learned_addr learned my addr 172.21.15.78:0/2593625161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f659c640 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee51774d0 0x559ee553fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f659c640 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5538970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 -- 172.21.15.78:0/2593625161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 msgr2=0x559ee5538970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5538970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 -- 172.21.15.78:0/2593625161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee51774d0 msgr2=0x559ee553fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee51774d0 0x559ee553fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 -- 172.21.15.78:0/2593625161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee54650d0 con 0x559ee516f2f0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f659c640 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee51774d0 0x559ee553fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87dffff640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87e00afae0 0x7f87e00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f659c640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87e00afae0 0x7f87e00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee516f2f0 0x559ee5538ef0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f87e800bce0 tx=0x7f87e800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b524f95848627b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.229+0000 7f87dffff640 1 -- 172.21.15.78:0/4092266662 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f87f0100470 con 0x7f87f81097c0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f87dd7fa640 1 -- 172.21.15.78:0/2593625161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8015070 con 0x559ee516f2f0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f87dd7fa640 1 -- 172.21.15.78:0/2593625161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e801a910 con 0x559ee516f2f0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f87dd7fa640 1 -- 172.21.15.78:0/2593625161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8011900 con 0x559ee516f2f0 2024-04-25T14:06:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f8803f69740 1 -- 172.21.15.78:0/2593625161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee516f2f0 msgr2=0x559ee5538ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f87f659c640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87e00afae0 0x7f87e00b1ff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x559ee5539b40 tx=0x7f87ec005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f8803f69740 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee516f2f0 0x559ee5538ef0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f87e800bce0 tx=0x7f87e800f730 comp rx=0 tx=0).stop 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f8803f69740 1 -- 172.21.15.78:0/2593625161 shutdown_connections 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f8803f69740 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee51774d0 0x559ee553fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.230+0000 7f8803f69740 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee516f2f0 0x559ee5538ef0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 --2- 172.21.15.78:0/2593625161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5538970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 -- 172.21.15.78:0/2593625161 >> 172.21.15.78:0/2593625161 conn(0x559ee518c300 msgr2=0x559ee5534300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 -- 172.21.15.78:0/2593625161 shutdown_connections 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 -- 172.21.15.78:0/2593625161 wait complete. 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 -- start start 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 Processor -- start 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5542670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5542bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5549b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ee5540840 con 0x559ee51774d0 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559ee5425b60 con 0x559ee516f2f0 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ee55409c0 con 0x559ee51981b0 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f8803f69740 10 client.0 _ll_register_callbacks cb 0x559ee544a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f87f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5542bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f87f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5542bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49062/0 (socket says 172.21.15.78:49062) 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f87f659c640 1 -- 172.21.15.78:0/1877967385 learned_addr learned my addr 172.21.15.78:0/1877967385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.231+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5542670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87f759e640 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5549b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87f659c640 1 -- 172.21.15.78:0/1877967385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 msgr2=0x559ee5549b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87f659c640 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5549b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87f659c640 1 -- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 msgr2=0x559ee5542670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87f659c640 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5542670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87f659c640 1 -- 172.21.15.78:0/1877967385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee51b0210 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87f659c640 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5542bb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f87ec011c30 tx=0x7f87ec005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27482bbac93dbf00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87daff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87daff5640 1 -- 172.21.15.78:0/1877967385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec011e60 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87daff5640 1 -- 172.21.15.78:0/1877967385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f87ec004030 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e801a910 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f87daff5640 1 -- 172.21.15.78:0/1877967385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f87ec004230 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f8803f69740 10 client.17448 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.232+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559ee527c620 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f87daff5640 1 -- 172.21.15.78:0/1877967385 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e801a910 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f87daff5640 1 -- 172.21.15.78:0/1877967385 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f87ec030050 con 0x559ee51774d0 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f8803f69740 10 client.17448 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f8803f69740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f8803f69740 20 client.17448 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f8803f69740 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee5156d80 0x559ee55503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f8803f69740 4 client.17448 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee54ac070 con 0x559ee5156d80 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee5156d80 0x559ee55503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f87daff5640 10 client.17448 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.233+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee5156d80 0x559ee55503c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=201a4840ba92885f server_cookie=8d8b21b69c7a7efc in_seq=0 out_seq=0 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f87daff5640 1 -- 172.21.15.78:0/1877967385 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ee54ac070 con 0x559ee5156d80 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f87daff5640 10 client.17448 handle_command_reply: tid=0 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 client.17448 shutdown 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 msgr2=0x559ee5542bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5542bb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f87ec011c30 tx=0x7f87ec005d30 comp rx=0 tx=0).stop 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 shutdown_connections 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee5156d80 0x559ee55503c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5549b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5542bb0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f87daff5640 0 client.17448 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 --2- 172.21.15.78:0/1877967385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5542670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 >> 172.21.15.78:0/1877967385 conn(0x559ee518c300 msgr2=0x559ee5537210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.234+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 shutdown_connections 2024-04-25T14:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.235+0000 7f8803f69740 1 -- 172.21.15.78:0/1877967385 wait complete. 2024-04-25T14:06:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.235+0000 7f8803f69740 20 client.17448 trim_cache size 0 max 16384 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.249+0000 7f8803f69740 1 Processor -- start 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f8803f69740 1 -- start start 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f8803f69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5537810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f8803f69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5549a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f8803f69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5533c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f8803f69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81adb10 con 0x559ee51774d0 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f8803f69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81ab3e0 con 0x559ee516f2f0 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f8803f69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a9230 con 0x559ee51981b0 2024-04-25T14:06:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5549a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5549a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49072/0 (socket says 172.21.15.78:49072) 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 -- 172.21.15.78:0/3657230086 learned_addr learned my addr 172.21.15.78:0/3657230086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f759e640 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5533c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 -- 172.21.15.78:0/3657230086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 msgr2=0x559ee5533c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5533c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5537810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 -- 172.21.15.78:0/3657230086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 msgr2=0x559ee5537810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5537810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 -- 172.21.15.78:0/3657230086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee553d4b0 con 0x559ee51774d0 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5537810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87f659c640 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5549a30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f87ec01f6e0 tx=0x7f87ec011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369eb99202d81587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.250+0000 7f87dcff9640 1 -- 172.21.15.78:0/3657230086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0043d0 con 0x559ee51774d0 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f87dcff9640 1 -- 172.21.15.78:0/3657230086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec01f8d0 con 0x559ee51774d0 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f87dcff9640 1 -- 172.21.15.78:0/3657230086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec003d00 con 0x559ee51774d0 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 -- 172.21.15.78:0/3657230086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 msgr2=0x559ee5549a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5549a30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f87ec01f6e0 tx=0x7f87ec011910 comp rx=0 tx=0).stop 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 -- 172.21.15.78:0/3657230086 shutdown_connections 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51981b0 0x559ee5533c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51774d0 0x559ee5549a30 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 --2- 172.21.15.78:0/3657230086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5537810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 -- 172.21.15.78:0/3657230086 >> 172.21.15.78:0/3657230086 conn(0x559ee518c300 msgr2=0x559ee5537100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 -- 172.21.15.78:0/3657230086 shutdown_connections 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.251+0000 7f8803f69740 1 -- 172.21.15.78:0/3657230086 wait complete. 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 -- start start 2024-04-25T14:06:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 Processor -- start 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5547570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51774d0 0x559ee5547ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51981b0 0x559ee5548990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a98a0 con 0x559ee51981b0 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81aad10 con 0x559ee516f2f0 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81a9f70 con 0x559ee51774d0 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f8803f69740 10 client.0 _ll_register_callbacks cb 0x7f87ec00eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51981b0 0x559ee5548990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51981b0 0x559ee5548990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49086/0 (socket says 172.21.15.78:49086) 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 -- 172.21.15.78:0/1371061904 learned_addr learned my addr 172.21.15.78:0/1371061904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f659c640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51774d0 0x559ee5547ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5547570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 -- 172.21.15.78:0/1371061904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51774d0 msgr2=0x559ee5547ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51774d0 0x559ee5547ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 -- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 msgr2=0x559ee5547570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5547570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f759e640 1 -- 172.21.15.78:0/1371061904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee5549f70 con 0x559ee51981b0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.252+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5547570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87f759e640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51981b0 0x559ee5548990 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f87f00fdf60 tx=0x7f87f002f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d32e50bdc13eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87daff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87daff5640 1 -- 172.21.15.78:0/1371061904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00098b0 con 0x559ee51981b0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87daff5640 1 -- 172.21.15.78:0/1371061904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f87f00046d0 con 0x559ee51981b0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ec01f8d0 con 0x559ee51981b0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87daff5640 1 -- 172.21.15.78:0/1371061904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f87f00c8940 con 0x559ee51981b0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 10 client.17454 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559ee55b88a0 con 0x559ee51981b0 2024-04-25T14:06:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87daff5640 1 -- 172.21.15.78:0/1371061904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec01f8d0 con 0x559ee51981b0 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87daff5640 1 -- 172.21.15.78:0/1371061904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f87f00eb0c0 con 0x559ee51981b0 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 10 client.17454 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 20 client.17454 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee512c1c0 0x559ee55bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 4 client.17454 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ee54a5ee0 con 0x559ee512c1c0 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.253+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee512c1c0 0x559ee55bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.254+0000 7f87daff5640 10 client.17454 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.254+0000 7f87f6d9d640 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee512c1c0 0x559ee55bb260 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede7889cbfd958e7 server_cookie=fc8203fc05a2effd in_seq=0 out_seq=0 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f87daff5640 1 -- 172.21.15.78:0/1371061904 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ee54a5ee0 con 0x559ee512c1c0 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f87daff5640 10 client.17454 handle_command_reply: tid=0 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 client.17454 shutdown 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51981b0 msgr2=0x559ee5548990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51981b0 0x559ee5548990 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f87f00fdf60 tx=0x7f87f002f6f0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 shutdown_connections 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ee51981b0 0x559ee5548990 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ee512c1c0 0x559ee55bb260 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ee51774d0 0x559ee5547ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 --2- 172.21.15.78:0/1371061904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ee516f2f0 0x559ee5547570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f87daff5640 0 client.17454 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.273+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 >> 172.21.15.78:0/1371061904 conn(0x559ee518c300 msgr2=0x559ee5540dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 shutdown_connections 2024-04-25T14:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f8803f69740 1 -- 172.21.15.78:0/1371061904 wait complete. 2024-04-25T14:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f8803f69740 20 client.17454 trim_cache size 0 max 16384 2024-04-25T14:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:06:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87e00afae0 msgr2=0x7f87e00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87e00afae0 0x7f87e00b1ff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x559ee5539b40 tx=0x7f87ec005da0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 msgr2=0x7f87f81a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f81a1680 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f87f00097b0 tx=0x7f87f0003c20 comp rx=0 tx=0).stop 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 shutdown_connections 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f87e00afae0 0x7f87e00b1ff0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f81097c0 0x7f87f81a1680 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87f8106c50 0x7f87f819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 --2- 172.21.15.78:0/4092266662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87f8101e70 0x7f87f819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 >> 172.21.15.78:0/4092266662 conn(0x7f87f80fd3a0 msgr2=0x7f87f81077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 shutdown_connections 2024-04-25T14:06:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:05.274+0000 7f87fd0b6640 1 -- 172.21.15.78:0/4092266662 wait complete. 2024-04-25T14:06:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:06 smithi129 ceph-mon[32569]: pgmap v545: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:06:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:06 smithi078 ceph-mon[29163]: pgmap v545: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:06:06.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:06 smithi112 ceph-mon[32617]: pgmap v545: 97 pgs: 97 active+clean; 4.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:06:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:08 smithi129 ceph-mon[32569]: pgmap v546: 97 pgs: 97 active+clean; 582 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-25T14:06:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:08 smithi078 ceph-mon[29163]: pgmap v546: 97 pgs: 97 active+clean; 582 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-25T14:06:08.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:08 smithi112 ceph-mon[32617]: pgmap v546: 97 pgs: 97 active+clean; 582 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-25T14:06:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:10 smithi129 ceph-mon[32569]: pgmap v547: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-25T14:06:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:10 smithi078 ceph-mon[29163]: pgmap v547: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-25T14:06:10.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:10 smithi112 ceph-mon[32617]: pgmap v547: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-25T14:06:12.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:12 smithi129 ceph-mon[32569]: pgmap v548: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-25T14:06:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:12 smithi078 ceph-mon[29163]: pgmap v548: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-25T14:06:12.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:12 smithi112 ceph-mon[32617]: pgmap v548: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-25T14:06:13.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:13 smithi129 ceph-mon[32569]: pgmap v549: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:13 smithi078 ceph-mon[29163]: pgmap v549: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:13 smithi112 ceph-mon[32617]: pgmap v549: 97 pgs: 97 active+clean; 647 KiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:15.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:15 smithi129 ceph-mon[32569]: pgmap v550: 97 pgs: 97 active+clean; 708 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:15.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:15 smithi078 ceph-mon[29163]: pgmap v550: 97 pgs: 97 active+clean; 708 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:15.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:15 smithi112 ceph-mon[32617]: pgmap v550: 97 pgs: 97 active+clean; 708 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:17.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:17 smithi129 ceph-mon[32569]: pgmap v551: 97 pgs: 97 active+clean; 708 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:17 smithi078 ceph-mon[29163]: pgmap v551: 97 pgs: 97 active+clean; 708 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:17.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:17 smithi112 ceph-mon[32617]: pgmap v551: 97 pgs: 97 active+clean; 708 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-25T14:06:18.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:06:18.124 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 -- 172.21.15.78:0/1899195464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f8106c40 msgr2=0x7fd4f8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/1899195464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f8106c40 0x7fd4f8109120 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009900 tx=0x7fd4ec02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 -- 172.21.15.78:0/1899195464 shutdown_connections 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/1899195464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4f81097b0 0x7fd4f810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/1899195464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f8106c40 0x7fd4f8109120 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/1899195464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101eb0 0x7fd4f81022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 -- 172.21.15.78:0/1899195464 >> 172.21.15.78:0/1899195464 conn(0x7fd4f80fd380 msgr2=0x7fd4f80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 -- 172.21.15.78:0/1899195464 shutdown_connections 2024-04-25T14:06:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.248+0000 7fd4fc97a640 1 -- 172.21.15.78:0/1899195464 wait complete. 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 Processor -- start 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 -- start start 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101eb0 0x7fd4f819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4f8106c40 0x7fd4f819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f81097b0 0x7fd4f81a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810f640 con 0x7fd4f81097b0 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810f4c0 con 0x7fd4f8101eb0 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4fc97a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810f7c0 con 0x7fd4f8106c40 2024-04-25T14:06:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f81097b0 0x7fd4f81a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f81097b0 0x7fd4f81a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44228/0 (socket says 172.21.15.78:44228) 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 -- 172.21.15.78:0/3585211756 learned_addr learned my addr 172.21.15.78:0/3585211756 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4f8106c40 0x7fd4f819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f659c640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101eb0 0x7fd4f819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 -- 172.21.15.78:0/3585211756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4f8106c40 msgr2=0x7fd4f819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4f8106c40 0x7fd4f819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 -- 172.21.15.78:0/3585211756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101eb0 msgr2=0x7fd4f819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101eb0 0x7fd4f819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.249+0000 7fd4f6d9d640 1 -- 172.21.15.78:0/3585211756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f81a1d50 con 0x7fd4f81097b0 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.250+0000 7fd4f6d9d640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f81097b0 0x7fd4f81a16e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000d750 tx=0x7fd4f000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.250+0000 7fd4e77fe640 1 -- 172.21.15.78:0/3585211756 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0017070 con 0x7fd4f81097b0 2024-04-25T14:06:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.250+0000 7fd4e77fe640 1 -- 172.21.15.78:0/3585211756 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f00054d0 con 0x7fd4f81097b0 2024-04-25T14:06:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.250+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f81a1fe0 con 0x7fd4f81097b0 2024-04-25T14:06:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.250+0000 7fd4e77fe640 1 -- 172.21.15.78:0/3585211756 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0004c60 con 0x7fd4f81097b0 2024-04-25T14:06:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.250+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a24a0 con 0x7fd4f81097b0 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.252+0000 7fd4e77fe640 1 -- 172.21.15.78:0/3585211756 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd4f0009bd0 con 0x7fd4f81097b0 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.252+0000 7fd50381f740 1 Processor -- start 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd50381f740 1 -- start start 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd50381f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f635a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd50381f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f635fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd50381f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f636c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd50381f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56039f59ad80 con 0x56039f294390 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd50381f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56039f5548b0 con 0x56039f26b4d0 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd50381f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56039f596d40 con 0x56039f2736b0 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4e77fe640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd4e00afaf0 0x7fd4e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f635a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f635a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44242/0 (socket says 172.21.15.78:44242) 2024-04-25T14:06:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3909460275 learned_addr learned my addr 172.21.15.78:0/3909460275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f6d9d640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd4e00afaf0 0x7fd4e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f659c640 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f635fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f636c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4e77fe640 1 -- 172.21.15.78:0/3585211756 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd4f00de4b0 con 0x7fd4f81097b0 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3909460275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 msgr2=0x56039f636c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f636c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3909460275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 msgr2=0x56039f635fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f635fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.253+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3909460275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039f525a40 con 0x56039f294390 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.254+0000 7fd4f659c640 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f635fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.254+0000 7fd4f6d9d640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd4e00afaf0 0x7fd4e00b2000 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd4f001d000 tx=0x7fd4f0075580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.254+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f635a60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0096c0 tx=0x7fd4ec009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709fb574c8070e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.254+0000 7fd4e4ff9640 1 -- 172.21.15.78:0/3909460275 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec035030 con 0x56039f294390 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.254+0000 7fd4e4ff9640 1 -- 172.21.15.78:0/3909460275 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4ec031720 con 0x56039f294390 2024-04-25T14:06:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.254+0000 7fd4e4ff9640 1 -- 172.21.15.78:0/3909460275 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec003c00 con 0x56039f294390 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 -- 172.21.15.78:0/3909460275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 msgr2=0x56039f635a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f635a60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0096c0 tx=0x7fd4ec009610 comp rx=0 tx=0).stop 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 -- 172.21.15.78:0/3909460275 shutdown_connections 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f636c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f635fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 --2- 172.21.15.78:0/3909460275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f635a60 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 -- 172.21.15.78:0/3909460275 >> 172.21.15.78:0/3909460275 conn(0x56039f2884e0 msgr2=0x56039f631270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 -- 172.21.15.78:0/3909460275 shutdown_connections 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 -- 172.21.15.78:0/3909460275 wait complete. 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 -- start start 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 Processor -- start 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f63f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.255+0000 7fd50381f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f63ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f646e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56039f637720 con 0x56039f294390 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56039f521d10 con 0x56039f26b4d0 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56039f6378a0 con 0x56039f2736b0 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 10 client.0 _ll_register_callbacks cb 0x56039f54e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f646e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f646e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44244/0 (socket says 172.21.15.78:44244) 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3602697856 learned_addr learned my addr 172.21.15.78:0/3602697856 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f63f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f659c640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f63ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3602697856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 msgr2=0x56039f63ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f63ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 msgr2=0x56039f63f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f63f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/3602697856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039f3767d0 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f659c640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f63ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f646e60 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec03e040 tx=0x7fd4ec046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c38007d3088c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4ddffb640 1 -- 172.21.15.78:0/3602697856 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec035070 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4ddffb640 1 -- 172.21.15.78:0/3602697856 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd4ec003980 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4ec031720 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd4ddffb640 1 -- 172.21.15.78:0/3602697856 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd4ec0318d0 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 10 client.17466 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.256+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56039f2ac3f0 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd4ddffb640 1 -- 172.21.15.78:0/3602697856 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4ec031720 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd4ddffb640 1 -- 172.21.15.78:0/3602697856 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd4ec04cca0 con 0x56039f294390 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd50381f740 10 client.17466 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd50381f740 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd50381f740 20 client.17466 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd50381f740 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f252f60 0x56039f64cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd50381f740 4 client.17466 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56039f5a8990 con 0x56039f252f60 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.257+0000 7fd4f659c640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f252f60 0x56039f64cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd4ddffb640 10 client.17466 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd4f659c640 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f252f60 0x56039f64cf00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ea056ef8768e server_cookie=9bb3c127fdcfb8d1 in_seq=0 out_seq=0 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd4ddffb640 1 -- 172.21.15.78:0/3602697856 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56039f5a8990 con 0x56039f252f60 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd4ddffb640 10 client.17466 handle_command_reply: tid=0 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd50381f740 1 client.17466 shutdown 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 msgr2=0x56039f646e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd50381f740 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f646e60 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec03e040 tx=0x7fd4ec046040 comp rx=0 tx=0).stop 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 shutdown_connections 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd50381f740 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f646e60 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.258+0000 7fd50381f740 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f252f60 0x56039f64cf00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.259+0000 7fd50381f740 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f2736b0 0x56039f63ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.259+0000 7fd50381f740 1 --2- 172.21.15.78:0/3602697856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f63f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.259+0000 7fd4ddffb640 0 client.17466 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.259+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 >> 172.21.15.78:0/3602697856 conn(0x56039f2884e0 msgr2=0x56039f634180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.259+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 shutdown_connections 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.259+0000 7fd50381f740 1 -- 172.21.15.78:0/3602697856 wait complete. 2024-04-25T14:06:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.259+0000 7fd50381f740 20 client.17466 trim_cache size 0 max 16384 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 Processor -- start 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 -- start start 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f62c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f2736b0 0x56039f63a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f294390 0x56039f63ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ada70 con 0x56039f2736b0 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81ab340 con 0x56039f26b4d0 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd50381f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81a9190 con 0x56039f294390 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f2736b0 0x56039f63a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd4f659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f2736b0 0x56039f63a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44254/0 (socket says 172.21.15.78:44254) 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd4f659c640 1 -- 172.21.15.78:0/4252783163 learned_addr learned my addr 172.21.15.78:0/4252783163 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.276+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f62c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f294390 0x56039f63ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4f659c640 1 -- 172.21.15.78:0/4252783163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f294390 msgr2=0x56039f63ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4f659c640 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f294390 0x56039f63ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4f659c640 1 -- 172.21.15.78:0/4252783163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 msgr2=0x56039f62c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4f659c640 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f62c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4f659c640 1 -- 172.21.15.78:0/4252783163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039f26c0f0 con 0x56039f2736b0 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4f659c640 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f2736b0 0x56039f63a720 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8000f80 tx=0x7fd4e8009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b49ac9ff6a420ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4dffff640 1 -- 172.21.15.78:0/4252783163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e801f020 con 0x56039f2736b0 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4dffff640 1 -- 172.21.15.78:0/4252783163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e8011dc0 con 0x56039f2736b0 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd4dffff640 1 -- 172.21.15.78:0/4252783163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80103d0 con 0x56039f2736b0 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd50381f740 1 -- 172.21.15.78:0/4252783163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f2736b0 msgr2=0x56039f63a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.277+0000 7fd50381f740 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f2736b0 0x56039f63a720 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8000f80 tx=0x7fd4e8009720 comp rx=0 tx=0).stop 2024-04-25T14:06:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- 172.21.15.78:0/4252783163 shutdown_connections 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f294390 0x56039f63ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f2736b0 0x56039f63a720 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 --2- 172.21.15.78:0/4252783163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f26b4d0 0x56039f62c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- 172.21.15.78:0/4252783163 >> 172.21.15.78:0/4252783163 conn(0x56039f2884e0 msgr2=0x56039f63f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- 172.21.15.78:0/4252783163 shutdown_connections 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- 172.21.15.78:0/4252783163 wait complete. 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- start start 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 Processor -- start 2024-04-25T14:06:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f26b4d0 0x56039f643800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f2736b0 0x56039f643d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f644c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81a9800 con 0x56039f294390 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81aac70 con 0x56039f2736b0 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81a9ed0 con 0x56039f26b4d0 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd50381f740 10 client.0 _ll_register_callbacks cb 0x7fd4ec03e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f644c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd4f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f644c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44266/0 (socket says 172.21.15.78:44266) 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/492359193 learned_addr learned my addr 172.21.15.78:0/492359193 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f26b4d0 0x56039f643800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.278+0000 7fd4f659c640 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f2736b0 0x56039f643d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/492359193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f26b4d0 msgr2=0x56039f643800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f26b4d0 0x56039f643800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f2736b0 msgr2=0x56039f643d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f2736b0 0x56039f643d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4f5d9b640 1 -- 172.21.15.78:0/492359193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039f55b4b0 con 0x56039f294390 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4f5d9b640 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f644c70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec031cd0 tx=0x7fd4ec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b234b6de7cf8279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4ddffb640 1 -- 172.21.15.78:0/492359193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec035040 con 0x56039f294390 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4ddffb640 1 -- 172.21.15.78:0/492359193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd4ec03ba90 con 0x56039f294390 2024-04-25T14:06:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e8011dc0 con 0x56039f294390 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4ddffb640 1 -- 172.21.15.78:0/492359193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd4ec03bc90 con 0x56039f294390 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd50381f740 10 client.17472 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56039f646320 con 0x56039f294390 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.279+0000 7fd4ddffb640 1 -- 172.21.15.78:0/492359193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4e8011dc0 con 0x56039f294390 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd4ddffb640 1 -- 172.21.15.78:0/492359193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd4ec070050 con 0x56039f294390 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd50381f740 10 client.17472 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd50381f740 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd50381f740 20 client.17472 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd50381f740 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f2283a0 0x56039f6bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd50381f740 4 client.17472 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:06:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56039f5a2800 con 0x56039f2283a0 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd4f659c640 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f2283a0 0x56039f6bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd4ddffb640 10 client.17472 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd4f659c640 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f2283a0 0x56039f6bfa20 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae6ae0051bbfeac server_cookie=5ce6b5fde9d2fb50 in_seq=0 out_seq=0 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd4ddffb640 1 -- 172.21.15.78:0/492359193 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56039f5a2800 con 0x56039f2283a0 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd4ddffb640 10 client.17472 handle_command_reply: tid=0 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.280+0000 7fd50381f740 1 client.17472 shutdown 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 msgr2=0x56039f644c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f644c70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec031cd0 tx=0x7fd4ec02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 shutdown_connections 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56039f294390 0x56039f644c70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56039f2736b0 0x56039f643d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56039f2283a0 0x56039f6bfa20 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 --2- 172.21.15.78:0/492359193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56039f26b4d0 0x56039f643800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4ddffb640 0 client.17472 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 >> 172.21.15.78:0/492359193 conn(0x56039f2884e0 msgr2=0x56039f630c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 shutdown_connections 2024-04-25T14:06:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 1 -- 172.21.15.78:0/492359193 wait complete. 2024-04-25T14:06:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd50381f740 20 client.17472 trim_cache size 0 max 16384 2024-04-25T14:06:18.290 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:06:18.291 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:06:18.291 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:06:18.291 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:06:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd4e00afaf0 msgr2=0x7fd4e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd4e00afaf0 0x7fd4e00b2000 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd4f001d000 tx=0x7fd4f0075580 comp rx=0 tx=0).stop 2024-04-25T14:06:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f81097b0 msgr2=0x7fd4f81a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f81097b0 0x7fd4f81a16e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000d750 tx=0x7fd4f000dc10 comp rx=0 tx=0).stop 2024-04-25T14:06:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 shutdown_connections 2024-04-25T14:06:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd4e00afaf0 0x7fd4e00b2000 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4f81097b0 0x7fd4f81a16e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4f8106c40 0x7fd4f819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 --2- 172.21.15.78:0/3585211756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4f8101eb0 0x7fd4f819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 >> 172.21.15.78:0/3585211756 conn(0x7fd4f80fd380 msgr2=0x7fd4f8107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 shutdown_connections 2024-04-25T14:06:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.281+0000 7fd4fc97a640 1 -- 172.21.15.78:0/3585211756 wait complete. 2024-04-25T14:06:18.293 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:06:18.293 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3489607094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b01098d0 msgr2=0x7f21b010bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3489607094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b01098d0 0x7f21b010bcf0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009ca0 tx=0x7f21a802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3489607094 shutdown_connections 2024-04-25T14:06:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3489607094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b01098d0 0x7f21b010bcf0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3489607094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21b0106f70 0x7f21b0109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3489607094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b0104610 0x7f21b0106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3489607094 >> 172.21.15.78:0/3489607094 conn(0x7f21b00fdc10 msgr2=0x7f21b01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3489607094 shutdown_connections 2024-04-25T14:06:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.376+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3489607094 wait complete. 2024-04-25T14:06:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 Processor -- start 2024-04-25T14:06:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 -- start start 2024-04-25T14:06:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0104610 0x7f21b0103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21b0106f70 0x7f21b0102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01098d0 0x7f21b0102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21b010f630 con 0x7f21b0104610 2024-04-25T14:06:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21b010f4b0 con 0x7f21b01098d0 2024-04-25T14:06:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b7b6b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21b010f7b0 con 0x7f21b0106f70 2024-04-25T14:06:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b5907640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0104610 0x7f21b0103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b5907640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0104610 0x7f21b0103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44280/0 (socket says 172.21.15.78:44280) 2024-04-25T14:06:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b5907640 1 -- 172.21.15.78:0/3282814735 learned_addr learned my addr 172.21.15.78:0/3282814735 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b5106640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21b0106f70 0x7f21b0102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b6108640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01098d0 0x7f21b0102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b5907640 1 -- 172.21.15.78:0/3282814735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21b0106f70 msgr2=0x7f21b0102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b5907640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21b0106f70 0x7f21b0102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.377+0000 7f21b5907640 1 -- 172.21.15.78:0/3282814735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01098d0 msgr2=0x7f21b0102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21b5907640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01098d0 0x7f21b0102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21b5907640 1 -- 172.21.15.78:0/3282814735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b0102d70 con 0x7f21b0104610 2024-04-25T14:06:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21b5106640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21b0106f70 0x7f21b0102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21b5907640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0104610 0x7f21b0103a60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00e9c0 tx=0x7f21ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21a6ffd640 1 -- 172.21.15.78:0/3282814735 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00cd80 con 0x7f21b0104610 2024-04-25T14:06:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21a6ffd640 1 -- 172.21.15.78:0/3282814735 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ac0054d0 con 0x7f21b0104610 2024-04-25T14:06:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b01a42c0 con 0x7f21b0104610 2024-04-25T14:06:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21a6ffd640 1 -- 172.21.15.78:0/3282814735 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac010430 con 0x7f21b0104610 2024-04-25T14:06:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.378+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b01a4780 con 0x7f21b0104610 2024-04-25T14:06:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 Processor -- start 2024-04-25T14:06:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 -- start start 2024-04-25T14:06:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a37970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a37ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a3edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae1399d460 con 0x55ae136764d0 2024-04-25T14:06:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae13957590 con 0x55ae1366e2f0 2024-04-25T14:06:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21bea18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ae13999420 con 0x55ae136971b0 2024-04-25T14:06:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21b5106640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a3edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21b5106640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a3edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44292/0 (socket says 172.21.15.78:44292) 2024-04-25T14:06:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21b5106640 1 -- 172.21.15.78:0/539119703 learned_addr learned my addr 172.21.15.78:0/539119703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.381+0000 7f21b6108640 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a37970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f21b5106640 1 -- 172.21.15.78:0/539119703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 msgr2=0x55ae13a37970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f21b5106640 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a37970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f21b5106640 1 -- 172.21.15.78:0/539119703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 msgr2=0x55ae13a37ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T14:06:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f21b5106640 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a37ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f21b5106640 1 -- 172.21.15.78:0/539119703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae139640d0 con 0x55ae136764d0 2024-04-25T14:06:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f21a6ffd640 1 -- 172.21.15.78:0/3282814735 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f21ac004fd0 con 0x7f21b0104610 2024-04-25T14:06:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f21b5106640 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a3edf0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f21a000bd30 tx=0x7f21a000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c05bea7aa6456b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f219bfff640 1 -- 172.21.15.78:0/539119703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0015070 con 0x55ae136764d0 2024-04-25T14:06:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f219bfff640 1 -- 172.21.15.78:0/539119703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a000be20 con 0x55ae136764d0 2024-04-25T14:06:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.382+0000 7f219bfff640 1 -- 172.21.15.78:0/539119703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a00116a0 con 0x55ae136764d0 2024-04-25T14:06:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21a6ffd640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f219c0afae0 0x7f219c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21b6108640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f219c0afae0 0x7f219c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21a6ffd640 1 -- 172.21.15.78:0/3282814735 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f21ac0df170 con 0x7f21b0104610 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 -- 172.21.15.78:0/539119703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 msgr2=0x55ae13a3edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a3edf0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f21a000bd30 tx=0x7f21a000f730 comp rx=0 tx=0).stop 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 -- 172.21.15.78:0/539119703 shutdown_connections 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a3edf0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a37ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 --2- 172.21.15.78:0/539119703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a37970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 -- 172.21.15.78:0/539119703 >> 172.21.15.78:0/539119703 conn(0x55ae1368b300 msgr2=0x55ae13a33300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21b6108640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f219c0afae0 0x7f219c0b1ff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55ae13a38b40 tx=0x7f21a8031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 -- 172.21.15.78:0/539119703 shutdown_connections 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.383+0000 7f21bea18740 1 -- 172.21.15.78:0/539119703 wait complete. 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 -- start start 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 Processor -- start 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae1366e2f0 0x55ae13a416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a41c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136971b0 0x55ae13a48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ae13a3f840 con 0x55ae136764d0 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae13924b60 con 0x55ae136971b0 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ae13a3f9c0 con 0x55ae1366e2f0 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21bea18740 10 client.0 _ll_register_callbacks cb 0x55ae139492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21b5106640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a41c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21b5106640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a41c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44306/0 (socket says 172.21.15.78:44306) 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21b5106640 1 -- 172.21.15.78:0/662565517 learned_addr learned my addr 172.21.15.78:0/662565517 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21b6108640 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136971b0 0x55ae13a48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.384+0000 7f21b5907640 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae1366e2f0 0x55ae13a416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21b5106640 1 -- 172.21.15.78:0/662565517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae1366e2f0 msgr2=0x55ae13a416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21b5106640 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae1366e2f0 0x55ae13a416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21b5106640 1 -- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136971b0 msgr2=0x55ae13a48b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21b5106640 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136971b0 0x55ae13a48b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21b5106640 1 -- 172.21.15.78:0/662565517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae136af210 con 0x55ae136764d0 2024-04-25T14:06:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21b5106640 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a41c20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f21a000b4b0 tx=0x7f21a0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cab69c080e08e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21997fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21997fa640 1 -- 172.21.15.78:0/662565517 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0028060 con 0x55ae136764d0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a000be20 con 0x55ae136764d0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21997fa640 1 -- 172.21.15.78:0/662565517 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f21a0004090 con 0x55ae136764d0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21997fa640 1 -- 172.21.15.78:0/662565517 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f21a0015070 con 0x55ae136764d0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21bea18740 10 client.17484 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ae1377b620 con 0x55ae136764d0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.385+0000 7f21997fa640 1 -- 172.21.15.78:0/662565517 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a000be20 con 0x55ae136764d0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21997fa640 1 -- 172.21.15.78:0/662565517 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f21a0011b40 con 0x55ae136764d0 2024-04-25T14:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21bea18740 10 client.17484 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21bea18740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21bea18740 20 client.17484 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21bea18740 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae13655d80 0x55ae13a4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21bea18740 4 client.17484 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae139ab070 con 0x55ae13655d80 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.386+0000 7f21b5907640 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae13655d80 0x55ae13a4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.387+0000 7f21997fa640 10 client.17484 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.387+0000 7f21b5907640 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae13655d80 0x55ae13a4f090 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1875950c3d42d5db server_cookie=e20a9dc545d123f1 in_seq=0 out_seq=0 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.387+0000 7f21997fa640 1 -- 172.21.15.78:0/662565517 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae139ab070 con 0x55ae13655d80 2024-04-25T14:06:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.387+0000 7f21997fa640 10 client.17484 handle_command_reply: tid=0 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.387+0000 7f21bea18740 1 client.17484 shutdown 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 msgr2=0x55ae13a41c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a41c20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f21a000b4b0 tx=0x7f21a0003820 comp rx=0 tx=0).stop 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 shutdown_connections 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136971b0 0x55ae13a48b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a41c20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae13655d80 0x55ae13a4f090 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 --2- 172.21.15.78:0/662565517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae1366e2f0 0x55ae13a416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21997fa640 0 client.17484 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 >> 172.21.15.78:0/662565517 conn(0x55ae1368b300 msgr2=0x55ae13a32250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 shutdown_connections 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 1 -- 172.21.15.78:0/662565517 wait complete. 2024-04-25T14:06:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.388+0000 7f21bea18740 20 client.17484 trim_cache size 0 max 16384 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 Processor -- start 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 -- start start 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae1366e2f0 0x55ae13a333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136764d0 0x55ae13a48aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01afdd0 con 0x55ae1366e2f0 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01ad6a0 con 0x55ae136764d0 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21bea18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01ab4f0 con 0x55ae136971b0 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5907640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae1366e2f0 0x55ae13a333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5907640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae1366e2f0 0x55ae13a333d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44308/0 (socket says 172.21.15.78:44308) 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5907640 1 -- 172.21.15.78:0/3085704482 learned_addr learned my addr 172.21.15.78:0/3085704482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b6108640 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5106640 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136764d0 0x55ae13a48aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5907640 1 -- 172.21.15.78:0/3085704482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 msgr2=0x55ae13a36100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5907640 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a36100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5907640 1 -- 172.21.15.78:0/3085704482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136764d0 msgr2=0x55ae13a48aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.403+0000 7f21b5907640 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136764d0 0x55ae13a48aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.404+0000 7f21b5907640 1 -- 172.21.15.78:0/3085704482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae13a3c500 con 0x55ae1366e2f0 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.404+0000 7f21b5907640 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae1366e2f0 0x55ae13a333d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0a75d0 tx=0x7f21ac009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c728c12f6cae762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.404+0000 7f219b7fe640 1 -- 172.21.15.78:0/3085704482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0a7150 con 0x55ae1366e2f0 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.404+0000 7f219b7fe640 1 -- 172.21.15.78:0/3085704482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ac0a7d40 con 0x55ae1366e2f0 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.404+0000 7f219b7fe640 1 -- 172.21.15.78:0/3085704482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0a38d0 con 0x55ae1366e2f0 2024-04-25T14:06:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 -- 172.21.15.78:0/3085704482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae1366e2f0 msgr2=0x55ae13a333d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae1366e2f0 0x55ae13a333d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0a75d0 tx=0x7f21ac009700 comp rx=0 tx=0).stop 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 -- 172.21.15.78:0/3085704482 shutdown_connections 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a36100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae136764d0 0x55ae13a48aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 --2- 172.21.15.78:0/3085704482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae1366e2f0 0x55ae13a333d0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 -- 172.21.15.78:0/3085704482 >> 172.21.15.78:0/3085704482 conn(0x55ae1368b300 msgr2=0x55ae13a32cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 -- 172.21.15.78:0/3085704482 shutdown_connections 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 -- 172.21.15.78:0/3085704482 wait complete. 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.405+0000 7f21bea18740 1 -- start start 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 1 Processor -- start 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a46170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01abb60 con 0x55ae136764d0 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01acfd0 con 0x55ae1366e2f0 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01ac230 con 0x55ae136971b0 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21bea18740 10 client.0 _ll_register_callbacks cb 0x7f21ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21b5106640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21b5106640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44316/0 (socket says 172.21.15.78:44316) 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21b5106640 1 -- 172.21.15.78:0/114787609 learned_addr learned my addr 172.21.15.78:0/114787609 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21b6108640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.406+0000 7f21b5907640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a46170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21b5106640 1 -- 172.21.15.78:0/114787609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 msgr2=0x55ae13a475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21b5106640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21b5106640 1 -- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 msgr2=0x55ae13a46170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21b5106640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a46170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21b5106640 1 -- 172.21.15.78:0/114787609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae13a47ba0 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21b5907640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a46170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21b5106640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a466b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f21a0003890 tx=0x7f21a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da2bb9df70cc7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21997fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21997fa640 1 -- 172.21.15.78:0/114787609 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0028040 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21997fa640 1 -- 172.21.15.78:0/114787609 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f21a0019a60 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ac0a7d40 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21997fa640 1 -- 172.21.15.78:0/114787609 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f21a003e050 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21bea18740 10 client.17490 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ae13a48e30 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.407+0000 7f21997fa640 1 -- 172.21.15.78:0/114787609 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21ac0a7d40 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21997fa640 1 -- 172.21.15.78:0/114787609 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f21a0005030 con 0x55ae136764d0 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21bea18740 10 client.17490 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21bea18740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21bea18740 20 client.17490 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21bea18740 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae1362b1c0 0x55ae13ab9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21bea18740 4 client.17490 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae139a4ee0 con 0x55ae1362b1c0 2024-04-25T14:06:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21b5907640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae1362b1c0 0x55ae13ab9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.408+0000 7f21997fa640 10 client.17490 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.409+0000 7f21b5907640 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae1362b1c0 0x55ae13ab9b50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e717ddc58365eac5 server_cookie=a391a165cdb24ccf in_seq=0 out_seq=0 2024-04-25T14:06:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 -- 172.21.15.78:0/611099149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7641029c0 msgr2=0x7fc7641036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 --2- 172.21.15.78:0/611099149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7641029c0 0x7fc7641036c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc754009900 tx=0x7fc75402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 -- 172.21.15.78:0/611099149 shutdown_connections 2024-04-25T14:06:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 --2- 172.21.15.78:0/611099149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc764108900 0x7fc76410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 --2- 172.21.15.78:0/611099149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc764103c00 0x7fc764104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 --2- 172.21.15.78:0/611099149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7641029c0 0x7fc7641036c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 -- 172.21.15.78:0/611099149 >> 172.21.15.78:0/611099149 conn(0x7fc7640fdbd0 msgr2=0x7fc764100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 -- 172.21.15.78:0/611099149 shutdown_connections 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.410+0000 7fc76891c640 1 -- 172.21.15.78:0/611099149 wait complete. 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 Processor -- start 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 -- start start 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7641029c0 0x7fc76406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc764103c00 0x7fc76419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc764108900 0x7fc7641a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc76410e4c0 con 0x7fc764103c00 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc76410e340 con 0x7fc764108900 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76891c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc76410e640 con 0x7fc7641029c0 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc764103c00 0x7fc76419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc764103c00 0x7fc76419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44336/0 (socket says 172.21.15.78:44336) 2024-04-25T14:06:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 -- 172.21.15.78:0/3854263079 learned_addr learned my addr 172.21.15.78:0/3854263079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc76259c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7641029c0 0x7fc76406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc762d9d640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc764108900 0x7fc7641a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 -- 172.21.15.78:0/3854263079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7641029c0 msgr2=0x7fc76406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7641029c0 0x7fc76406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 -- 172.21.15.78:0/3854263079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc764108900 msgr2=0x7fc7641a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc764108900 0x7fc7641a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 -- 172.21.15.78:0/3854263079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7641a1d10 con 0x7fc764103c00 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc762d9d640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc764108900 0x7fc7641a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.411+0000 7fc761d9b640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc764103c00 0x7fc76419efa0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc75800e9f0 tx=0x7fc75800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.412+0000 7fc7537fe640 1 -- 172.21.15.78:0/3854263079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75800cd80 con 0x7fc764103c00 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.412+0000 7fc7537fe640 1 -- 172.21.15.78:0/3854263079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7580054d0 con 0x7fc764103c00 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.412+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7641a1fa0 con 0x7fc764103c00 2024-04-25T14:06:18.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.412+0000 7fc7537fe640 1 -- 172.21.15.78:0/3854263079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758010690 con 0x7fc764103c00 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.412+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7641a2490 con 0x7fc764103c00 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.413+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc764107710 con 0x7fc764103c00 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.415+0000 7fc7537fe640 1 -- 172.21.15.78:0/3854263079 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc758004e30 con 0x7fc764103c00 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.416+0000 7fc7537fe640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc74c0afb70 0x7fc74c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.416+0000 7fc7537fe640 1 -- 172.21.15.78:0/3854263079 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc7580df260 con 0x7fc764103c00 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.416+0000 7fc76259c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc74c0afb70 0x7fc74c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.417+0000 7fc76259c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc74c0afb70 0x7fc74c0b2080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc754006010 tx=0x7fc754005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.420+0000 7fc7537fe640 1 -- 172.21.15.78:0/3854263079 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc758053ff0 con 0x7fc764103c00 2024-04-25T14:06:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21997fa640 1 -- 172.21.15.78:0/114787609 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae139a4ee0 con 0x55ae1362b1c0 2024-04-25T14:06:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21997fa640 10 client.17490 handle_command_reply: tid=0 2024-04-25T14:06:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 client.17490 shutdown 2024-04-25T14:06:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 msgr2=0x55ae13a466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a466b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f21a0003890 tx=0x7f21a000b330 comp rx=0 tx=0).stop 2024-04-25T14:06:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 shutdown_connections 2024-04-25T14:06:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55ae1362b1c0 0x55ae13ab9b50 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ae136971b0 0x55ae13a475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ae136764d0 0x55ae13a466b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21997fa640 0 client.17490 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 --2- 172.21.15.78:0/114787609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae1366e2f0 0x55ae13a46170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 >> 172.21.15.78:0/114787609 conn(0x55ae1368b300 msgr2=0x55ae13a31820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 shutdown_connections 2024-04-25T14:06:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 1 -- 172.21.15.78:0/114787609 wait complete. 2024-04-25T14:06:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.423+0000 7f21bea18740 20 client.17490 trim_cache size 0 max 16384 2024-04-25T14:06:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:06:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:06:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:06:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:06:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f219c0afae0 msgr2=0x7f219c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f219c0afae0 0x7f219c0b1ff0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55ae13a38b40 tx=0x7f21a8031af0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0104610 msgr2=0x7f21b0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0104610 0x7f21b0103a60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00e9c0 tx=0x7f21ac00ee80 comp rx=0 tx=0).stop 2024-04-25T14:06:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 shutdown_connections 2024-04-25T14:06:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21b01098d0 0x7f21b0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21b0106f70 0x7f21b0102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f219c0afae0 0x7f219c0b1ff0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 --2- 172.21.15.78:0/3282814735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21b0104610 0x7f21b0103a60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 >> 172.21.15.78:0/3282814735 conn(0x7f21b00fdc10 msgr2=0x7f21b0105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 shutdown_connections 2024-04-25T14:06:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:18.424+0000 7f21b7b6b640 1 -- 172.21.15.78:0/3282814735 wait complete. 2024-04-25T14:06:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.637+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc76402db50 con 0x7fc764103c00 2024-04-25T14:06:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.638+0000 7fc7537fe640 1 -- 172.21.15.78:0/3854263079 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fc7580a7e30 con 0x7fc764103c00 2024-04-25T14:06:18.638 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:06:18.638 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:06:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc74c0afb70 msgr2=0x7fc74c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc74c0afb70 0x7fc74c0b2080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc754006010 tx=0x7fc754005d50 comp rx=0 tx=0).stop 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc764103c00 msgr2=0x7fc76419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc764103c00 0x7fc76419efa0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc75800e9f0 tx=0x7fc75800edf0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 shutdown_connections 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc764108900 0x7fc7641a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc74c0afb70 0x7fc74c0b2080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc764103c00 0x7fc76419efa0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 --2- 172.21.15.78:0/3854263079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc7641029c0 0x7fc76406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 >> 172.21.15.78:0/3854263079 conn(0x7fc7640fdbd0 msgr2=0x7fc7641048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 shutdown_connections 2024-04-25T14:06:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.640+0000 7fc76891c640 1 -- 172.21.15.78:0/3854263079 wait complete. 2024-04-25T14:06:18.651 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:06:18.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 -- 172.21.15.78:0/3782745655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144109a40 msgr2=0x7f314410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 --2- 172.21.15.78:0/3782745655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144109a40 0x7f314410be60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f313c009ca0 tx=0x7f313c02a8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 -- 172.21.15.78:0/3782745655 shutdown_connections 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 --2- 172.21.15.78:0/3782745655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144109a40 0x7f314410be60 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 --2- 172.21.15.78:0/3782745655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144104d40 0x7f3144105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 --2- 172.21.15.78:0/3782745655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144103b00 0x7f3144104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 -- 172.21.15.78:0/3782745655 >> 172.21.15.78:0/3782745655 conn(0x7f31440fecf0 msgr2=0x7f31441011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 -- 172.21.15.78:0/3782745655 shutdown_connections 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 -- 172.21.15.78:0/3782745655 wait complete. 2024-04-25T14:06:18.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 Processor -- start 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.761+0000 7f314a71f640 1 -- start start 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f314a71f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144103b00 0x7f314419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f314a71f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144104d40 0x7f314419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f314a71f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144109a40 0x7f31441a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f314a71f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f314410f700 con 0x7f3144103b00 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f314a71f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f314410f580 con 0x7f3144104d40 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f314a71f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f314410f880 con 0x7f3144109a40 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144103b00 0x7f314419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144103b00 0x7f314419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44348/0 (socket says 172.21.15.78:44348) 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 -- 172.21.15.78:0/298284946 learned_addr learned my addr 172.21.15.78:0/298284946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3148cbc640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144109a40 0x7f31441a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 -- 172.21.15.78:0/298284946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144109a40 msgr2=0x7f31441a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f31437fe640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144104d40 0x7f314419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144109a40 0x7f31441a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 -- 172.21.15.78:0/298284946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144104d40 msgr2=0x7f314419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144104d40 0x7f314419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 -- 172.21.15.78:0/298284946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31441a1e50 con 0x7f3144103b00 2024-04-25T14:06:18.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3148cbc640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144109a40 0x7f31441a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f3143fff640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144103b00 0x7f314419a310 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3134006fd0 tx=0x7f313400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.762+0000 7f31417fa640 1 -- 172.21.15.78:0/298284946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134011830 con 0x7f3144103b00 2024-04-25T14:06:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.763+0000 7f31417fa640 1 -- 172.21.15.78:0/298284946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3134016070 con 0x7f3144103b00 2024-04-25T14:06:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.763+0000 7f31417fa640 1 -- 172.21.15.78:0/298284946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134010e00 con 0x7f3144103b00 2024-04-25T14:06:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.763+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31441a2140 con 0x7f3144103b00 2024-04-25T14:06:18.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.763+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31441a26c0 con 0x7f3144103b00 2024-04-25T14:06:18.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31417fa640 1 -- 172.21.15.78:0/298284946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f313401b460 con 0x7f3144103b00 2024-04-25T14:06:18.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 Processor -- start 2024-04-25T14:06:18.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 -- start start 2024-04-25T14:06:18.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac3282bd80 con 0x55ac325046b0 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac327e5a20 con 0x55ac32525390 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31515d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ac32827d40 con 0x55ac324fc4d0 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44350/0 (socket says 172.21.15.78:44350) 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f31437fe640 1 -- 172.21.15.78:0/2023067099 learned_addr learned my addr 172.21.15.78:0/2023067099 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.765+0000 7f3148cbc640 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f3143fff640 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31417fa640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f312c0afaf0 0x7f312c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31437fe640 1 -- 172.21.15.78:0/2023067099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 msgr2=0x55ac328c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31437fe640 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31437fe640 1 -- 172.21.15.78:0/2023067099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 msgr2=0x55ac328c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31437fe640 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31437fe640 1 -- 172.21.15.78:0/2023067099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac327f0c30 con 0x55ac325046b0 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f3148cbc640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f312c0afaf0 0x7f312c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31437fe640 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328c8a20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f313800a8e0 tx=0x7f313800ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c9a3e44327ba11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f31417fa640 1 -- 172.21.15.78:0/298284946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f31340de320 con 0x7f3144103b00 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f312a7fc640 1 -- 172.21.15.78:0/2023067099 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138015070 con 0x55ac325046b0 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f312a7fc640 1 -- 172.21.15.78:0/2023067099 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31380113f0 con 0x55ac325046b0 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.766+0000 7f312a7fc640 1 -- 172.21.15.78:0/2023067099 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31380185a0 con 0x55ac325046b0 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 -- 172.21.15.78:0/2023067099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 msgr2=0x55ac328c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f3148cbc640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f312c0afaf0 0x7f312c0b2000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f313c031870 tx=0x7f313c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328c8a20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f313800a8e0 tx=0x7f313800ada0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 -- 172.21.15.78:0/2023067099 shutdown_connections 2024-04-25T14:06:18.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328c8a20 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 --2- 172.21.15.78:0/2023067099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 -- 172.21.15.78:0/2023067099 >> 172.21.15.78:0/2023067099 conn(0x55ac325194e0 msgr2=0x55ac328c3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 -- 172.21.15.78:0/2023067099 shutdown_connections 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 -- 172.21.15.78:0/2023067099 wait complete. 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.767+0000 7f31515d4740 1 -- start start 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 1 Processor -- start 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac328c9530 con 0x55ac324fc4d0 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac327b2dc0 con 0x55ac32525390 2024-04-25T14:06:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ac328c96b0 con 0x55ac325046b0 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31515d4740 10 client.0 _ll_register_callbacks cb 0x55ac327ce600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44366/0 (socket says 172.21.15.78:44366) 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 -- 172.21.15.78:0/4021869503 learned_addr learned my addr 172.21.15.78:0/4021869503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31437fe640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3148cbc640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 -- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 msgr2=0x55ac328d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 -- 172.21.15.78:0/4021869503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 msgr2=0x55ac328d89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328d89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3143fff640 1 -- 172.21.15.78:0/4021869503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3253d3f0 con 0x55ac324fc4d0 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f31437fe640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.768+0000 7f3148cbc640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f3143fff640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d1530 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f31340a3a60 tx=0x7f31340047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa1d814245e7497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31237fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31237fe640 1 -- 172.21.15.78:0/4021869503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340a6300 con 0x55ac324fc4d0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31237fe640 1 -- 172.21.15.78:0/4021869503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f31340c90f0 con 0x55ac324fc4d0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31380113f0 con 0x55ac324fc4d0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31237fe640 1 -- 172.21.15.78:0/4021869503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3134021250 con 0x55ac324fc4d0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 10 client.17508 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ac326077d0 con 0x55ac324fc4d0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31237fe640 1 -- 172.21.15.78:0/4021869503 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31380113f0 con 0x55ac324fc4d0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31237fe640 1 -- 172.21.15.78:0/4021869503 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f31340e1c50 con 0x55ac324fc4d0 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 10 client.17508 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 10 client.17508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:06:18.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 20 client.17508 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac324e3f60 0x55ac328df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 4 client.17508 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.769+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac32839990 con 0x55ac324e3f60 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.770+0000 7f31437fe640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac324e3f60 0x55ac328df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.770+0000 7f31237fe640 10 client.17508 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.770+0000 7f31437fe640 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac324e3f60 0x55ac328df210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e4ffaba906e46ee server_cookie=e1d6cb523225b73 in_seq=0 out_seq=0 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31237fe640 1 -- 172.21.15.78:0/4021869503 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac32839990 con 0x55ac324e3f60 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31237fe640 10 client.17508 handle_command_reply: tid=0 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 client.17508 shutdown 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 msgr2=0x55ac328d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d1530 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f31340a3a60 tx=0x7f31340047d0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 shutdown_connections 2024-04-25T14:06:18.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac324e3f60 0x55ac328df210 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac32525390 0x55ac328d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31237fe640 0 client.17508 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 --2- 172.21.15.78:0/4021869503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d1530 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 >> 172.21.15.78:0/4021869503 conn(0x55ac325194e0 msgr2=0x55ac328c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 shutdown_connections 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 1 -- 172.21.15.78:0/4021869503 wait complete. 2024-04-25T14:06:18.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.771+0000 7f31515d4740 20 client.17508 trim_cache size 0 max 16384 2024-04-25T14:06:18.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.784+0000 7f31515d4740 1 Processor -- start 2024-04-25T14:06:18.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31515d4740 1 -- start start 2024-04-25T14:06:18.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31515d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31515d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31515d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31515d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31441add70 con 0x55ac325046b0 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31515d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31441ab640 con 0x55ac324e3f60 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31515d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31441a9490 con 0x55ac324fc4d0 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44378/0 (socket says 172.21.15.78:44378) 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 -- 172.21.15.78:0/2902229446 learned_addr learned my addr 172.21.15.78:0/2902229446 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3143fff640 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f31437fe640 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 -- 172.21.15.78:0/2902229446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 msgr2=0x55ac328e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 -- 172.21.15.78:0/2902229446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 msgr2=0x55ac328e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 -- 172.21.15.78:0/2902229446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac328cf640 con 0x55ac325046b0 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3143fff640 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.785+0000 7f3148cbc640 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328cfcb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f313c031d70 tx=0x7f313c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d730e2623b668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f3129ffb640 1 -- 172.21.15.78:0/2902229446 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0047c0 con 0x55ac325046b0 2024-04-25T14:06:18.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f3129ffb640 1 -- 172.21.15.78:0/2902229446 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c04e030 con 0x55ac325046b0 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f3129ffb640 1 -- 172.21.15.78:0/2902229446 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c04a690 con 0x55ac325046b0 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 -- 172.21.15.78:0/2902229446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 msgr2=0x55ac328cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328cfcb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f313c031d70 tx=0x7f313c03f000 comp rx=0 tx=0).stop 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 -- 172.21.15.78:0/2902229446 shutdown_connections 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac325046b0 0x55ac328cfcb0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac324fc4d0 0x55ac328e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 --2- 172.21.15.78:0/2902229446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 -- 172.21.15.78:0/2902229446 >> 172.21.15.78:0/2902229446 conn(0x55ac325194e0 msgr2=0x55ac328cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 -- 172.21.15.78:0/2902229446 shutdown_connections 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.786+0000 7f31515d4740 1 -- 172.21.15.78:0/2902229446 wait complete. 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 -- start start 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 Processor -- start 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328d6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31441a9b00 con 0x55ac324fc4d0 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31441aaf70 con 0x55ac324e3f60 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31441aa1d0 con 0x55ac325046b0 2024-04-25T14:06:18.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31515d4740 10 client.0 _ll_register_callbacks cb 0x7f31340e1910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44388/0 (socket says 172.21.15.78:44388) 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f31437fe640 1 -- 172.21.15.78:0/3615006627 learned_addr learned my addr 172.21.15.78:0/3615006627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f3143fff640 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328d6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.787+0000 7f3148cbc640 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31437fe640 1 -- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 msgr2=0x55ac328d72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31437fe640 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31437fe640 1 -- 172.21.15.78:0/3615006627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 msgr2=0x55ac328d6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31437fe640 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328d6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31437fe640 1 -- 172.21.15.78:0/3615006627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac32945570 con 0x55ac324fc4d0 2024-04-25T14:06:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31437fe640 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d6d80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x55ac328c3db0 tx=0x7f313800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19ece415b768af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31237fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31237fe640 1 -- 172.21.15.78:0/3615006627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313802b030 con 0x55ac324fc4d0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31237fe640 1 -- 172.21.15.78:0/3615006627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f313800c940 con 0x55ac324fc4d0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f313c04e030 con 0x55ac324fc4d0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31237fe640 1 -- 172.21.15.78:0/3615006627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3138015040 con 0x55ac324fc4d0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31515d4740 10 client.17514 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ac32946770 con 0x55ac324fc4d0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31237fe640 1 -- 172.21.15.78:0/3615006627 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c04e030 con 0x55ac324fc4d0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31237fe640 1 -- 172.21.15.78:0/3615006627 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f31380324c0 con 0x55ac324fc4d0 2024-04-25T14:06:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31515d4740 10 client.17514 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31515d4740 10 client.17514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.788+0000 7f31515d4740 20 client.17514 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f31515d4740 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac32525390 0x55ac32948ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f31515d4740 4 client.17514 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac32833800 con 0x55ac32525390 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f3143fff640 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac32525390 0x55ac32948ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f31237fe640 10 client.17514 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f3143fff640 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac32525390 0x55ac32948ee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc4563d76e28f11a server_cookie=cc63ea7c35e29dfc in_seq=0 out_seq=0 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f31237fe640 1 -- 172.21.15.78:0/3615006627 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac32833800 con 0x55ac32525390 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f31237fe640 10 client.17514 handle_command_reply: tid=0 2024-04-25T14:06:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.789+0000 7f31515d4740 1 client.17514 shutdown 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 msgr2=0x55ac328d6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d6d80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x55ac328c3db0 tx=0x7f313800a620 comp rx=0 tx=0).stop 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 shutdown_connections 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55ac32525390 0x55ac32948ee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac325046b0 0x55ac328d72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac324fc4d0 0x55ac328d6d80 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 --2- 172.21.15.78:0/3615006627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac324e3f60 0x55ac328d6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31237fe640 0 client.17514 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 >> 172.21.15.78:0/3615006627 conn(0x55ac325194e0 msgr2=0x55ac328cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 shutdown_connections 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 1 -- 172.21.15.78:0/3615006627 wait complete. 2024-04-25T14:06:18.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f31515d4740 20 client.17514 trim_cache size 0 max 16384 2024-04-25T14:06:18.800 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:06:18.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f312c0afaf0 msgr2=0x7f312c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f312c0afaf0 0x7f312c0b2000 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f313c031870 tx=0x7f313c0095c0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144103b00 msgr2=0x7f314419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144103b00 0x7f314419a310 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3134006fd0 tx=0x7f313400b8e0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 shutdown_connections 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144109a40 0x7f31441a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144104d40 0x7f314419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f312c0afaf0 0x7f312c0b2000 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 --2- 172.21.15.78:0/298284946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144103b00 0x7f314419a310 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 >> 172.21.15.78:0/298284946 conn(0x7f31440fecf0 msgr2=0x7f3144105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 shutdown_connections 2024-04-25T14:06:18.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.790+0000 7f314a71f640 1 -- 172.21.15.78:0/298284946 wait complete. 2024-04-25T14:06:18.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:06:18.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 -- 172.21.15.78:0/988080952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 msgr2=0x7fb488109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/988080952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb488109100 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb484009900 tx=0x7fb48402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 -- 172.21.15.78:0/988080952 shutdown_connections 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/988080952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488109790 0x7fb48810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/988080952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb488109100 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/988080952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488101e70 0x7fb488106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 -- 172.21.15.78:0/988080952 >> 172.21.15.78:0/988080952 conn(0x7fb4880fd380 msgr2=0x7fb4880ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 -- 172.21.15.78:0/988080952 shutdown_connections 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.913+0000 7fb48fb4d640 1 -- 172.21.15.78:0/988080952 wait complete. 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 Processor -- start 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 -- start start 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488101e70 0x7fb48819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb48819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488109790 0x7fb4881a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810f5e0 con 0x7fb488102370 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810f460 con 0x7fb488109790 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48fb4d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810f760 con 0x7fb488101e70 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb48819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb48819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44396/0 (socket says 172.21.15.78:44396) 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 -- 172.21.15.78:0/3820537949 learned_addr learned my addr 172.21.15.78:0/3820537949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488101e70 0x7fb48819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488109790 0x7fb4881a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 -- 172.21.15.78:0/3820537949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488101e70 msgr2=0x7fb48819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488101e70 0x7fb48819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 -- 172.21.15.78:0/3820537949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488109790 msgr2=0x7fb4881a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488109790 0x7fb4881a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48d0e8640 1 -- 172.21.15.78:0/3820537949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4881a1d50 con 0x7fb488102370 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.914+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488109790 0x7fb4881a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.915+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb48819a750 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.915+0000 7fb47effd640 1 -- 172.21.15.78:0/3820537949 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48403f030 con 0x7fb488102370 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.915+0000 7fb47effd640 1 -- 172.21.15.78:0/3820537949 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb484003aa0 con 0x7fb488102370 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.915+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4881a1fe0 con 0x7fb488102370 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.915+0000 7fb47effd640 1 -- 172.21.15.78:0/3820537949 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484035070 con 0x7fb488102370 2024-04-25T14:06:18.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.915+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4881a23f0 con 0x7fb488102370 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.917+0000 7fb47effd640 1 -- 172.21.15.78:0/3820537949 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb484042030 con 0x7fb488102370 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.917+0000 7fb496a04740 1 Processor -- start 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb496a04740 1 -- start start 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb496a04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65920b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb496a04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a65920bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb496a04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a65920ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb496a04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a65916fd80 con 0x55a658e69390 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb496a04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a659129a20 con 0x55a658e404d0 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb496a04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a65916bd40 con 0x55a658e486b0 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65920b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65920b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44406/0 (socket says 172.21.15.78:44406) 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 -- 172.21.15.78:0/3838582561 learned_addr learned my addr 172.21.15.78:0/3838582561 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a65920bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a65920ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb47effd640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4740afaf0 0x7fb4740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 -- 172.21.15.78:0/3838582561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 msgr2=0x55a65920ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a65920ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 -- 172.21.15.78:0/3838582561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 msgr2=0x55a65920bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a65920bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 -- 172.21.15.78:0/3838582561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a659134c30 con 0x55a658e69390 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a65920ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4740afaf0 0x7fb4740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.918+0000 7fb47effd640 1 -- 172.21.15.78:0/3820537949 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb4840ff250 con 0x7fb488102370 2024-04-25T14:06:18.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.919+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65920b810 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb48000f4b0 tx=0x7fb48000f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a4ea0a036f1fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.919+0000 7fb473fff640 1 -- 172.21.15.78:0/3838582561 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480018070 con 0x55a658e69390 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.919+0000 7fb473fff640 1 -- 172.21.15.78:0/3838582561 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48001cd90 con 0x55a658e69390 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.919+0000 7fb473fff640 1 -- 172.21.15.78:0/3838582561 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480025ce0 con 0x55a658e69390 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 -- 172.21.15.78:0/3838582561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 msgr2=0x55a65920b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4740afaf0 0x7fb4740b2000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb48001f000 tx=0x7fb48001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65920b810 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb48000f4b0 tx=0x7fb48000f970 comp rx=0 tx=0).stop 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 -- 172.21.15.78:0/3838582561 shutdown_connections 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a65920ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a65920bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 --2- 172.21.15.78:0/3838582561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65920b810 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 -- 172.21.15.78:0/3838582561 >> 172.21.15.78:0/3838582561 conn(0x55a658e5d4e0 msgr2=0x55a659207020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 -- 172.21.15.78:0/3838582561 shutdown_connections 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 -- 172.21.15.78:0/3838582561 wait complete. 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 -- start start 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.920+0000 7fb496a04740 1 Processor -- start 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e404d0 0x55a659215560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e486b0 0x55a659215aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a65921ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a65920d530 con 0x55a658e486b0 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a6590f6dc0 con 0x55a658e69390 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a65920d6b0 con 0x55a658e404d0 2024-04-25T14:06:18.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 10 client.0 _ll_register_callbacks cb 0x55a659112600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e486b0 0x55a659215aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e486b0 0x55a659215aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44410/0 (socket says 172.21.15.78:44410) 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 -- 172.21.15.78:0/1046691127 learned_addr learned my addr 172.21.15.78:0/1046691127 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a65921ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e404d0 0x55a659215560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 -- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e404d0 msgr2=0x55a659215560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e404d0 0x55a659215560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 -- 172.21.15.78:0/1046691127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 msgr2=0x55a65921ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a65921ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 -- 172.21.15.78:0/1046691127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a658e813f0 con 0x55a658e486b0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e486b0 0x55a659215aa0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb4840fd0b0 tx=0x7fb4840095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a15db2f5bfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb4717fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb4717fa640 1 -- 172.21.15.78:0/1046691127 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484049070 con 0x55a658e486b0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb4717fa640 1 -- 172.21.15.78:0/1046691127 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb4840c7980 con 0x55a658e486b0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.921+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48001cd90 con 0x55a658e486b0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb4717fa640 1 -- 172.21.15.78:0/1046691127 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb4840c6e90 con 0x55a658e486b0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 10 client.17526 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a658f4b7d0 con 0x55a658e486b0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb4717fa640 1 -- 172.21.15.78:0/1046691127 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48001cd90 con 0x55a658e486b0 2024-04-25T14:06:18.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb4717fa640 1 -- 172.21.15.78:0/1046691127 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb4840d00f0 con 0x55a658e486b0 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 10 client.17526 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 10 client.17526 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 20 client.17526 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658e27f60 0x55a659223180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 4 client.17526 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a65917d990 con 0x55a658e27f60 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.922+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658e27f60 0x55a659223180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.923+0000 7fb4717fa640 10 client.17526 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.923+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658e27f60 0x55a659223180 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2421d5cbc22f97af server_cookie=c574c709559e7ea1 in_seq=0 out_seq=0 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.923+0000 7fb4717fa640 1 -- 172.21.15.78:0/1046691127 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a65917d990 con 0x55a658e27f60 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.923+0000 7fb4717fa640 10 client.17526 handle_command_reply: tid=0 2024-04-25T14:06:18.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.923+0000 7fb496a04740 1 client.17526 shutdown 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e486b0 msgr2=0x55a659215aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e486b0 0x55a659215aa0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb4840fd0b0 tx=0x7fb4840095e0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 shutdown_connections 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a65921ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e486b0 0x55a659215aa0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658e27f60 0x55a659223180 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 --2- 172.21.15.78:0/1046691127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e404d0 0x55a659215560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 >> 172.21.15.78:0/1046691127 conn(0x55a658e5d4e0 msgr2=0x55a659209f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb4717fa640 0 client.17526 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:18.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 shutdown_connections 2024-04-25T14:06:18.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 1 -- 172.21.15.78:0/1046691127 wait complete. 2024-04-25T14:06:18.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.924+0000 7fb496a04740 20 client.17526 trim_cache size 0 max 16384 2024-04-25T14:06:18.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 Processor -- start 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 -- start start 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e404d0 0x55a659206a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a659206f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a659213cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881adac0 con 0x55a658e404d0 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881ab390 con 0x55a658e69390 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.937+0000 7fb496a04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881a91e0 con 0x55a658e486b0 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e404d0 0x55a659206a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e404d0 0x55a659206a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44412/0 (socket says 172.21.15.78:44412) 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 -- 172.21.15.78:0/1496535500 learned_addr learned my addr 172.21.15.78:0/1496535500 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 -- 172.21.15.78:0/1496535500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 msgr2=0x55a659206f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a659206f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a659213cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a659206f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 -- 172.21.15.78:0/1496535500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 msgr2=0x55a659213cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a659213cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 -- 172.21.15.78:0/1496535500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6592166a0 con 0x55a658e404d0 2024-04-25T14:06:18.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a659206f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a659213cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e404d0 0x55a659206a50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb47800b620 tx=0x7fb47800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcc41d102bc5d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb4737fe640 1 -- 172.21.15.78:0/1496535500 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478003ea0 con 0x55a658e404d0 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb4737fe640 1 -- 172.21.15.78:0/1496535500 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478023030 con 0x55a658e404d0 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.938+0000 7fb4737fe640 1 -- 172.21.15.78:0/1496535500 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47801f0f0 con 0x55a658e404d0 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 -- 172.21.15.78:0/1496535500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e404d0 msgr2=0x55a659206a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e404d0 0x55a659206a50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb47800b620 tx=0x7fb47800b390 comp rx=0 tx=0).stop 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 -- 172.21.15.78:0/1496535500 shutdown_connections 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e69390 0x55a659213cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a659206f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 --2- 172.21.15.78:0/1496535500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e404d0 0x55a659206a50 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 -- 172.21.15.78:0/1496535500 >> 172.21.15.78:0/1496535500 conn(0x55a658e5d4e0 msgr2=0x55a6592102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 -- 172.21.15.78:0/1496535500 shutdown_connections 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 -- 172.21.15.78:0/1496535500 wait complete. 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 -- start start 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.939+0000 7fb496a04740 1 Processor -- start 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb496a04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a659219280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb496a04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a6592197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb496a04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65921a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb496a04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881a9850 con 0x55a658e69390 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb496a04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881aacc0 con 0x55a658e404d0 2024-04-25T14:06:18.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb496a04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881a9f20 con 0x55a658e486b0 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb496a04740 10 client.0 _ll_register_callbacks cb 0x7fb484005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65921a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65921a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44414/0 (socket says 172.21.15.78:44414) 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 -- 172.21.15.78:0/2309254961 learned_addr learned my addr 172.21.15.78:0/2309254961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48d0e8640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a6592197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a659219280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 -- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 msgr2=0x55a6592197c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a6592197c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 -- 172.21.15.78:0/2309254961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 msgr2=0x55a659219280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a659219280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 -- 172.21.15.78:0/2309254961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a65921ac60 con 0x55a658e69390 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a659219280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:18.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb48e0ea640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65921a6a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb48000f450 tx=0x7fb480027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d64da4e1b1b31a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb4717fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.940+0000 7fb4717fa640 1 -- 172.21.15.78:0/2309254961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480033030 con 0x55a658e69390 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb4717fa640 1 -- 172.21.15.78:0/2309254961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb48001ca70 con 0x55a658e69390 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb478023030 con 0x55a658e69390 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb4717fa640 1 -- 172.21.15.78:0/2309254961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb480018040 con 0x55a658e69390 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 10 client.17532 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a65921be30 con 0x55a658e69390 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb4717fa640 1 -- 172.21.15.78:0/2309254961 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478023030 con 0x55a658e69390 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb4717fa640 1 -- 172.21.15.78:0/2309254961 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb4800258e0 con 0x55a658e69390 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 10 client.17532 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 10 client.17532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 20 client.17532 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658dfd3a0 0x55a65928d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:18.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 4 client.17532 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a659177800 con 0x55a658dfd3a0 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.941+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658dfd3a0 0x55a65928d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb4717fa640 10 client.17532 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb48d8e9640 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658dfd3a0 0x55a65928d670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19da17b4932c93d8 server_cookie=ba78d2a03ab7d05c in_seq=0 out_seq=0 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb4717fa640 1 -- 172.21.15.78:0/2309254961 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a659177800 con 0x55a658dfd3a0 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb4717fa640 10 client.17532 handle_command_reply: tid=0 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 client.17532 shutdown 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 msgr2=0x55a65921a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65921a6a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb48000f450 tx=0x7fb480027000 comp rx=0 tx=0).stop 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 shutdown_connections 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a658e69390 0x55a65921a6a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55a658dfd3a0 0x55a65928d670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a658e486b0 0x55a6592197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 --2- 172.21.15.78:0/2309254961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a658e404d0 0x55a659219280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb4717fa640 0 client.17532 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 >> 172.21.15.78:0/2309254961 conn(0x55a658e5d4e0 msgr2=0x55a6592069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 shutdown_connections 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 1 -- 172.21.15.78:0/2309254961 wait complete. 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.942+0000 7fb496a04740 20 client.17532 trim_cache size 0 max 16384 2024-04-25T14:06:18.951 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4740afaf0 msgr2=0x7fb4740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4740afaf0 0x7fb4740b2000 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb48001f000 tx=0x7fb48001d000 comp rx=0 tx=0).stop 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 msgr2=0x7fb48819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb48819a750 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484031a50 comp rx=0 tx=0).stop 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 shutdown_connections 2024-04-25T14:06:18.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488109790 0x7fb4881a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb4740afaf0 0x7fb4740b2000 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb488102370 0x7fb48819a750 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 --2- 172.21.15.78:0/3820537949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488101e70 0x7fb48819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:18.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 >> 172.21.15.78:0/3820537949 conn(0x7fb4880fd380 msgr2=0x7fb488105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:18.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 shutdown_connections 2024-04-25T14:06:18.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:18.943+0000 7fb48fb4d640 1 -- 172.21.15.78:0/3820537949 wait complete. 2024-04-25T14:06:18.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:06:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 -- 172.21.15.78:0/4078371019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0103930 msgr2=0x7fc6f0103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/4078371019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0103930 0x7fc6f0103de0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 -- 172.21.15.78:0/4078371019 shutdown_connections 2024-04-25T14:06:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/4078371019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6f0104470 0x7fc6f010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/4078371019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0103930 0x7fc6f0103de0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/4078371019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f0102f50 0x7fc6f0103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 -- 172.21.15.78:0/4078371019 >> 172.21.15.78:0/4078371019 conn(0x7fc6f00fd350 msgr2=0x7fc6f00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 -- 172.21.15.78:0/4078371019 shutdown_connections 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.063+0000 7fc6f6d84640 1 -- 172.21.15.78:0/4078371019 wait complete. 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 Processor -- start 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 -- start start 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6f0102f50 0x7fc6f019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f0103930 0x7fc6f019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0104470 0x7fc6f01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010e520 con 0x7fc6f0104470 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010e3a0 con 0x7fc6f0103930 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f6d84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010e6a0 con 0x7fc6f0102f50 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0104470 0x7fc6f01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0104470 0x7fc6f01a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44430/0 (socket says 172.21.15.78:44430) 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 -- 172.21.15.78:0/1712700687 learned_addr learned my addr 172.21.15.78:0/1712700687 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6effff640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f0103930 0x7fc6f019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6f0102f50 0x7fc6f019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 -- 172.21.15.78:0/1712700687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6f0102f50 msgr2=0x7fc6f019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6f0102f50 0x7fc6f019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 -- 172.21.15.78:0/1712700687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f0103930 msgr2=0x7fc6f019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f0103930 0x7fc6f019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 -- 172.21.15.78:0/1712700687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f01a1ca0 con 0x7fc6f0104470 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.064+0000 7fc6f5321640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0104470 0x7fc6f01a16e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800cce0 tx=0x7fc6e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.065+0000 7fc6edffb640 1 -- 172.21.15.78:0/1712700687 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800f450 con 0x7fc6f0104470 2024-04-25T14:06:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.065+0000 7fc6edffb640 1 -- 172.21.15.78:0/1712700687 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e800faa0 con 0x7fc6f0104470 2024-04-25T14:06:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.065+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f01a1f90 con 0x7fc6f0104470 2024-04-25T14:06:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.065+0000 7fc6edffb640 1 -- 172.21.15.78:0/1712700687 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8018b70 con 0x7fc6f0104470 2024-04-25T14:06:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.065+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f01a23f0 con 0x7fc6f0104470 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6edffb640 1 -- 172.21.15.78:0/1712700687 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc6e8018d10 con 0x7fc6f0104470 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 Processor -- start 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 -- start start 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56148184ed80 con 0x5614815276b0 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561481808a20 con 0x56148151f4d0 2024-04-25T14:06:19.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.067+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56148184ad40 con 0x561481548390 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6edffb640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6e00afb70 0x7fc6e00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818eba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44446/0 (socket says 172.21.15.78:44446) 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f4b20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6effff640 1 -- 172.21.15.78:0/2519112119 learned_addr learned my addr 172.21.15.78:0/2519112119 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6effff640 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f5321640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6e00afb70 0x7fc6e00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6edffb640 1 -- 172.21.15.78:0/1712700687 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc6e80def60 con 0x7fc6f0104470 2024-04-25T14:06:19.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f4b20640 1 -- 172.21.15.78:0/2519112119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 msgr2=0x5614818ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f4b20640 1 -- 172.21.15.78:0/2519112119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 msgr2=0x5614818eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f4b20640 1 -- 172.21.15.78:0/2519112119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561481813c30 con 0x56148151f4d0 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6effff640 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f5321640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6e00afb70 0x7fc6e00b2080 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80dcdc0 tx=0x7fc6e8075f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818ead90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc6f006b4a0 tx=0x7fc6e400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdca3f7bfb2c0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.068+0000 7fc6daffd640 1 -- 172.21.15.78:0/2519112119 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4004020 con 0x56148151f4d0 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6daffd640 1 -- 172.21.15.78:0/2519112119 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e40041c0 con 0x56148151f4d0 2024-04-25T14:06:19.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6daffd640 1 -- 172.21.15.78:0/2519112119 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40186a0 con 0x56148151f4d0 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/2519112119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 msgr2=0x5614818ead90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818ead90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc6f006b4a0 tx=0x7fc6e400f3c0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/2519112119 shutdown_connections 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818ead90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/2519112119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/2519112119 >> 172.21.15.78:0/2519112119 conn(0x56148153c4e0 msgr2=0x5614818e6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/2519112119 shutdown_connections 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.069+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/2519112119 wait complete. 2024-04-25T14:06:19.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 -- start start 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 Processor -- start 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56148151f4d0 0x5614818f4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614815276b0 0x5614818f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5614818ec530 con 0x56148151f4d0 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5614817d5dc0 con 0x5614815276b0 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5614818ec6b0 con 0x561481548390 2024-04-25T14:06:19.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6fdc3b740 10 client.0 _ll_register_callbacks cb 0x5614817f1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56148151f4d0 0x5614818f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56148151f4d0 0x5614818f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44456/0 (socket says 172.21.15.78:44456) 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6effff640 1 -- 172.21.15.78:0/71233384 learned_addr learned my addr 172.21.15.78:0/71233384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.070+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614815276b0 0x5614818f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6effff640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6effff640 1 -- 172.21.15.78:0/71233384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 msgr2=0x5614818fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6effff640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6effff640 1 -- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614815276b0 msgr2=0x5614818f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6effff640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614815276b0 0x5614818f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6effff640 1 -- 172.21.15.78:0/71233384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614815603f0 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614815276b0 0x5614818f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6effff640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56148151f4d0 0x5614818f4510 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005ab0 tx=0x7fc6dc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e828a1dd6696e2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6cffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6cffff640 1 -- 172.21.15.78:0/71233384 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc035040 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6cffff640 1 -- 172.21.15.78:0/71233384 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc6dc004240 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e40041c0 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6cffff640 1 -- 172.21.15.78:0/71233384 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc6dc004440 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6fdc3b740 10 client.17544 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56148162a7d0 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.071+0000 7fc6cffff640 1 -- 172.21.15.78:0/71233384 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e40041c0 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6cffff640 1 -- 172.21.15.78:0/71233384 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc6dc032240 con 0x56148151f4d0 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6fdc3b740 10 client.17544 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:19.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6fdc3b740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6fdc3b740 20 client.17544 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481506f60 0x5614819022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6fdc3b740 4 client.17544 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148185c990 con 0x561481506f60 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481506f60 0x5614819022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6cffff640 10 client.17544 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.072+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481506f60 0x5614819022a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44fa75941aca70d server_cookie=6ce724904287f993 in_seq=0 out_seq=0 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6cffff640 1 -- 172.21.15.78:0/71233384 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56148185c990 con 0x561481506f60 2024-04-25T14:06:19.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6cffff640 10 client.17544 handle_command_reply: tid=0 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 client.17544 shutdown 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56148151f4d0 msgr2=0x5614818f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56148151f4d0 0x5614818f4510 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005ab0 tx=0x7fc6dc03f040 comp rx=0 tx=0).stop 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 shutdown_connections 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481506f60 0x5614819022a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481548390 0x5614818fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6cffff640 0 client.17544 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5614815276b0 0x5614818f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/71233384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56148151f4d0 0x5614818f4510 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.073+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 >> 172.21.15.78:0/71233384 conn(0x56148153c4e0 msgr2=0x5614818e8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.074+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 shutdown_connections 2024-04-25T14:06:19.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.074+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/71233384 wait complete. 2024-04-25T14:06:19.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.074+0000 7fc6fdc3b740 20 client.17544 trim_cache size 0 max 16384 2024-04-25T14:06:19.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 Processor -- start 2024-04-25T14:06:19.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 -- start start 2024-04-25T14:06:19.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481506f60 0x5614818f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ad990 con 0x5614815276b0 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ab260 con 0x56148151f4d0 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01a90b0 con 0x561481506f60 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44472/0 (socket says 172.21.15.78:44472) 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6effff640 1 -- 172.21.15.78:0/671640980 learned_addr learned my addr 172.21.15.78:0/671640980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.087+0000 7fc6effff640 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481506f60 0x5614818f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6effff640 1 -- 172.21.15.78:0/671640980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481506f60 msgr2=0x5614818f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6effff640 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481506f60 0x5614818f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6effff640 1 -- 172.21.15.78:0/671640980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 msgr2=0x5614818f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6effff640 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6effff640 1 -- 172.21.15.78:0/671640980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614818efc20 con 0x5614815276b0 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6effff640 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818f4620 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc9707fd0ccb744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6da7fc640 1 -- 172.21.15.78:0/671640980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc032300 con 0x5614815276b0 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6da7fc640 1 -- 172.21.15.78:0/671640980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc03d960 con 0x5614815276b0 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.088+0000 7fc6da7fc640 1 -- 172.21.15.78:0/671640980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc051720 con 0x5614815276b0 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/671640980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 msgr2=0x5614818f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818f4620 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc005fd0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/671640980 shutdown_connections 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614815276b0 0x5614818f4620 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/671640980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561481506f60 0x5614818f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/671640980 >> 172.21.15.78:0/671640980 conn(0x56148153c4e0 msgr2=0x5614818f2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/671640980 shutdown_connections 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/671640980 wait complete. 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 -- start start 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.089+0000 7fc6fdc3b740 1 Processor -- start 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561481506f60 0x5614818f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614815276b0 0x561481968640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01a9720 con 0x561481506f60 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01aab90 con 0x56148151f4d0 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01a9df0 con 0x5614815276b0 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 10 client.0 _ll_register_callbacks cb 0x7fc6dc005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561481506f60 0x5614818f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561481506f60 0x5614818f9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44486/0 (socket says 172.21.15.78:44486) 2024-04-25T14:06:19.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 -- 172.21.15.78:0/3395337489 learned_addr learned my addr 172.21.15.78:0/3395337489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 -- 172.21.15.78:0/3395337489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614815276b0 msgr2=0x561481968640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614815276b0 0x561481968640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 -- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 msgr2=0x5614818fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 -- 172.21.15.78:0/3395337489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561481968b80 con 0x561481506f60 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6effff640 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561481506f60 0x5614818f9f70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005a20 tx=0x7fc6dc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983a4313512cc5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6cffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6cffff640 1 -- 172.21.15.78:0/3395337489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc035040 con 0x561481506f60 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6cffff640 1 -- 172.21.15.78:0/3395337489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc6dc03dd10 con 0x561481506f60 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6dc03d960 con 0x561481506f60 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6cffff640 1 -- 172.21.15.78:0/3395337489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc6dc051500 con 0x561481506f60 2024-04-25T14:06:19.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 10 client.17550 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.090+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561481969cb0 con 0x561481506f60 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6cffff640 1 -- 172.21.15.78:0/3395337489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc03d960 con 0x561481506f60 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6cffff640 1 -- 172.21.15.78:0/3395337489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc6dc05e600 con 0x561481506f60 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6fdc3b740 10 client.17550 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6fdc3b740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6fdc3b740 20 client.17550 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481548390 0x56148196c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6fdc3b740 4 client.17550 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561481856800 con 0x561481548390 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.091+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481548390 0x56148196c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6cffff640 10 client.17550 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6f4b20640 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481548390 0x56148196c5d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6570ef089cf82241 server_cookie=c3838f1c56460b21 in_seq=0 out_seq=0 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6cffff640 1 -- 172.21.15.78:0/3395337489 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561481856800 con 0x561481548390 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6cffff640 10 client.17550 handle_command_reply: tid=0 2024-04-25T14:06:19.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 client.17550 shutdown 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561481506f60 msgr2=0x5614818f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561481506f60 0x5614818f9f70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc005a20 tx=0x7fc6dc031400 comp rx=0 tx=0).stop 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 shutdown_connections 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561481548390 0x56148196c5d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5614815276b0 0x561481968640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56148151f4d0 0x5614818fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6cffff640 0 client.17550 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 --2- 172.21.15.78:0/3395337489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561481506f60 0x5614818f9f70 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 >> 172.21.15.78:0/3395337489 conn(0x56148153c4e0 msgr2=0x5614818f40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 shutdown_connections 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 1 -- 172.21.15.78:0/3395337489 wait complete. 2024-04-25T14:06:19.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.092+0000 7fc6fdc3b740 20 client.17550 trim_cache size 0 max 16384 2024-04-25T14:06:19.099 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:06:19.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6e00afb70 msgr2=0x7fc6e00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6e00afb70 0x7fc6e00b2080 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80dcdc0 tx=0x7fc6e8075f80 comp rx=0 tx=0).stop 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0104470 msgr2=0x7fc6f01a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0104470 0x7fc6f01a16e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800cce0 tx=0x7fc6e800a620 comp rx=0 tx=0).stop 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 shutdown_connections 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc6e00afb70 0x7fc6e00b2080 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6f0104470 0x7fc6f01a16e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6f0103930 0x7fc6f019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 --2- 172.21.15.78:0/1712700687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6f0102f50 0x7fc6f019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 >> 172.21.15.78:0/1712700687 conn(0x7fc6f00fd350 msgr2=0x7fc6f0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 shutdown_connections 2024-04-25T14:06:19.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:19.093+0000 7fc6f6d84640 1 -- 172.21.15.78:0/1712700687 wait complete. 2024-04-25T14:06:19.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:19 smithi129 ceph-mon[32569]: pgmap v552: 97 pgs: 97 active+clean; 763 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:06:19.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:19 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3854263079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:19 smithi078 ceph-mon[29163]: pgmap v552: 97 pgs: 97 active+clean; 763 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:06:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:19 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3854263079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:19.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:19 smithi112 ceph-mon[32617]: pgmap v552: 97 pgs: 97 active+clean; 763 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:06:19.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:19 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3854263079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:20.104 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:06:20.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:20.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:06:20.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 -- 172.21.15.78:0/462904679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474102a00 msgr2=0x7fa474103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 --2- 172.21.15.78:0/462904679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474102a00 0x7fa474103700 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa464009900 tx=0x7fa46402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:20.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 -- 172.21.15.78:0/462904679 shutdown_connections 2024-04-25T14:06:20.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 --2- 172.21.15.78:0/462904679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa474108940 0x7fa47410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 --2- 172.21.15.78:0/462904679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474103c40 0x7fa474104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 --2- 172.21.15.78:0/462904679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474102a00 0x7fa474103700 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 -- 172.21.15.78:0/462904679 >> 172.21.15.78:0/462904679 conn(0x7fa4740fdbf0 msgr2=0x7fa4741000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 -- 172.21.15.78:0/462904679 shutdown_connections 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 -- 172.21.15.78:0/462904679 wait complete. 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 Processor -- start 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.239+0000 7fa47a91f640 1 -- start start 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa47a91f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa474102a00 0x7fa474075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa47a91f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474103c40 0x7fa47419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa47a91f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474108940 0x7fa4741a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa47a91f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa47410e4e0 con 0x7fa474108940 2024-04-25T14:06:20.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa47a91f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa47410e360 con 0x7fa474102a00 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa47a91f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa47410e660 con 0x7fa474103c40 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474108940 0x7fa4741a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474108940 0x7fa4741a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44508/0 (socket says 172.21.15.78:44508) 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 -- 172.21.15.78:0/1310336084 learned_addr learned my addr 172.21.15.78:0/1310336084 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa4737fe640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474103c40 0x7fa47419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa473fff640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa474102a00 0x7fa474075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 -- 172.21.15.78:0/1310336084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474103c40 msgr2=0x7fa47419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474103c40 0x7fa47419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 -- 172.21.15.78:0/1310336084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa474102a00 msgr2=0x7fa474075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa474102a00 0x7fa474075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa478ebc640 1 -- 172.21.15.78:0/1310336084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4741a1d30 con 0x7fa474108940 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.240+0000 7fa473fff640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa474102a00 0x7fa474075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.241+0000 7fa478ebc640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474108940 0x7fa4741a1510 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00eb10 tx=0x7fa46c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.241+0000 7fa4717fa640 1 -- 172.21.15.78:0/1310336084 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c004b20 con 0x7fa474108940 2024-04-25T14:06:20.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.241+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4741a1fc0 con 0x7fa474108940 2024-04-25T14:06:20.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.241+0000 7fa4717fa640 1 -- 172.21.15.78:0/1310336084 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c004cc0 con 0x7fa474108940 2024-04-25T14:06:20.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.241+0000 7fa4717fa640 1 -- 172.21.15.78:0/1310336084 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c010670 con 0x7fa474108940 2024-04-25T14:06:20.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.241+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4741a2480 con 0x7fa474108940 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.243+0000 7fa4717fa640 1 -- 172.21.15.78:0/1310336084 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa46c005020 con 0x7fa474108940 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.243+0000 7fa4817d1740 1 Processor -- start 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.243+0000 7fa4817d1740 1 -- start start 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd41410 0x561a1d0e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0e9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4817d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561a1d047b00 con 0x561a1cd20730 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4817d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a1cfe3920 con 0x561a1cd41410 2024-04-25T14:06:20.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4817d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561a1d043ac0 con 0x561a1cd18550 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0e9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44512/0 (socket says 172.21.15.78:44512) 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 -- 172.21.15.78:0/227518935 learned_addr learned my addr 172.21.15.78:0/227518935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd41410 0x561a1d0e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4717fa640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa45c0afaf0 0x7fa45c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa473fff640 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa478ebc640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa45c0afaf0 0x7fa45c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 -- 172.21.15.78:0/227518935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 msgr2=0x561a1d0e2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0e2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 -- 172.21.15.78:0/227518935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd41410 msgr2=0x561a1d0e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd41410 0x561a1d0e24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 -- 172.21.15.78:0/227518935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1ce217e0 con 0x561a1cd20730 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4717fa640 1 -- 172.21.15.78:0/1310336084 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa46c0df200 con 0x7fa474108940 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.244+0000 7fa4737fe640 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0e9920 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa4680127f0 tx=0x7fa46800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56607d5bd80bafcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa478ebc640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa45c0afaf0 0x7fa45c0b2000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0a3750 tx=0x7fa46c076220 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa45a7fc640 1 -- 172.21.15.78:0/227518935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468012e30 con 0x561a1cd20730 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa45a7fc640 1 -- 172.21.15.78:0/227518935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468010770 con 0x561a1cd20730 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa45a7fc640 1 -- 172.21.15.78:0/227518935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468003850 con 0x561a1cd20730 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa4817d1740 1 -- 172.21.15.78:0/227518935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 msgr2=0x561a1d0e9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa4817d1740 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0e9920 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa4680127f0 tx=0x7fa46800b860 comp rx=0 tx=0).stop 2024-04-25T14:06:20.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa4817d1740 1 -- 172.21.15.78:0/227518935 shutdown_connections 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa4817d1740 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0e9920 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa4817d1740 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0e2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.245+0000 7fa4817d1740 1 --2- 172.21.15.78:0/227518935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd41410 0x561a1d0e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 -- 172.21.15.78:0/227518935 >> 172.21.15.78:0/227518935 conn(0x561a1cd35560 msgr2=0x561a1d0dde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 -- 172.21.15.78:0/227518935 shutdown_connections 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 -- 172.21.15.78:0/227518935 wait complete. 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 -- start start 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 Processor -- start 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd18550 0x561a1d0ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd41410 0x561a1d0f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561a1d0ea370 con 0x561a1cd20730 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a1cfe15d0 con 0x561a1cd18550 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.246+0000 7fa4817d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561a1d0ea4f0 con 0x561a1cd41410 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4817d1740 10 client.0 _ll_register_callbacks cb 0x561a1cfcf080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0ec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44524/0 (socket says 172.21.15.78:44524) 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 -- 172.21.15.78:0/3596895919 learned_addr learned my addr 172.21.15.78:0/3596895919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd18550 0x561a1d0ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 -- 172.21.15.78:0/3596895919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd41410 msgr2=0x561a1d0f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd41410 0x561a1d0f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd41410 0x561a1d0f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 -- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd18550 msgr2=0x561a1d0ec290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd18550 0x561a1d0ec290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 -- 172.21.15.78:0/3596895919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1ce81f70 con 0x561a1cd20730 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa473fff640 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0ec7d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa464000c00 tx=0x7fa464031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a210096f8d797c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4537fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4537fe640 1 -- 172.21.15.78:0/3596895919 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464035070 con 0x561a1cd20730 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4537fe640 1 -- 172.21.15.78:0/3596895919 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa464004240 con 0x561a1cd20730 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa468010770 con 0x561a1cd20730 2024-04-25T14:06:20.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4537fe640 1 -- 172.21.15.78:0/3596895919 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa464004440 con 0x561a1cd20730 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4817d1740 10 client.17562 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.247+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561a1cd19170 con 0x561a1cd20730 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4537fe640 1 -- 172.21.15.78:0/3596895919 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468010770 con 0x561a1cd20730 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4537fe640 1 -- 172.21.15.78:0/3596895919 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa464004b40 con 0x561a1cd20730 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4817d1740 10 client.17562 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4817d1740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4817d1740 20 client.17562 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1cd001f0 0x561a1d0f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4817d1740 4 client.17562 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:20.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a1d055710 con 0x561a1cd001f0 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.248+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1cd001f0 0x561a1d0f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.249+0000 7fa4537fe640 10 client.17562 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.249+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1cd001f0 0x561a1d0f9c50 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7bbd46e3a2c89d server_cookie=e300b8667c957c1d in_seq=0 out_seq=0 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.249+0000 7fa4537fe640 1 -- 172.21.15.78:0/3596895919 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a1d055710 con 0x561a1cd001f0 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.249+0000 7fa4537fe640 10 client.17562 handle_command_reply: tid=0 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.249+0000 7fa4817d1740 1 client.17562 shutdown 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 msgr2=0x561a1d0ec7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0ec7d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa464000c00 tx=0x7fa464031850 comp rx=0 tx=0).stop 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 shutdown_connections 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1cd001f0 0x561a1d0f9c50 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd41410 0x561a1d0f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd20730 0x561a1d0ec7d0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4537fe640 0 client.17562 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3596895919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd18550 0x561a1d0ec290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 >> 172.21.15.78:0/3596895919 conn(0x561a1cd35560 msgr2=0x561a1d0dcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 shutdown_connections 2024-04-25T14:06:20.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 1 -- 172.21.15.78:0/3596895919 wait complete. 2024-04-25T14:06:20.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.250+0000 7fa4817d1740 20 client.17562 trim_cache size 0 max 16384 2024-04-25T14:06:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.267+0000 7fa4817d1740 1 Processor -- start 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4817d1740 1 -- start start 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4817d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4741adae0 con 0x561a1cd41410 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4817d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4741ab3b0 con 0x561a1cd20730 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4817d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4741a9200 con 0x561a1cd18550 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44536/0 (socket says 172.21.15.78:44536) 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 -- 172.21.15.78:0/3100784513 learned_addr learned my addr 172.21.15.78:0/3100784513 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa473fff640 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 -- 172.21.15.78:0/3100784513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 msgr2=0x561a1d0de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0de050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 -- 172.21.15.78:0/3100784513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 msgr2=0x561a1d0e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0e6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 -- 172.21.15.78:0/3100784513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1d0d8160 con 0x561a1cd41410 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f3650 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa468001eb0 tx=0x7fa468005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ee6f5977e8c0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.268+0000 7fa459ffb640 1 -- 172.21.15.78:0/3100784513 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800bd50 con 0x561a1cd41410 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa459ffb640 1 -- 172.21.15.78:0/3100784513 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468016070 con 0x561a1cd41410 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa459ffb640 1 -- 172.21.15.78:0/3100784513 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468010de0 con 0x561a1cd41410 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 -- 172.21.15.78:0/3100784513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 msgr2=0x561a1d0f3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f3650 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa468001eb0 tx=0x7fa468005fb0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 -- 172.21.15.78:0/3100784513 shutdown_connections 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f3650 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3100784513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 -- 172.21.15.78:0/3100784513 >> 172.21.15.78:0/3100784513 conn(0x561a1cd35560 msgr2=0x561a1d0ea890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 -- 172.21.15.78:0/3100784513 shutdown_connections 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.269+0000 7fa4817d1740 1 -- 172.21.15.78:0/3100784513 wait complete. 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 -- start start 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 Processor -- start 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4741a9870 con 0x561a1cd41410 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa4741aace0 con 0x561a1cd20730 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4741a9f40 con 0x561a1cd18550 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4817d1740 10 client.0 _ll_register_callbacks cb 0x7fa464005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44552/0 (socket says 172.21.15.78:44552) 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4737fe640 1 -- 172.21.15.78:0/3106723750 learned_addr learned my addr 172.21.15.78:0/3106723750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa473fff640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.270+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4737fe640 1 -- 172.21.15.78:0/3106723750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 msgr2=0x561a1d0f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4737fe640 1 -- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 msgr2=0x561a1d0f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4737fe640 1 -- 172.21.15.78:0/3106723750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1d0f2580 con 0x561a1cd41410 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa473fff640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4737fe640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f1fc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa468010960 tx=0x7fa46802d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf95e61f1643c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4537fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4537fe640 1 -- 172.21.15.78:0/3106723750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46802b560 con 0x561a1cd41410 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4537fe640 1 -- 172.21.15.78:0/3106723750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa46802b700 con 0x561a1cd41410 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa468016070 con 0x561a1cd41410 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4537fe640 1 -- 172.21.15.78:0/3106723750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa46802b900 con 0x561a1cd41410 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4817d1740 10 client.17568 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561a1d0f3840 con 0x561a1cd41410 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.271+0000 7fa4537fe640 1 -- 172.21.15.78:0/3106723750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa468016070 con 0x561a1cd41410 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4537fe640 1 -- 172.21.15.78:0/3106723750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa4680046f0 con 0x561a1cd41410 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4817d1740 10 client.17568 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4817d1740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4817d1740 20 client.17568 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1ccd5630 0x561a1d164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4817d1740 4 client.17568 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a1d04f580 con 0x561a1ccd5630 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa473fff640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1ccd5630 0x561a1d164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa4537fe640 10 client.17568 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.272+0000 7fa473fff640 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1ccd5630 0x561a1d164490 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c4ed0c123af48e server_cookie=a83f2b6a757cc60e in_seq=0 out_seq=0 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4537fe640 1 -- 172.21.15.78:0/3106723750 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a1d04f580 con 0x561a1ccd5630 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4537fe640 10 client.17568 handle_command_reply: tid=0 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 client.17568 shutdown 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 msgr2=0x561a1d0f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f1fc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa468010960 tx=0x7fa46802d040 comp rx=0 tx=0).stop 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 shutdown_connections 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a1cd41410 0x561a1d0f1fc0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a1cd20730 0x561a1d0f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561a1ccd5630 0x561a1d164490 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 --2- 172.21.15.78:0/3106723750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561a1cd18550 0x561a1d0f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4537fe640 0 client.17568 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 >> 172.21.15.78:0/3106723750 conn(0x561a1cd35560 msgr2=0x561a1d0dddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 shutdown_connections 2024-04-25T14:06:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 1 -- 172.21.15.78:0/3106723750 wait complete. 2024-04-25T14:06:20.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa4817d1740 20 client.17568 trim_cache size 0 max 16384 2024-04-25T14:06:20.282 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:06:20.282 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:06:20.282 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "9d873589-fefc-42b0-9bf1-77e4eee66447", 2024-04-25T14:06:20.282 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:06:20.282 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:06:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa45c0afaf0 msgr2=0x7fa45c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa47a91f640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa45c0afaf0 0x7fa45c0b2000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0a3750 tx=0x7fa46c076220 comp rx=0 tx=0).stop 2024-04-25T14:06:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474108940 msgr2=0x7fa4741a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.274+0000 7fa47a91f640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474108940 0x7fa4741a1510 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00eb10 tx=0x7fa46c00c2d0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 shutdown_connections 2024-04-25T14:06:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa45c0afaf0 0x7fa45c0b2000 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa474108940 0x7fa4741a1510 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474103c40 0x7fa47419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 --2- 172.21.15.78:0/1310336084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa474102a00 0x7fa474075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 >> 172.21.15.78:0/1310336084 conn(0x7fa4740fdbf0 msgr2=0x7fa474104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 shutdown_connections 2024-04-25T14:06:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.275+0000 7fa47a91f640 1 -- 172.21.15.78:0/1310336084 wait complete. 2024-04-25T14:06:20.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:06:20.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:06:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 -- 172.21.15.78:0/1812762962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920102180 msgr2=0x7fb920102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 --2- 172.21.15.78:0/1812762962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920102180 0x7fb920102e80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb914009900 tx=0x7fb91402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:20.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 -- 172.21.15.78:0/1812762962 shutdown_connections 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 --2- 172.21.15.78:0/1812762962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9201080c0 0x7fb92010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 --2- 172.21.15.78:0/1812762962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9201033c0 0x7fb9201037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 --2- 172.21.15.78:0/1812762962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb920102180 0x7fb920102e80 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 -- 172.21.15.78:0/1812762962 >> 172.21.15.78:0/1812762962 conn(0x7fb9200fd350 msgr2=0x7fb9200ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 -- 172.21.15.78:0/1812762962 shutdown_connections 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.398+0000 7fb926d6b640 1 -- 172.21.15.78:0/1812762962 wait complete. 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 Processor -- start 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 -- start start 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb920102180 0x7fb92019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9201033c0 0x7fb92019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9201080c0 0x7fb9201a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb92010e510 con 0x7fb9201033c0 2024-04-25T14:06:20.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb92010e390 con 0x7fb9201080c0 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb926d6b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb92010e690 con 0x7fb920102180 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9201033c0 0x7fb92019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9201033c0 0x7fb92019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44560/0 (socket says 172.21.15.78:44560) 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 -- 172.21.15.78:0/4056810861 learned_addr learned my addr 172.21.15.78:0/4056810861 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb925308640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9201080c0 0x7fb9201a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb924b07640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb920102180 0x7fb92019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 -- 172.21.15.78:0/4056810861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb920102180 msgr2=0x7fb92019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb920102180 0x7fb92019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 -- 172.21.15.78:0/4056810861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9201080c0 msgr2=0x7fb9201a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9201080c0 0x7fb9201a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.399+0000 7fb91ffff640 1 -- 172.21.15.78:0/4056810861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9201a1d90 con 0x7fb9201033c0 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.400+0000 7fb924b07640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb920102180 0x7fb92019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.400+0000 7fb91ffff640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9201033c0 0x7fb92019a800 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00d950 tx=0x7fb90c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.400+0000 7fb91dffb640 1 -- 172.21.15.78:0/4056810861 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c017070 con 0x7fb9201033c0 2024-04-25T14:06:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.400+0000 7fb91dffb640 1 -- 172.21.15.78:0/4056810861 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90c009c30 con 0x7fb9201033c0 2024-04-25T14:06:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.400+0000 7fb91dffb640 1 -- 172.21.15.78:0/4056810861 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00f620 con 0x7fb9201033c0 2024-04-25T14:06:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.400+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9201a2080 con 0x7fb9201033c0 2024-04-25T14:06:20.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.400+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201a2620 con 0x7fb9201033c0 2024-04-25T14:06:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.402+0000 7fb91dffb640 1 -- 172.21.15.78:0/4056810861 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb90c010460 con 0x7fb9201033c0 2024-04-25T14:06:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.402+0000 7fb92dc18740 1 Processor -- start 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb92dc18740 1 -- start start 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18ea2390 0x559f19243a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18e794d0 0x559f19243fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f19244c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb92dc18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f191a8d80 con 0x559f18e816b0 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb92dc18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f191628b0 con 0x559f18e794d0 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb92dc18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559f191a4d40 con 0x559f18ea2390 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f19244c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91dffb640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb9100afb70 0x7fb9100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f19244c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44574/0 (socket says 172.21.15.78:44574) 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91ffff640 1 -- 172.21.15.78:0/2812855209 learned_addr learned my addr 172.21.15.78:0/2812855209 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb925308640 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18ea2390 0x559f19243a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb925308640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb9100afb70 0x7fb9100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb924b07640 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18e794d0 0x559f19243fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91dffb640 1 -- 172.21.15.78:0/4056810861 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb90c0de880 con 0x7fb9201033c0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91ffff640 1 -- 172.21.15.78:0/2812855209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18ea2390 msgr2=0x559f19243a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91ffff640 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18ea2390 0x559f19243a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91ffff640 1 -- 172.21.15.78:0/2812855209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18e794d0 msgr2=0x559f19243fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.403+0000 7fb91ffff640 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18e794d0 0x559f19243fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.404+0000 7fb91ffff640 1 -- 172.21.15.78:0/2812855209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f19133a40 con 0x559f18e816b0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.404+0000 7fb924b07640 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18e794d0 0x559f19243fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.404+0000 7fb925308640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb9100afb70 0x7fb9100b2080 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x559f19244ab0 tx=0x7fb918009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.404+0000 7fb91ffff640 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f19244c70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0a2d70 tx=0x7fb90c0a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be914ea57146991 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.404+0000 7fb90affd640 1 -- 172.21.15.78:0/2812855209 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c020070 con 0x559f18e816b0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.404+0000 7fb90affd640 1 -- 172.21.15.78:0/2812855209 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90c0a73f0 con 0x559f18e816b0 2024-04-25T14:06:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.404+0000 7fb90affd640 1 -- 172.21.15.78:0/2812855209 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0a2660 con 0x559f18e816b0 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 -- 172.21.15.78:0/2812855209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 msgr2=0x559f19244c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f19244c70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0a2d70 tx=0x7fb90c0a6520 comp rx=0 tx=0).stop 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 -- 172.21.15.78:0/2812855209 shutdown_connections 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f19244c70 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18e794d0 0x559f19243fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2812855209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18ea2390 0x559f19243a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 -- 172.21.15.78:0/2812855209 >> 172.21.15.78:0/2812855209 conn(0x559f18e964e0 msgr2=0x559f1923f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 -- 172.21.15.78:0/2812855209 shutdown_connections 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 -- 172.21.15.78:0/2812855209 wait complete. 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 -- start start 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.405+0000 7fb92dc18740 1 Processor -- start 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e794d0 0x559f1924d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f1924dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19254d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f19245720 con 0x559f18e816b0 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f1912fd10 con 0x559f18ea2390 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559f192458a0 con 0x559f18e794d0 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 10 client.0 _ll_register_callbacks cb 0x559f1915c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f1924dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f1924dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44586/0 (socket says 172.21.15.78:44586) 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 -- 172.21.15.78:0/1945508553 learned_addr learned my addr 172.21.15.78:0/1945508553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb924b07640 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e794d0 0x559f1924d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 -- 172.21.15.78:0/1945508553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e794d0 msgr2=0x559f1924d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e794d0 0x559f1924d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb925308640 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19254d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 -- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 msgr2=0x559f19254d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19254d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 -- 172.21.15.78:0/1945508553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f18f847d0 con 0x559f18e816b0 2024-04-25T14:06:20.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb91ffff640 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f1924dde0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0a6c80 tx=0x7fb90c0afb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186f0cc7e3a83d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb8fffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb8fffff640 1 -- 172.21.15.78:0/1945508553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c020040 con 0x559f18e816b0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb8fffff640 1 -- 172.21.15.78:0/1945508553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb90c0a5f50 con 0x559f18e816b0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb90c0a73f0 con 0x559f18e816b0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb8fffff640 1 -- 172.21.15.78:0/1945508553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb90c0c13d0 con 0x559f18e816b0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 10 client.17580 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.406+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559f18eba3f0 con 0x559f18e816b0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb8fffff640 1 -- 172.21.15.78:0/1945508553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90c0a73f0 con 0x559f18e816b0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb8fffff640 1 -- 172.21.15.78:0/1945508553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb90c0a59c0 con 0x559f18e816b0 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb92dc18740 10 client.17580 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb92dc18740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb92dc18740 20 client.17580 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e60f60 0x559f1925ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb92dc18740 4 client.17580 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f191b6990 con 0x559f18e60f60 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb924b07640 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e60f60 0x559f1925ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb8fffff640 10 client.17580 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.407+0000 7fb924b07640 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e60f60 0x559f1925ae00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562e0bcdf95b328b server_cookie=9be9740f87e1e673 in_seq=0 out_seq=0 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb8fffff640 1 -- 172.21.15.78:0/1945508553 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f191b6990 con 0x559f18e60f60 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb8fffff640 10 client.17580 handle_command_reply: tid=0 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 client.17580 shutdown 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 msgr2=0x559f1924dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f1924dde0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0a6c80 tx=0x7fb90c0afb50 comp rx=0 tx=0).stop 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 shutdown_connections 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19254d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e816b0 0x559f1924dde0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e60f60 0x559f1925ae00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1945508553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e794d0 0x559f1924d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb8fffff640 0 client.17580 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 >> 172.21.15.78:0/1945508553 conn(0x559f18e964e0 msgr2=0x559f1923e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.408+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 shutdown_connections 2024-04-25T14:06:20.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.409+0000 7fb92dc18740 1 -- 172.21.15.78:0/1945508553 wait complete. 2024-04-25T14:06:20.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.409+0000 7fb92dc18740 20 client.17580 trim_cache size 0 max 16384 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 Processor -- start 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 -- start start 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f1923d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19254c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f1923df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201adba0 con 0x559f18e794d0 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201ab470 con 0x559f18ea2390 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb92dc18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201a92c0 con 0x559f18e816b0 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb924b07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f1923d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb924b07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f1923d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44588/0 (socket says 172.21.15.78:44588) 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.426+0000 7fb924b07640 1 -- 172.21.15.78:0/1623372905 learned_addr learned my addr 172.21.15.78:0/1623372905 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb91ffff640 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19254c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb925308640 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f1923df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb924b07640 1 -- 172.21.15.78:0/1623372905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 msgr2=0x559f19254c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb924b07640 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19254c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb924b07640 1 -- 172.21.15.78:0/1623372905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 msgr2=0x559f1923df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb924b07640 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f1923df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb924b07640 1 -- 172.21.15.78:0/1623372905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f18e7a0f0 con 0x559f18e794d0 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb924b07640 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f1923d9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb914009230 tx=0x7fb914005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bab2d66f7b92f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb90a7fc640 1 -- 172.21.15.78:0/1623372905 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914048070 con 0x559f18e794d0 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb90a7fc640 1 -- 172.21.15.78:0/1623372905 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9140054f0 con 0x559f18e794d0 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.427+0000 7fb90a7fc640 1 -- 172.21.15.78:0/1623372905 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914035030 con 0x559f18e794d0 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 -- 172.21.15.78:0/1623372905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 msgr2=0x559f1923d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f1923d9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb914009230 tx=0x7fb914005d50 comp rx=0 tx=0).stop 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 -- 172.21.15.78:0/1623372905 shutdown_connections 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f1923df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19254c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 --2- 172.21.15.78:0/1623372905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f1923d9c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 -- 172.21.15.78:0/1623372905 >> 172.21.15.78:0/1623372905 conn(0x559f18e964e0 msgr2=0x559f1924bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 -- 172.21.15.78:0/1623372905 shutdown_connections 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 -- 172.21.15.78:0/1623372905 wait complete. 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 -- start start 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 Processor -- start 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f192522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19252810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.428+0000 7fb92dc18740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19253740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb92dc18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201a9930 con 0x559f18e794d0 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb92dc18740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201aada0 con 0x559f18ea2390 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb92dc18740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9201aa000 con 0x559f18e816b0 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb92dc18740 10 client.0 _ll_register_callbacks cb 0x7fb918009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f192522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f192522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44598/0 (socket says 172.21.15.78:44598) 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 -- 172.21.15.78:0/2062966740 learned_addr learned my addr 172.21.15.78:0/2062966740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb91ffff640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19252810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb925308640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19253740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 -- 172.21.15.78:0/2062966740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 msgr2=0x559f19252810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19252810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 -- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 msgr2=0x559f19253740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:20.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19253740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 -- 172.21.15.78:0/2062966740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f191694b0 con 0x559f18e794d0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb925308640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19253740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb924b07640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f192522d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb914006010 tx=0x7fb914005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9cdd6d80a8d131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb8fffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb8fffff640 1 -- 172.21.15.78:0/2062966740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914048040 con 0x559f18e794d0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb8fffff640 1 -- 172.21.15.78:0/2062966740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb9140048c0 con 0x559f18e794d0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9140054f0 con 0x559f18e794d0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb8fffff640 1 -- 172.21.15.78:0/2062966740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb914035040 con 0x559f18e794d0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb92dc18740 10 client.17586 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.429+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559f19254dc0 con 0x559f18e794d0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb8fffff640 1 -- 172.21.15.78:0/2062966740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9140054f0 con 0x559f18e794d0 2024-04-25T14:06:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb8fffff640 1 -- 172.21.15.78:0/2062966740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb9140436c0 con 0x559f18e794d0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb92dc18740 10 client.17586 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb92dc18740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb92dc18740 20 client.17586 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e363a0 0x559f192c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb92dc18740 4 client.17586 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f191b0800 con 0x559f18e363a0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb91ffff640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e363a0 0x559f192c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb8fffff640 10 client.17586 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.430+0000 7fb91ffff640 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e363a0 0x559f192c5600 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b9e4bc31c875f68 server_cookie=e53a1b2dfeb82290 in_seq=0 out_seq=0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb8fffff640 1 -- 172.21.15.78:0/2062966740 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559f191b0800 con 0x559f18e363a0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb8fffff640 10 client.17586 handle_command_reply: tid=0 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 client.17586 shutdown 2024-04-25T14:06:20.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 msgr2=0x559f192522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f192522d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb914006010 tx=0x7fb914005dc0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 shutdown_connections 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f18ea2390 0x559f19253740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559f18e363a0 0x559f192c5600 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f18e816b0 0x559f19252810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb8fffff640 0 client.17586 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 --2- 172.21.15.78:0/2062966740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f18e794d0 0x559f192522d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 >> 172.21.15.78:0/2062966740 conn(0x559f18e964e0 msgr2=0x559f19242070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 shutdown_connections 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 1 -- 172.21.15.78:0/2062966740 wait complete. 2024-04-25T14:06:20.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.431+0000 7fb92dc18740 20 client.17586 trim_cache size 0 max 16384 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: "9d873589-fefc-42b0-9bf1-77e4eee66447": { 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "9d873589-fefc-42b0-9bf1-77e4eee66447", 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:06:20.439 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb9100afb70 msgr2=0x7fb9100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb9100afb70 0x7fb9100b2080 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x559f19244ab0 tx=0x7fb918009600 comp rx=0 tx=0).stop 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9201033c0 msgr2=0x7fb92019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9201033c0 0x7fb92019a800 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00d950 tx=0x7fb90c00de10 comp rx=0 tx=0).stop 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 shutdown_connections 2024-04-25T14:06:20.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9201080c0 0x7fb9201a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb9100afb70 0x7fb9100b2080 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9201033c0 0x7fb92019a800 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 --2- 172.21.15.78:0/4056810861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb920102180 0x7fb92019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 >> 172.21.15.78:0/4056810861 conn(0x7fb9200fd350 msgr2=0x7fb920109610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 shutdown_connections 2024-04-25T14:06:20.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:20.432+0000 7fb926d6b640 1 -- 172.21.15.78:0/4056810861 wait complete. 2024-04-25T14:06:20.442 INFO:tasks.cephfs.filesystem:scrub status for tag:9d873589-fefc-42b0-9bf1-77e4eee66447 - {'path': '/', 'tag': '9d873589-fefc-42b0-9bf1-77e4eee66447', 'options': 'recursive,force'} 2024-04-25T14:06:21.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:21 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:06:21.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:21 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:21 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:21 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:21 smithi129 ceph-mon[32569]: pgmap v553: 97 pgs: 97 active+clean; 763 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:06:21.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:21 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:06:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:21 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:06:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:21 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:21 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:06:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:21 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:21 smithi078 ceph-mon[29163]: pgmap v553: 97 pgs: 97 active+clean; 763 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:06:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:21 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:06:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:21 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:06:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:21 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:21 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:06:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:21 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:21 smithi112 ceph-mon[32617]: pgmap v553: 97 pgs: 97 active+clean; 763 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:06:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:21 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:06:23.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:23 smithi129 ceph-mon[32569]: pgmap v554: 97 pgs: 97 active+clean; 763 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:06:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:23 smithi078 ceph-mon[29163]: pgmap v554: 97 pgs: 97 active+clean; 763 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:06:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:23 smithi112 ceph-mon[32617]: pgmap v554: 97 pgs: 97 active+clean; 763 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:06:25.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:25 smithi129 ceph-mon[32569]: pgmap v555: 97 pgs: 97 active+clean; 835 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:06:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:25 smithi078 ceph-mon[29163]: pgmap v555: 97 pgs: 97 active+clean; 835 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:06:25.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:25 smithi112 ceph-mon[32617]: pgmap v555: 97 pgs: 97 active+clean; 835 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:06:27.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:27 smithi129 ceph-mon[32569]: pgmap v556: 97 pgs: 97 active+clean; 835 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:27 smithi078 ceph-mon[29163]: pgmap v556: 97 pgs: 97 active+clean; 835 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:27.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:27 smithi112 ceph-mon[32617]: pgmap v556: 97 pgs: 97 active+clean; 835 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:29.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:29 smithi129 ceph-mon[32569]: pgmap v557: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:06:29.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:29 smithi078 ceph-mon[29163]: pgmap v557: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:06:29.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:29 smithi112 ceph-mon[32617]: pgmap v557: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:06:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 -- 172.21.15.78:0/234960448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 msgr2=0x7f143810be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 --2- 172.21.15.78:0/234960448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f143810be90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1430009ca0 tx=0x7f143002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 -- 172.21.15.78:0/234960448 shutdown_connections 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 --2- 172.21.15.78:0/234960448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f143810be90 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 --2- 172.21.15.78:0/234960448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1438104d70 0x7f1438105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 --2- 172.21.15.78:0/234960448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1438103b30 0x7f1438104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 -- 172.21.15.78:0/234960448 >> 172.21.15.78:0/234960448 conn(0x7f14380fed40 msgr2=0x7f14381011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 -- 172.21.15.78:0/234960448 shutdown_connections 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.533+0000 7f14401f3640 1 -- 172.21.15.78:0/234960448 wait complete. 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 Processor -- start 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 -- start start 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1438103b30 0x7f143819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1438104d70 0x7f143819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f14381a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f143810f6b0 con 0x7f1438109a70 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f143810f530 con 0x7f1438103b30 2024-04-25T14:06:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f14401f3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f143810f830 con 0x7f1438104d70 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f143e790640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f14381a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f143e790640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f14381a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51536/0 (socket says 172.21.15.78:51536) 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f143e790640 1 -- 172.21.15.78:0/3131194653 learned_addr learned my addr 172.21.15.78:0/3131194653 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.534+0000 7f143df8f640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1438103b30 0x7f143819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f143d78e640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1438104d70 0x7f143819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f143e790640 1 -- 172.21.15.78:0/3131194653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1438104d70 msgr2=0x7f143819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f143e790640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1438104d70 0x7f143819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f143e790640 1 -- 172.21.15.78:0/3131194653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1438103b30 msgr2=0x7f143819a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f143e790640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1438103b30 0x7f143819a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f143e790640 1 -- 172.21.15.78:0/3131194653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14381a1d40 con 0x7f1438109a70 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f143e790640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f14381a16d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f14300097b0 tx=0x7f1430003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f142effd640 1 -- 172.21.15.78:0/3131194653 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430004240 con 0x7f1438109a70 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f142effd640 1 -- 172.21.15.78:0/3131194653 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14300317c0 con 0x7f1438109a70 2024-04-25T14:06:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14381a1fd0 con 0x7f1438109a70 2024-04-25T14:06:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f142effd640 1 -- 172.21.15.78:0/3131194653 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143002fde0 con 0x7f1438109a70 2024-04-25T14:06:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.535+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14381a24b0 con 0x7f1438109a70 2024-04-25T14:06:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f142effd640 1 -- 172.21.15.78:0/3131194653 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f14300043e0 con 0x7f1438109a70 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 Processor -- start 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 -- start start 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0da81b0 0x558bb114cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb114d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb1154200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558bb10ae460 con 0x558bb0da81b0 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558bb1068590 con 0x558bb0d7f2f0 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f1447180740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558bb10aa420 con 0x558bb0d874d0 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f143d78e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0da81b0 0x558bb114cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f143d78e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0da81b0 0x558bb114cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51538/0 (socket says 172.21.15.78:51538) 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f143d78e640 1 -- 172.21.15.78:0/1295997479 learned_addr learned my addr 172.21.15.78:0/1295997479 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f143df8f640 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb114d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f143d78e640 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb1154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.538+0000 7f142effd640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f14240afae0 0x7f14240b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143d78e640 1 -- 172.21.15.78:0/1295997479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 msgr2=0x558bb1154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143d78e640 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb1154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143d78e640 1 -- 172.21.15.78:0/1295997479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 msgr2=0x558bb114d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143d78e640 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb114d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143d78e640 1 -- 172.21.15.78:0/1295997479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb10750d0 con 0x558bb0da81b0 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143e790640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f14240afae0 0x7f14240b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f142effd640 1 -- 172.21.15.78:0/3131194653 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1430100570 con 0x7f1438109a70 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143d78e640 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0da81b0 0x558bb114cd80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1428009f90 tx=0x7f1428009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644d4dcc0d299019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f1423fff640 1 -- 172.21.15.78:0/1295997479 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142801d070 con 0x558bb0da81b0 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f1423fff640 1 -- 172.21.15.78:0/1295997479 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1428010040 con 0x558bb0da81b0 2024-04-25T14:06:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f1423fff640 1 -- 172.21.15.78:0/1295997479 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428018920 con 0x558bb0da81b0 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.539+0000 7f143e790640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f14240afae0 0x7f14240b1ff0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f14300c89d0 tx=0x7f14300f7190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 -- 172.21.15.78:0/1295997479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0da81b0 msgr2=0x558bb114cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0da81b0 0x558bb114cd80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1428009f90 tx=0x7f1428009420 comp rx=0 tx=0).stop 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 -- 172.21.15.78:0/1295997479 shutdown_connections 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb1154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb114d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 --2- 172.21.15.78:0/1295997479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0da81b0 0x558bb114cd80 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 -- 172.21.15.78:0/1295997479 >> 172.21.15.78:0/1295997479 conn(0x558bb0d9c300 msgr2=0x558bb1144300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 -- 172.21.15.78:0/1295997479 shutdown_connections 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 -- 172.21.15.78:0/1295997479 wait complete. 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.540+0000 7f1447180740 1 -- start start 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 1 Processor -- start 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1156b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb1157090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb115df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558bb1154c50 con 0x558bb0d874d0 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558bb1035b60 con 0x558bb0d7f2f0 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558bb1154dd0 con 0x558bb0da81b0 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f1447180740 10 client.0 _ll_register_callbacks cb 0x558bb105a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb1157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb1157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51550/0 (socket says 172.21.15.78:51550) 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 -- 172.21.15.78:0/274217755 learned_addr learned my addr 172.21.15.78:0/274217755 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143d78e640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb115df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143d78e640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 -- 172.21.15.78:0/274217755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 msgr2=0x558bb115df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb115df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 -- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 msgr2=0x558bb1156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143df8f640 1 -- 172.21.15.78:0/274217755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb0dc0210 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.541+0000 7f143d78e640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f143df8f640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb1157090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1434010820 tx=0x7f1434010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8623886ef5189239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f14217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f14217fa640 1 -- 172.21.15.78:0/274217755 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143400fc50 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f14217fa640 1 -- 172.21.15.78:0/274217755 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f143400fdf0 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1428010040 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f14217fa640 1 -- 172.21.15.78:0/274217755 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1434004030 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 10 client.17598 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558bb0e8c620 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f14217fa640 1 -- 172.21.15.78:0/274217755 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1428010040 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f14217fa640 1 -- 172.21.15.78:0/274217755 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1434019400 con 0x558bb0d874d0 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 10 client.17598 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 20 client.17598 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d66d80 0x558bb11648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 4 client.17598 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.542+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bb10bc070 con 0x558bb0d66d80 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.543+0000 7f143d78e640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d66d80 0x558bb11648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.543+0000 7f14217fa640 10 client.17598 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.543+0000 7f143d78e640 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d66d80 0x558bb11648a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af73ea640875ae11 server_cookie=78cabacaa832ec2 in_seq=0 out_seq=0 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f14217fa640 1 -- 172.21.15.78:0/274217755 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558bb10bc070 con 0x558bb0d66d80 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f14217fa640 10 client.17598 handle_command_reply: tid=0 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 client.17598 shutdown 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 msgr2=0x558bb1157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb1157090 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1434010820 tx=0x7f1434010ce0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 shutdown_connections 2024-04-25T14:06:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d66d80 0x558bb11648a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb115df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb1157090 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 --2- 172.21.15.78:0/274217755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f14217fa640 0 client.17598 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 >> 172.21.15.78:0/274217755 conn(0x558bb0d9c300 msgr2=0x558bb1147210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 shutdown_connections 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 1 -- 172.21.15.78:0/274217755 wait complete. 2024-04-25T14:06:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.544+0000 7f1447180740 20 client.17598 trim_cache size 0 max 16384 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.559+0000 7f1447180740 1 Processor -- start 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.559+0000 7f1447180740 1 -- start start 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f1447180740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1155880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f1447180740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb115dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f1447180740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb116a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f1447180740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14381ab8f0 con 0x558bb0d874d0 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f1447180740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14381a91c0 con 0x558bb0d7f2f0 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f1447180740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14381a7010 con 0x558bb0da81b0 2024-04-25T14:06:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb115dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb115dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51558/0 (socket says 172.21.15.78:51558) 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 -- 172.21.15.78:0/634591085 learned_addr learned my addr 172.21.15.78:0/634591085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143d78e640 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb116a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 -- 172.21.15.78:0/634591085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 msgr2=0x558bb116a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb116a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 -- 172.21.15.78:0/634591085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 msgr2=0x558bb1155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 -- 172.21.15.78:0/634591085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb115e3e0 con 0x558bb0d874d0 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f143df8f640 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb115dea0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f143400ac90 tx=0x7f143400f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64020f3a7665053d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f14237fe640 1 -- 172.21.15.78:0/634591085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434030020 con 0x558bb0d874d0 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f14237fe640 1 -- 172.21.15.78:0/634591085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1434019ab0 con 0x558bb0d874d0 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.560+0000 7f14237fe640 1 -- 172.21.15.78:0/634591085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434028040 con 0x558bb0d874d0 2024-04-25T14:06:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 -- 172.21.15.78:0/634591085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 msgr2=0x558bb115dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb115dea0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f143400ac90 tx=0x7f143400f870 comp rx=0 tx=0).stop 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 -- 172.21.15.78:0/634591085 shutdown_connections 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0da81b0 0x558bb116a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d874d0 0x558bb115dea0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 --2- 172.21.15.78:0/634591085 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0d7f2f0 0x558bb1155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 -- 172.21.15.78:0/634591085 >> 172.21.15.78:0/634591085 conn(0x558bb0d9c300 msgr2=0x558bb1155170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 -- 172.21.15.78:0/634591085 shutdown_connections 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 -- 172.21.15.78:0/634591085 wait complete. 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 -- start start 2024-04-25T14:06:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.561+0000 7f1447180740 1 Processor -- start 2024-04-25T14:06:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f1447180740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d7f2f0 0x558bb115b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f1447180740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb115bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f1447180740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0da81b0 0x558bb115caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f1447180740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14381a7680 con 0x558bb0d7f2f0 2024-04-25T14:06:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f1447180740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14381a8af0 con 0x558bb0da81b0 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f1447180740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14381a7d50 con 0x558bb0d874d0 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f1447180740 10 client.0 _ll_register_callbacks cb 0x7f143400a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d7f2f0 0x558bb115b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d7f2f0 0x558bb115b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51566/0 (socket says 172.21.15.78:51566) 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 -- 172.21.15.78:0/3575659507 learned_addr learned my addr 172.21.15.78:0/3575659507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143df8f640 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb115bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 -- 172.21.15.78:0/3575659507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 msgr2=0x558bb115bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb115bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 -- 172.21.15.78:0/3575659507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0da81b0 msgr2=0x558bb115caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0da81b0 0x558bb115caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.562+0000 7f143d78e640 1 -- 172.21.15.78:0/3575659507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb115d0b0 con 0x558bb0d7f2f0 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f143d78e640 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d7f2f0 0x558bb115b680 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1428009f90 tx=0x7f14280093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be640fbcbcfd141e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f14217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f14217fa640 1 -- 172.21.15.78:0/3575659507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142802e060 con 0x558bb0d7f2f0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f14217fa640 1 -- 172.21.15.78:0/3575659507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f14280188a0 con 0x558bb0d7f2f0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1434019ab0 con 0x558bb0d7f2f0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f14217fa640 1 -- 172.21.15.78:0/3575659507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f142801d040 con 0x558bb0d7f2f0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 10 client.17604 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558bb11ccc50 con 0x558bb0d7f2f0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f14217fa640 1 -- 172.21.15.78:0/3575659507 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1434019ab0 con 0x558bb0d7f2f0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f14217fa640 1 -- 172.21.15.78:0/3575659507 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1428010040 con 0x558bb0d7f2f0 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 10 client.17604 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 20 client.17604 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d3c1c0 0x558bb11cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 4 client.17604 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.563+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558bb10b5ee0 con 0x558bb0d3c1c0 2024-04-25T14:06:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.564+0000 7f143df8f640 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d3c1c0 0x558bb11cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.564+0000 7f143df8f640 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d3c1c0 0x558bb11cf420 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3009993fcc32c113 server_cookie=47f517b3b7fb04cb in_seq=0 out_seq=0 2024-04-25T14:06:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.564+0000 7f14217fa640 10 client.17604 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f14217fa640 1 -- 172.21.15.78:0/3575659507 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558bb10b5ee0 con 0x558bb0d3c1c0 2024-04-25T14:06:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f14217fa640 10 client.17604 handle_command_reply: tid=0 2024-04-25T14:06:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 client.17604 shutdown 2024-04-25T14:06:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d7f2f0 msgr2=0x558bb115b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d7f2f0 0x558bb115b680 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1428009f90 tx=0x7f14280093a0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 shutdown_connections 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558bb0da81b0 0x558bb115caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x558bb0d3c1c0 0x558bb11cf420 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558bb0d874d0 0x558bb115bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 --2- 172.21.15.78:0/3575659507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bb0d7f2f0 0x558bb115b680 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f14217fa640 0 client.17604 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 >> 172.21.15.78:0/3575659507 conn(0x558bb0d9c300 msgr2=0x558bb1147100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 shutdown_connections 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 1 -- 172.21.15.78:0/3575659507 wait complete. 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.579+0000 7f1447180740 20 client.17604 trim_cache size 0 max 16384 2024-04-25T14:06:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:06:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:06:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:06:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:06:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f14240afae0 msgr2=0x7f14240b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f14240afae0 0x7f14240b1ff0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f14300c89d0 tx=0x7f14300f7190 comp rx=0 tx=0).stop 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 msgr2=0x7f14381a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f14381a16d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f14300097b0 tx=0x7f1430003c20 comp rx=0 tx=0).stop 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 shutdown_connections 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f14240afae0 0x7f14240b1ff0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1438109a70 0x7f14381a16d0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1438104d70 0x7f143819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 --2- 172.21.15.78:0/3131194653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1438103b30 0x7f143819a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 >> 172.21.15.78:0/3131194653 conn(0x7f14380fed40 msgr2=0x7f1438105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 shutdown_connections 2024-04-25T14:06:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:31.580+0000 7f14401f3640 1 -- 172.21.15.78:0/3131194653 wait complete. 2024-04-25T14:06:31.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:31 smithi129 ceph-mon[32569]: pgmap v558: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:31 smithi078 ceph-mon[29163]: pgmap v558: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:31.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:31 smithi112 ceph-mon[32617]: pgmap v558: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:33.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:33 smithi129 ceph-mon[32569]: pgmap v559: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:33 smithi078 ceph-mon[29163]: pgmap v559: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:33.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:33 smithi112 ceph-mon[32617]: pgmap v559: 97 pgs: 97 active+clean; 894 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:35.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:35 smithi129 ceph-mon[32569]: pgmap v560: 97 pgs: 97 active+clean; 965 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:06:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:35 smithi078 ceph-mon[29163]: pgmap v560: 97 pgs: 97 active+clean; 965 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:06:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:35 smithi112 ceph-mon[32617]: pgmap v560: 97 pgs: 97 active+clean; 965 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:06:37.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:37 smithi129 ceph-mon[32569]: pgmap v561: 97 pgs: 97 active+clean; 965 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:37 smithi078 ceph-mon[29163]: pgmap v561: 97 pgs: 97 active+clean; 965 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:37.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:37 smithi112 ceph-mon[32617]: pgmap v561: 97 pgs: 97 active+clean; 965 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:39.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:39 smithi129 ceph-mon[32569]: pgmap v562: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:06:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:39 smithi078 ceph-mon[29163]: pgmap v562: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:06:39.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:39 smithi112 ceph-mon[32617]: pgmap v562: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:06:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:40 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:40 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:40.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:40 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:06:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: pgmap v563: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:06:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: Unable to set osd_memory_target on smithi129 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:41.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:06:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: pgmap v563: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:06:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: Unable to set osd_memory_target on smithi129 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: pgmap v563: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: Unable to set osd_memory_target on smithi129 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T14:06:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: Unable to set osd_memory_target on smithi112 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:06:41.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:06:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:43 smithi129 ceph-mon[32569]: pgmap v564: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:06:43.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:43 smithi078 ceph-mon[29163]: pgmap v564: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:06:43.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:43 smithi112 ceph-mon[32617]: pgmap v564: 97 pgs: 97 active+clean; 1022 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:06:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 -- 172.21.15.78:0/1405955662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 msgr2=0x7f0850105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1405955662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f0850105190 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f0840009900 tx=0x7f084002a410 comp rx=0 tx=0).stop 2024-04-25T14:06:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 -- 172.21.15.78:0/1405955662 shutdown_connections 2024-04-25T14:06:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1405955662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0850109a80 0x7f085010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1405955662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f0850105190 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1405955662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0850103b40 0x7f0850104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 -- 172.21.15.78:0/1405955662 >> 172.21.15.78:0/1405955662 conn(0x7f08500fed10 msgr2=0x7f08501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 -- 172.21.15.78:0/1405955662 shutdown_connections 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.687+0000 7f0857ee2640 1 -- 172.21.15.78:0/1405955662 wait complete. 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 Processor -- start 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 -- start start 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0850103b40 0x7f085019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f085019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0850109a80 0x7f08501a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f085010f660 con 0x7f0850104d80 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f085010f4e0 con 0x7f0850103b40 2024-04-25T14:06:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f0857ee2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f085010f7e0 con 0x7f0850109a80 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f085019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f085019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52698/0 (socket says 172.21.15.78:52698) 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 -- 172.21.15.78:0/1381608095 learned_addr learned my addr 172.21.15.78:0/1381608095 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085647f640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0850109a80 0x7f08501a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 -- 172.21.15.78:0/1381608095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0850109a80 msgr2=0x7f08501a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0850109a80 0x7f08501a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 -- 172.21.15.78:0/1381608095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0850103b40 msgr2=0x7f085019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0850103b40 0x7f085019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 -- 172.21.15.78:0/1381608095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08501a1da0 con 0x7f0850104d80 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085647f640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0850109a80 0x7f08501a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.688+0000 7f085547d640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f085019a7a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0840006010 tx=0x7f0840004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.689+0000 7f0846ffd640 1 -- 172.21.15.78:0/1381608095 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084003f030 con 0x7f0850104d80 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.689+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08501a2030 con 0x7f0850104d80 2024-04-25T14:06:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.689+0000 7f0846ffd640 1 -- 172.21.15.78:0/1381608095 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0840003cb0 con 0x7f0850104d80 2024-04-25T14:06:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.689+0000 7f0846ffd640 1 -- 172.21.15.78:0/1381608095 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840035070 con 0x7f0850104d80 2024-04-25T14:06:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.689+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08501a2520 con 0x7f0850104d80 2024-04-25T14:06:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.691+0000 7f085ed8c740 1 Processor -- start 2024-04-25T14:06:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f0846ffd640 1 -- 172.21.15.78:0/1381608095 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0840042030 con 0x7f0850104d80 2024-04-25T14:06:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085ed8c740 1 -- start start 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9f91b0 0x557e6ad99970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d02f0 0x557e6ad99ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d84d0 0x557e6ada0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085ed8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557e6acff460 con 0x557e6a9f91b0 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085ed8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e6acb9590 con 0x557e6a9d84d0 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085ed8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557e6acfb420 con 0x557e6a9d02f0 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9f91b0 0x557e6ad99970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9f91b0 0x557e6ad99970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52702/0 (socket says 172.21.15.78:52702) 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 -- 172.21.15.78:0/2693838216 learned_addr learned my addr 172.21.15.78:0/2693838216 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085547d640 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d84d0 0x557e6ada0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f0855c7e640 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d02f0 0x557e6ad99ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 -- 172.21.15.78:0/2693838216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d02f0 msgr2=0x557e6ad99ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d02f0 0x557e6ad99ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 -- 172.21.15.78:0/2693838216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d84d0 msgr2=0x557e6ada0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d84d0 0x557e6ada0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 -- 172.21.15.78:0/2693838216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6acc60d0 con 0x557e6a9f91b0 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f0846ffd640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f083c0afae0 0x7f083c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f0855c7e640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f083c0afae0 0x7f083c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.692+0000 7f085647f640 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9f91b0 0x557e6ad99970 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0848009f10 tx=0x7f084800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81482379492c424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.693+0000 7f0846ffd640 1 -- 172.21.15.78:0/1381608095 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f08400ff370 con 0x7f0850104d80 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.693+0000 7f083bfff640 1 -- 172.21.15.78:0/2693838216 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848016070 con 0x557e6a9f91b0 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.693+0000 7f083bfff640 1 -- 172.21.15.78:0/2693838216 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0848004d20 con 0x557e6a9f91b0 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.693+0000 7f083bfff640 1 -- 172.21.15.78:0/2693838216 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08480117a0 con 0x557e6a9f91b0 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 -- 172.21.15.78:0/2693838216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9f91b0 msgr2=0x557e6ad99970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f0855c7e640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f083c0afae0 0x7f083c0b1ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x557e6ad9ad50 tx=0x7f084c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9f91b0 0x557e6ad99970 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0848009f10 tx=0x7f084800e6c0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 -- 172.21.15.78:0/2693838216 shutdown_connections 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d84d0 0x557e6ada0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d02f0 0x557e6ad99ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 --2- 172.21.15.78:0/2693838216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9f91b0 0x557e6ad99970 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 -- 172.21.15.78:0/2693838216 >> 172.21.15.78:0/2693838216 conn(0x557e6a9ed300 msgr2=0x557e6ad95300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 -- 172.21.15.78:0/2693838216 shutdown_connections 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 -- 172.21.15.78:0/2693838216 wait complete. 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 -- start start 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.694+0000 7f085ed8c740 1 Processor -- start 2024-04-25T14:06:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d02f0 0x557e6ada3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d84d0 0x557e6ada3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9f91b0 0x557e6adaaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557e6ada1840 con 0x557e6a9d02f0 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e6ac86b60 con 0x557e6a9f91b0 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557e6ada19c0 con 0x557e6a9d84d0 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d02f0 0x557e6ada3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 10 client.0 _ll_register_callbacks cb 0x557e6acab2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d02f0 0x557e6ada3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52712/0 (socket says 172.21.15.78:52712) 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 -- 172.21.15.78:0/3621352499 learned_addr learned my addr 172.21.15.78:0/3621352499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085547d640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d84d0 0x557e6ada3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085647f640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9f91b0 0x557e6adaaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 -- 172.21.15.78:0/3621352499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d84d0 msgr2=0x557e6ada3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d84d0 0x557e6ada3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 -- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9f91b0 msgr2=0x557e6adaaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9f91b0 0x557e6adaaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 -- 172.21.15.78:0/3621352499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6aa11210 con 0x557e6a9d02f0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085547d640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d84d0 0x557e6ada3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d02f0 0x557e6ada3700 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f084c00ffd0 tx=0x7f084c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfcd50e181d770f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f08397fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f08397fa640 1 -- 172.21.15.78:0/3621352499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c022070 con 0x557e6a9d02f0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f08397fa640 1 -- 172.21.15.78:0/3621352499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f084c01e860 con 0x557e6a9d02f0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.695+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0848004d20 con 0x557e6a9d02f0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f08397fa640 1 -- 172.21.15.78:0/3621352499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f084c02e050 con 0x557e6a9d02f0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 10 client.17616 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557e6aadd620 con 0x557e6a9d02f0 2024-04-25T14:06:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f08397fa640 1 -- 172.21.15.78:0/3621352499 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0848004d20 con 0x557e6a9d02f0 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f08397fa640 1 -- 172.21.15.78:0/3621352499 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f084c010a40 con 0x557e6a9d02f0 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 10 client.17616 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 20 client.17616 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a9b7d80 0x557e6adb1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 4 client.17616 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e6ad0d070 con 0x557e6a9b7d80 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.696+0000 7f085547d640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a9b7d80 0x557e6adb1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.697+0000 7f08397fa640 10 client.17616 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.697+0000 7f085547d640 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a9b7d80 0x557e6adb1430 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b64b3b25323d22 server_cookie=647c703d396957f3 in_seq=0 out_seq=0 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f08397fa640 1 -- 172.21.15.78:0/3621352499 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e6ad0d070 con 0x557e6a9b7d80 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f08397fa640 10 client.17616 handle_command_reply: tid=0 2024-04-25T14:06:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 client.17616 shutdown 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d02f0 msgr2=0x557e6ada3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d02f0 0x557e6ada3700 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f084c00ffd0 tx=0x7f084c005c90 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 shutdown_connections 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9f91b0 0x557e6adaaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a9b7d80 0x557e6adb1430 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9d84d0 0x557e6ada3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3621352499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d02f0 0x557e6ada3700 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f08397fa640 0 client.17616 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 >> 172.21.15.78:0/3621352499 conn(0x557e6a9ed300 msgr2=0x557e6ad98210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 shutdown_connections 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 1 -- 172.21.15.78:0/3621352499 wait complete. 2024-04-25T14:06:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.698+0000 7f085ed8c740 20 client.17616 trim_cache size 0 max 16384 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 Processor -- start 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 -- start start 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 0x557e6adb6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08501adc00 con 0x557e6a9d84d0 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08501ab4d0 con 0x557e6a9d02f0 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085ed8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08501a9320 con 0x557e6a9f91b0 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085547d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085547d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52718/0 (socket says 172.21.15.78:52718) 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085547d640 1 -- 172.21.15.78:0/3793019740 learned_addr learned my addr 172.21.15.78:0/3793019740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.713+0000 7f085547d640 1 -- 172.21.15.78:0/3793019740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 msgr2=0x557e6adb6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f085547d640 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 0x557e6adb6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f085547d640 1 -- 172.21.15.78:0/3793019740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 msgr2=0x557e6ada2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f085547d640 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f085547d640 1 -- 172.21.15.78:0/3793019740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6ada1ac0 con 0x557e6a9d84d0 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f085547d640 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaab30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f08400c1fd0 tx=0x7f0840101880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124f1b2ac5c52fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f083b7fe640 1 -- 172.21.15.78:0/3793019740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840049040 con 0x557e6a9d84d0 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f083b7fe640 1 -- 172.21.15.78:0/3793019740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0840003e60 con 0x557e6a9d84d0 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.714+0000 7f083b7fe640 1 -- 172.21.15.78:0/3793019740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08400c7840 con 0x557e6a9d84d0 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- 172.21.15.78:0/3793019740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 msgr2=0x557e6adaab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaab30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f08400c1fd0 tx=0x7f0840101880 comp rx=0 tx=0).stop 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- 172.21.15.78:0/3793019740 shutdown_connections 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 0x557e6adb6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaab30 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3793019740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- 172.21.15.78:0/3793019740 >> 172.21.15.78:0/3793019740 conn(0x557e6a9ed300 msgr2=0x557e6ada1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- 172.21.15.78:0/3793019740 shutdown_connections 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- 172.21.15.78:0/3793019740 wait complete. 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- start start 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 Processor -- start 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 0x557e6ae17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08501a9990 con 0x557e6a9d84d0 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08501aae00 con 0x557e6a9d02f0 2024-04-25T14:06:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.715+0000 7f085ed8c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08501aa060 con 0x557e6a9f91b0 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085ed8c740 10 client.0 _ll_register_callbacks cb 0x7f084c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52734/0 (socket says 172.21.15.78:52734) 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 -- 172.21.15.78:0/3677350736 learned_addr learned my addr 172.21.15.78:0/3677350736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 -- 172.21.15.78:0/3677350736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 msgr2=0x557e6ae17a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085647f640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 0x557e6ae17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 0x557e6ae17a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 -- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 msgr2=0x557e6ada9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 -- 172.21.15.78:0/3677350736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6ae18000 con 0x557e6a9d84d0 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085547d640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaa430 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f08400fceb0 tx=0x7f084003b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f78de9732fe551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f08397fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f08397fa640 1 -- 172.21.15.78:0/3677350736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840049040 con 0x557e6a9d84d0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f08397fa640 1 -- 172.21.15.78:0/3677350736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f08400c3ad0 con 0x557e6a9d84d0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e6ad06ee0 con 0x557e6a9d84d0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f08397fa640 1 -- 172.21.15.78:0/3677350736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f08400cbbf0 con 0x557e6a9d84d0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085ed8c740 10 client.17622 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557e6ae192c0 con 0x557e6a9d84d0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.716+0000 7f08397fa640 1 -- 172.21.15.78:0/3677350736 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e6ad06ee0 con 0x557e6a9d84d0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f08397fa640 1 -- 172.21.15.78:0/3677350736 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f08400aece0 con 0x557e6a9d84d0 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f085ed8c740 10 client.17622 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f085ed8c740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f085ed8c740 20 client.17622 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a98d1c0 0x557e6ae1bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f085ed8c740 4 client.17622 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e6ac825f0 con 0x557e6a98d1c0 2024-04-25T14:06:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a98d1c0 0x557e6ae1bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f08397fa640 10 client.17622 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.717+0000 7f0855c7e640 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a98d1c0 0x557e6ae1bc90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8dbc60f65e3f9ca server_cookie=a237d598f6737b2a in_seq=0 out_seq=0 2024-04-25T14:06:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f08397fa640 1 -- 172.21.15.78:0/3677350736 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e6ac825f0 con 0x557e6a98d1c0 2024-04-25T14:06:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f08397fa640 10 client.17622 handle_command_reply: tid=0 2024-04-25T14:06:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 client.17622 shutdown 2024-04-25T14:06:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 msgr2=0x557e6adaa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaa430 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f08400fceb0 tx=0x7f084003b640 comp rx=0 tx=0).stop 2024-04-25T14:06:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 shutdown_connections 2024-04-25T14:06:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557e6a98d1c0 0x557e6ae1bc90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e6a9f91b0 0x557e6ae17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f08397fa640 0 client.17622 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557e6a9d84d0 0x557e6adaa430 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 --2- 172.21.15.78:0/3677350736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e6a9d02f0 0x557e6ada9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 >> 172.21.15.78:0/3677350736 conn(0x557e6a9ed300 msgr2=0x557e6ada32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 shutdown_connections 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 1 -- 172.21.15.78:0/3677350736 wait complete. 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.732+0000 7f085ed8c740 20 client.17622 trim_cache size 0 max 16384 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:06:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:06:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:06:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f083c0afae0 msgr2=0x7f083c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f083c0afae0 0x7f083c0b1ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x557e6ad9ad50 tx=0x7f084c005f50 comp rx=0 tx=0).stop 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 msgr2=0x7f085019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f085019a7a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0840006010 tx=0x7f0840004340 comp rx=0 tx=0).stop 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 shutdown_connections 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0850109a80 0x7f08501a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f083c0afae0 0x7f083c0b1ff0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850104d80 0x7f085019a7a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 --2- 172.21.15.78:0/1381608095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0850103b40 0x7f085019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 >> 172.21.15.78:0/1381608095 conn(0x7f08500fed10 msgr2=0x7f0850105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 shutdown_connections 2024-04-25T14:06:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:44.733+0000 7f0857ee2640 1 -- 172.21.15.78:0/1381608095 wait complete. 2024-04-25T14:06:45.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:45 smithi129 ceph-mon[32569]: pgmap v565: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T14:06:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:45 smithi078 ceph-mon[29163]: pgmap v565: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T14:06:45.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:45 smithi112 ceph-mon[32617]: pgmap v565: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T14:06:47.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:47 smithi129 ceph-mon[32569]: pgmap v566: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:06:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:47 smithi078 ceph-mon[29163]: pgmap v566: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:06:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:47 smithi112 ceph-mon[32617]: pgmap v566: 97 pgs: 97 active+clean; 1.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:06:49.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:49 smithi129 ceph-mon[32569]: pgmap v567: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:06:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:49 smithi078 ceph-mon[29163]: pgmap v567: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:06:49.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:49 smithi112 ceph-mon[32617]: pgmap v567: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:06:50.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:06:50.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:06:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 -- 172.21.15.78:0/2327004325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0106df0 msgr2=0x7fe5e0109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/2327004325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0106df0 0x7fe5e0109210 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009900 tx=0x7fe5cc02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 -- 172.21.15.78:0/2327004325 shutdown_connections 2024-04-25T14:06:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/2327004325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e01097a0 0x7fe5e010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/2327004325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e0106df0 0x7fe5e0109210 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/2327004325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5e0101830 0x7fe5e0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 -- 172.21.15.78:0/2327004325 >> 172.21.15.78:0/2327004325 conn(0x7fe5e00fcf20 msgr2=0x7fe5e00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 -- 172.21.15.78:0/2327004325 shutdown_connections 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 -- 172.21.15.78:0/2327004325 wait complete. 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.565+0000 7fe5e6b64640 1 Processor -- start 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e6b64640 1 -- start start 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e6b64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5e0101830 0x7fe5e019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e6b64640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e0106df0 0x7fe5e019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e6b64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e01097a0 0x7fe5e01a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e6b64640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010f620 con 0x7fe5e01097a0 2024-04-25T14:06:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e6b64640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010f4a0 con 0x7fe5e0101830 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e6b64640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010f7a0 con 0x7fe5e0106df0 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e01097a0 0x7fe5e01a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e01097a0 0x7fe5e01a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55178/0 (socket says 172.21.15.78:55178) 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 -- 172.21.15.78:0/445020637 learned_addr learned my addr 172.21.15.78:0/445020637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e4900640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5e0101830 0x7fe5e019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5dffff640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e0106df0 0x7fe5e019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 -- 172.21.15.78:0/445020637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e0106df0 msgr2=0x7fe5e019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e0106df0 0x7fe5e019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 -- 172.21.15.78:0/445020637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5e0101830 msgr2=0x7fe5e019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5e0101830 0x7fe5e019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 -- 172.21.15.78:0/445020637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e01a1da0 con 0x7fe5e01097a0 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e4900640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5e0101830 0x7fe5e019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5e5101640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e01097a0 0x7fe5e01a1730 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800eaa0 tx=0x7fe5d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5ddffb640 1 -- 172.21.15.78:0/445020637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800cdb0 con 0x7fe5e01097a0 2024-04-25T14:06:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.566+0000 7fe5ddffb640 1 -- 172.21.15.78:0/445020637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d8005020 con 0x7fe5e01097a0 2024-04-25T14:06:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.567+0000 7fe5ddffb640 1 -- 172.21.15.78:0/445020637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80116d0 con 0x7fe5e01097a0 2024-04-25T14:06:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.567+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e01a2090 con 0x7fe5e01097a0 2024-04-25T14:06:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.567+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01a2630 con 0x7fe5e01097a0 2024-04-25T14:06:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.569+0000 7fe5ddffb640 1 -- 172.21.15.78:0/445020637 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe5d80051d0 con 0x7fe5e01097a0 2024-04-25T14:06:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.569+0000 7fe5eda12740 1 Processor -- start 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.569+0000 7fe5eda12740 1 -- start start 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.569+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 0x56279a0f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.569+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a0f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5eda12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56279a05dd80 con 0x562799d366b0 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5eda12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56279a0178b0 con 0x562799d2e4d0 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5eda12740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56279a059d40 con 0x562799d57390 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5ddffb640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe5d00afb70 0x7fe5d00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a0f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a0f9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55182/0 (socket says 172.21.15.78:55182) 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 -- 172.21.15.78:0/3568742978 learned_addr learned my addr 172.21.15.78:0/3568742978 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5e5101640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe5d00afb70 0x7fe5d00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5e4900640 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5ddffb640 1 -- 172.21.15.78:0/445020637 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe5d80dee10 con 0x7fe5e01097a0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 -- 172.21.15.78:0/3568742978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 msgr2=0x56279a0f8a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 0x56279a0f8a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 -- 172.21.15.78:0/3568742978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 msgr2=0x56279a0f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 -- 172.21.15.78:0/3568742978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562799fe8a40 con 0x562799d366b0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5e4900640 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5dffff640 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a0f9c70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005940 tx=0x7fe5cc032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef52206e37164b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5e5101640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe5d00afb70 0x7fe5d00b2080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80a4550 tx=0x7fe5d809d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5caffd640 1 -- 172.21.15.78:0/3568742978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc035070 con 0x562799d366b0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5caffd640 1 -- 172.21.15.78:0/3568742978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5cc004030 con 0x562799d366b0 2024-04-25T14:06:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.570+0000 7fe5caffd640 1 -- 172.21.15.78:0/3568742978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc005520 con 0x562799d366b0 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 -- 172.21.15.78:0/3568742978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 msgr2=0x56279a0f9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a0f9c70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005940 tx=0x7fe5cc032b20 comp rx=0 tx=0).stop 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 -- 172.21.15.78:0/3568742978 shutdown_connections 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a0f9c70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3568742978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 0x56279a0f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 -- 172.21.15.78:0/3568742978 >> 172.21.15.78:0/3568742978 conn(0x562799d4b4e0 msgr2=0x56279a0f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 -- 172.21.15.78:0/3568742978 shutdown_connections 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.571+0000 7fe5eda12740 1 -- 172.21.15.78:0/3568742978 wait complete. 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 -- start start 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 Processor -- start 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a102940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a102e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d57390 0x56279a109de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56279a0fa720 con 0x562799d57390 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562799fe4d10 con 0x562799d2e4d0 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56279a0fa8a0 con 0x562799d366b0 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5eda12740 10 client.0 _ll_register_callbacks cb 0x56279a011210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d57390 0x56279a109de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d57390 0x56279a109de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55184/0 (socket says 172.21.15.78:55184) 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5dffff640 1 -- 172.21.15.78:0/3555564007 learned_addr learned my addr 172.21.15.78:0/3555564007 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5dffff640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.572+0000 7fe5e4900640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5dffff640 1 -- 172.21.15.78:0/3555564007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 msgr2=0x56279a102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5dffff640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5dffff640 1 -- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 msgr2=0x56279a102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5dffff640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5dffff640 1 -- 172.21.15.78:0/3555564007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562799e397d0 con 0x562799d57390 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5e4900640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5dffff640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d57390 0x56279a109de0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc000c00 tx=0x7fe5cc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bc7a1688f7907b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5bffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5bffff640 1 -- 172.21.15.78:0/3555564007 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc047030 con 0x562799d57390 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5bffff640 1 -- 172.21.15.78:0/3555564007 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe5cc0317b0 con 0x562799d57390 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5cc004030 con 0x562799d57390 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5bffff640 1 -- 172.21.15.78:0/3555564007 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe5cc059050 con 0x562799d57390 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5eda12740 10 client.17634 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562799d6f3f0 con 0x562799d57390 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5bffff640 1 -- 172.21.15.78:0/3555564007 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5cc004030 con 0x562799d57390 2024-04-25T14:06:50.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.573+0000 7fe5bffff640 1 -- 172.21.15.78:0/3555564007 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe5cc051020 con 0x562799d57390 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5eda12740 10 client.17634 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5eda12740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5eda12740 20 client.17634 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799d15f60 0x56279a10fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5eda12740 4 client.17634 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56279a06b990 con 0x562799d15f60 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5e4900640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799d15f60 0x56279a10fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5bffff640 10 client.17634 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.574+0000 7fe5e4900640 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799d15f60 0x56279a10fea0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30547431afdf9fae server_cookie=224b5ebefb45c6b0 in_seq=0 out_seq=0 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5bffff640 1 -- 172.21.15.78:0/3555564007 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56279a06b990 con 0x562799d15f60 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5bffff640 10 client.17634 handle_command_reply: tid=0 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 client.17634 shutdown 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d57390 msgr2=0x56279a109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d57390 0x56279a109de0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc000c00 tx=0x7fe5cc005040 comp rx=0 tx=0).stop 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 shutdown_connections 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d57390 0x56279a109de0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799d15f60 0x56279a10fea0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 --2- 172.21.15.78:0/3555564007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5bffff640 0 client.17634 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 >> 172.21.15.78:0/3555564007 conn(0x562799d4b4e0 msgr2=0x56279a0f7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 shutdown_connections 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 1 -- 172.21.15.78:0/3555564007 wait complete. 2024-04-25T14:06:50.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.575+0000 7fe5eda12740 20 client.17634 trim_cache size 0 max 16384 2024-04-25T14:06:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 Processor -- start 2024-04-25T14:06:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 -- start start 2024-04-25T14:06:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a109d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 0x56279a0f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01adb90 con 0x562799d366b0 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ab460 con 0x562799d2e4d0 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5eda12740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01a92b0 con 0x562799d57390 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55200/0 (socket says 172.21.15.78:55200) 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 -- 172.21.15.78:0/106205521 learned_addr learned my addr 172.21.15.78:0/106205521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5dffff640 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 -- 172.21.15.78:0/106205521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 msgr2=0x56279a0f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 0x56279a0f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 -- 172.21.15.78:0/106205521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 msgr2=0x56279a0f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 -- 172.21.15.78:0/106205521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562799d2f0f0 con 0x562799d366b0 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.593+0000 7fe5e4900640 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a109d00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe5e01028d0 tx=0x7fe5d400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b1d8685129d6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5ca7fc640 1 -- 172.21.15.78:0/106205521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40042f0 con 0x562799d366b0 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5ca7fc640 1 -- 172.21.15.78:0/106205521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d4020070 con 0x562799d366b0 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5ca7fc640 1 -- 172.21.15.78:0/106205521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4004db0 con 0x562799d366b0 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 -- 172.21.15.78:0/106205521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 msgr2=0x56279a109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a109d00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe5e01028d0 tx=0x7fe5d400b850 comp rx=0 tx=0).stop 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 -- 172.21.15.78:0/106205521 shutdown_connections 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d57390 0x56279a0f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d366b0 0x56279a109d00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 --2- 172.21.15.78:0/106205521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d2e4d0 0x56279a0f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 -- 172.21.15.78:0/106205521 >> 172.21.15.78:0/106205521 conn(0x562799d4b4e0 msgr2=0x56279a0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 -- 172.21.15.78:0/106205521 shutdown_connections 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.594+0000 7fe5eda12740 1 -- 172.21.15.78:0/106205521 wait complete. 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 -- start start 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 Processor -- start 2024-04-25T14:06:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d2e4d0 0x56279a106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a107350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d57390 0x56279a108280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01a9920 con 0x562799d2e4d0 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01aad90 con 0x562799d57390 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01a9ff0 con 0x562799d366b0 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5eda12740 10 client.0 _ll_register_callbacks cb 0x7fe5cc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d2e4d0 0x56279a106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d2e4d0 0x56279a106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55210/0 (socket says 172.21.15.78:55210) 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5dffff640 1 -- 172.21.15.78:0/340356929 learned_addr learned my addr 172.21.15.78:0/340356929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5dffff640 1 -- 172.21.15.78:0/340356929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 msgr2=0x56279a107350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5dffff640 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5dffff640 1 -- 172.21.15.78:0/340356929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d57390 msgr2=0x56279a108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.595+0000 7fe5dffff640 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d57390 0x56279a108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5dffff640 1 -- 172.21.15.78:0/340356929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279a01e4b0 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5dffff640 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d2e4d0 0x56279a106e10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc02a930 tx=0x7fe5cc02ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8ba75376feda0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5bffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5bffff640 1 -- 172.21.15.78:0/340356929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc047040 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5bffff640 1 -- 172.21.15.78:0/340356929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe5cc04e040 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d4020070 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5bffff640 1 -- 172.21.15.78:0/340356929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe5cc035040 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 10 client.17640 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56279a109930 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5bffff640 1 -- 172.21.15.78:0/340356929 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d4020070 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5bffff640 1 -- 172.21.15.78:0/340356929 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe5cc051070 con 0x562799d2e4d0 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 10 client.17640 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 20 client.17640 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799ceb3a0 0x56279a17a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 4 client.17640 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.596+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56279a065800 con 0x562799ceb3a0 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5e4900640 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799ceb3a0 0x56279a17a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5e4900640 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799ceb3a0 0x56279a17a170 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d082d692b9a26437 server_cookie=4ab60ddec1ba310c in_seq=0 out_seq=0 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5bffff640 10 client.17640 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5bffff640 1 -- 172.21.15.78:0/340356929 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56279a065800 con 0x562799ceb3a0 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5bffff640 10 client.17640 handle_command_reply: tid=0 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 client.17640 shutdown 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d2e4d0 msgr2=0x56279a106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d2e4d0 0x56279a106e10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc02a930 tx=0x7fe5cc02ab00 comp rx=0 tx=0).stop 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 shutdown_connections 2024-04-25T14:06:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562799d57390 0x56279a108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562799ceb3a0 0x56279a17a170 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562799d366b0 0x56279a107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 --2- 172.21.15.78:0/340356929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562799d2e4d0 0x56279a106e10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5bffff640 0 client.17640 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 >> 172.21.15.78:0/340356929 conn(0x562799d4b4e0 msgr2=0x56279a0f3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.597+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 shutdown_connections 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5eda12740 1 -- 172.21.15.78:0/340356929 wait complete. 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5eda12740 20 client.17640 trim_cache size 0 max 16384 2024-04-25T14:06:50.604 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:06:50.605 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:06:50.605 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:06:50.605 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:06:50.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe5d00afb70 msgr2=0x7fe5d00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe5d00afb70 0x7fe5d00b2080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80a4550 tx=0x7fe5d809d040 comp rx=0 tx=0).stop 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e01097a0 msgr2=0x7fe5e01a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e01097a0 0x7fe5e01a1730 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800eaa0 tx=0x7fe5d800ef60 comp rx=0 tx=0).stop 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 shutdown_connections 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe5d00afb70 0x7fe5d00b2080 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e01097a0 0x7fe5e01a1730 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e0106df0 0x7fe5e019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 --2- 172.21.15.78:0/445020637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5e0101830 0x7fe5e019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 >> 172.21.15.78:0/445020637 conn(0x7fe5e00fcf20 msgr2=0x7fe5e0107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 shutdown_connections 2024-04-25T14:06:50.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.598+0000 7fe5e6b64640 1 -- 172.21.15.78:0/445020637 wait complete. 2024-04-25T14:06:50.609 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:06:50.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:06:50.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 -- 172.21.15.78:0/2203496781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0102f50 msgr2=0x7fe8b0103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 --2- 172.21.15.78:0/2203496781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0102f50 0x7fe8b0103360 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009ca0 tx=0x7fe8ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 -- 172.21.15.78:0/2203496781 shutdown_connections 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 --2- 172.21.15.78:0/2203496781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0104450 0x7fe8b010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 --2- 172.21.15.78:0/2203496781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b0103930 0x7fe8b0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 --2- 172.21.15.78:0/2203496781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0102f50 0x7fe8b0103360 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 -- 172.21.15.78:0/2203496781 >> 172.21.15.78:0/2203496781 conn(0x7fe8b00fd330 msgr2=0x7fe8b00ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 -- 172.21.15.78:0/2203496781 shutdown_connections 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.717+0000 7fe8b8267640 1 -- 172.21.15.78:0/2203496781 wait complete. 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 Processor -- start 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 -- start start 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b0102f50 0x7fe8b01021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0103930 0x7fe8b019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0104450 0x7fe8b01a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b010e520 con 0x7fe8b0103930 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b010e3a0 con 0x7fe8b0102f50 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b8267640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b010e6a0 con 0x7fe8b0104450 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b5802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0103930 0x7fe8b019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b5802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0103930 0x7fe8b019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55232/0 (socket says 172.21.15.78:55232) 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.718+0000 7fe8b5802640 1 -- 172.21.15.78:0/3451372052 learned_addr learned my addr 172.21.15.78:0/3451372052 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b6003640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b0102f50 0x7fe8b01021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b6804640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0104450 0x7fe8b01a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b5802640 1 -- 172.21.15.78:0/3451372052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0104450 msgr2=0x7fe8b01a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b5802640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0104450 0x7fe8b01a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b5802640 1 -- 172.21.15.78:0/3451372052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b0102f50 msgr2=0x7fe8b01021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b5802640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b0102f50 0x7fe8b01021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b5802640 1 -- 172.21.15.78:0/3451372052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b01a1d10 con 0x7fe8b0103930 2024-04-25T14:06:50.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b5802640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0103930 0x7fe8b019efa0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e9f0 tx=0x7fe8a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8a77fe640 1 -- 172.21.15.78:0/3451372052 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000cd80 con 0x7fe8b0103930 2024-04-25T14:06:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b01a1fa0 con 0x7fe8b0103930 2024-04-25T14:06:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8a77fe640 1 -- 172.21.15.78:0/3451372052 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a00108e0 con 0x7fe8b0103930 2024-04-25T14:06:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8a77fe640 1 -- 172.21.15.78:0/3451372052 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0018980 con 0x7fe8b0103930 2024-04-25T14:06:50.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.719+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b01a2490 con 0x7fe8b0103930 2024-04-25T14:06:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.720+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b0069310 con 0x7fe8b0103930 2024-04-25T14:06:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.722+0000 7fe8a77fe640 1 -- 172.21.15.78:0/3451372052 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe8a0010430 con 0x7fe8b0103930 2024-04-25T14:06:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.722+0000 7fe8a77fe640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe89c0afaf0 0x7fe89c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.723+0000 7fe8b6003640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe89c0afaf0 0x7fe89c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.723+0000 7fe8a77fe640 1 -- 172.21.15.78:0/3451372052 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe8a00df490 con 0x7fe8b0103930 2024-04-25T14:06:50.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.723+0000 7fe8b6003640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe89c0afaf0 0x7fe89c0b2000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac005e30 tx=0x7fe8ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:50.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.726+0000 7fe8a77fe640 1 -- 172.21.15.78:0/3451372052 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8a00a8060 con 0x7fe8b0103930 2024-04-25T14:06:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.939+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8b01078f0 con 0x7fe8b0103930 2024-04-25T14:06:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.940+0000 7fe8a77fe640 1 -- 172.21.15.78:0/3451372052 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fe8a00b00e0 con 0x7fe8b0103930 2024-04-25T14:06:50.940 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:06:50.940 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:06:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:06:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe89c0afaf0 msgr2=0x7fe89c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe89c0afaf0 0x7fe89c0b2000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac005e30 tx=0x7fe8ac005fd0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0103930 msgr2=0x7fe8b019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0103930 0x7fe8b019efa0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e9f0 tx=0x7fe8a000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 shutdown_connections 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8b0104450 0x7fe8b01a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe89c0afaf0 0x7fe89c0b2000 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8b0103930 0x7fe8b019efa0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 --2- 172.21.15.78:0/3451372052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b0102f50 0x7fe8b01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 >> 172.21.15.78:0/3451372052 conn(0x7fe8b00fd330 msgr2=0x7fe8b0104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 shutdown_connections 2024-04-25T14:06:50.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:50.942+0000 7fe8b8267640 1 -- 172.21.15.78:0/3451372052 wait complete. 2024-04-25T14:06:50.953 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:06:51.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/1941558489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c108930 msgr2=0x7fbf9c10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/1941558489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c108930 0x7fbf9c10ad50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009900 tx=0x7fbf9402a410 comp rx=0 tx=0).stop 2024-04-25T14:06:51.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/1941558489 shutdown_connections 2024-04-25T14:06:51.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/1941558489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c108930 0x7fbf9c10ad50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/1941558489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9c103c30 0x7fbf9c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/1941558489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c1029f0 0x7fbf9c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/1941558489 >> 172.21.15.78:0/1941558489 conn(0x7fbf9c0fdbe0 msgr2=0x7fbf9c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/1941558489 shutdown_connections 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.064+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/1941558489 wait complete. 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 Processor -- start 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 -- start start 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c1029f0 0x7fbf9c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9c103c30 0x7fbf9c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c108930 0x7fbf9c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c10e520 con 0x7fbf9c1029f0 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c10e3a0 con 0x7fbf9c103c30 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbfa0ee0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c10e6a0 con 0x7fbf9c108930 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c1029f0 0x7fbf9c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c1029f0 0x7fbf9c199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55254/0 (socket says 172.21.15.78:55254) 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 -- 172.21.15.78:0/2855483950 learned_addr learned my addr 172.21.15.78:0/2855483950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c108930 0x7fbf9c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 -- 172.21.15.78:0/2855483950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c108930 msgr2=0x7fbf9c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c108930 0x7fbf9c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf99d9b640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9c103c30 0x7fbf9c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 -- 172.21.15.78:0/2855483950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9c103c30 msgr2=0x7fbf9c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9c103c30 0x7fbf9c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 -- 172.21.15.78:0/2855483950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c1a0ca0 con 0x7fbf9c1029f0 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c108930 0x7fbf9c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf99d9b640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9c103c30 0x7fbf9c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.065+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c1029f0 0x7fbf9c199160 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00cce0 tx=0x7fbf8c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.066+0000 7fbf8b7fe640 1 -- 172.21.15.78:0/2855483950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c004b20 con 0x7fbf9c1029f0 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.066+0000 7fbf8b7fe640 1 -- 172.21.15.78:0/2855483950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c003e40 con 0x7fbf9c1029f0 2024-04-25T14:06:51.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.066+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c1a0f90 con 0x7fbf9c1029f0 2024-04-25T14:06:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.066+0000 7fbf8b7fe640 1 -- 172.21.15.78:0/2855483950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0047b0 con 0x7fbf9c1029f0 2024-04-25T14:06:51.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.066+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c1a13f0 con 0x7fbf9c1029f0 2024-04-25T14:06:51.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.068+0000 7fbf8b7fe640 1 -- 172.21.15.78:0/2855483950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbf8c003990 con 0x7fbf9c1029f0 2024-04-25T14:06:51.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.068+0000 7fbfa7d8a740 1 Processor -- start 2024-04-25T14:06:51.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.068+0000 7fbfa7d8a740 1 -- start start 2024-04-25T14:06:51.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d86a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d87c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc64ce8d80 con 0x55bc649b94d0 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc64ca2a20 con 0x55bc649e2390 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bc64ce4d40 con 0x55bc649c16b0 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf8b7fe640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbf840afb70 0x7fbf840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbf840afb70 0x7fbf840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d86fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55258/0 (socket says 172.21.15.78:55258) 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3447486006 learned_addr learned my addr 172.21.15.78:0/3447486006 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d86a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf8b7fe640 1 -- 172.21.15.78:0/2855483950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbf8c0decb0 con 0x7fbf9c1029f0 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf99d9b640 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d87c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3447486006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 msgr2=0x55bc64d87c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d87c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3447486006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 msgr2=0x55bc64d86a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d86a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3447486006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc64cadc30 con 0x55bc649b94d0 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbf840afb70 0x7fbf840b2080 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbf9402a930 tx=0x7fbf94005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.069+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d86fa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0a7110 tx=0x7fbf8c075cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f6e41c981ace7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.070+0000 7fbf88ff9640 1 -- 172.21.15.78:0/3447486006 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c01d030 con 0x55bc649b94d0 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.070+0000 7fbf88ff9640 1 -- 172.21.15.78:0/3447486006 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c004020 con 0x55bc649b94d0 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.070+0000 7fbf88ff9640 1 -- 172.21.15.78:0/3447486006 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0ab530 con 0x55bc649b94d0 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.070+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3447486006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 msgr2=0x55bc64d86fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.070+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d86fa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0a7110 tx=0x7fbf8c075cd0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.070+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3447486006 shutdown_connections 2024-04-25T14:06:51.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d87c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d86fa0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3447486006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d86a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3447486006 >> 172.21.15.78:0/3447486006 conn(0x55bc649d64e0 msgr2=0x55bc64d80020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3447486006 shutdown_connections 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3447486006 wait complete. 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 -- start start 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 Processor -- start 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d92a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d92f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc64d88740 con 0x55bc649b94d0 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc64c6fdc0 con 0x55bc649e2390 2024-04-25T14:06:51.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bc64d888c0 con 0x55bc649c16b0 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbfa7d8a740 10 client.0 _ll_register_callbacks cb 0x55bc64c8b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d92a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d92a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55274/0 (socket says 172.21.15.78:55274) 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.071+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3802001757 learned_addr learned my addr 172.21.15.78:0/3802001757 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf99d9b640 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d92f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 msgr2=0x55bc64d92f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d92f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3802001757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 msgr2=0x55bc64d99e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d99e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3802001757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc649fa3f0 con 0x55bc649b94d0 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d92a10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c08fe40 tx=0x7fbf8c0a3640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d467960467d4024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf81ffb640 1 -- 172.21.15.78:0/3802001757 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c01d040 con 0x55bc649b94d0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf81ffb640 1 -- 172.21.15.78:0/3802001757 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbf8c0a3770 con 0x55bc649b94d0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc64cf6990 con 0x55bc649b94d0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf81ffb640 1 -- 172.21.15.78:0/3802001757 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbf8c0affe0 con 0x55bc649b94d0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbfa7d8a740 10 client.17658 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bc64ac47d0 con 0x55bc649b94d0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf81ffb640 1 -- 172.21.15.78:0/3802001757 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bc64cf6990 con 0x55bc649b94d0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbf81ffb640 1 -- 172.21.15.78:0/3802001757 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbf8c0a2af0 con 0x55bc649b94d0 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbfa7d8a740 10 client.17658 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbfa7d8a740 10 client.17658 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.072+0000 7fbfa7d8a740 20 client.17658 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.073+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649a0f60 0x55bc64da0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.073+0000 7fbfa7d8a740 4 client.17658 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.073+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc64cf0800 con 0x55bc649a0f60 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.073+0000 7fbf99d9b640 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649a0f60 0x55bc64da0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.073+0000 7fbf99d9b640 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649a0f60 0x55bc64da0290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18dcd88db2a79e34 server_cookie=8af4c458eb3ca6ce in_seq=0 out_seq=0 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.073+0000 7fbf81ffb640 10 client.17658 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbf81ffb640 1 -- 172.21.15.78:0/3802001757 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc64cf0800 con 0x55bc649a0f60 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbf81ffb640 10 client.17658 handle_command_reply: tid=0 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 client.17658 shutdown 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 msgr2=0x55bc64d92a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d92a10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c08fe40 tx=0x7fbf8c0a3640 comp rx=0 tx=0).stop 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 shutdown_connections 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649a0f60 0x55bc64da0290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649e2390 0x55bc64d99e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbf81ffb640 0 client.17658 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d92f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3802001757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649b94d0 0x55bc64d92a10 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 >> 172.21.15.78:0/3802001757 conn(0x55bc649d64e0 msgr2=0x55bc64d7e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 shutdown_connections 2024-04-25T14:06:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3802001757 wait complete. 2024-04-25T14:06:51.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.074+0000 7fbfa7d8a740 20 client.17658 trim_cache size 0 max 16384 2024-04-25T14:06:51.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 Processor -- start 2024-04-25T14:06:51.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 -- start start 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649a0f60 0x55bc64d81e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649b94d0 0x55bc64d925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d92b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1aba40 con 0x55bc649a0f60 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a9310 con 0x55bc649b94d0 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a7160 con 0x55bc649c16b0 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649a0f60 0x55bc64d81e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649a0f60 0x55bc64d81e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55280/0 (socket says 172.21.15.78:55280) 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3353609558 learned_addr learned my addr 172.21.15.78:0/3353609558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3353609558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 msgr2=0x55bc64d92b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3353609558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d92b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3353609558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649b94d0 msgr2=0x55bc64d925e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3353609558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649b94d0 0x55bc64d925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 -- 172.21.15.78:0/3353609558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc64d83390 con 0x55bc649a0f60 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.088+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/3353609558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649a0f60 0x55bc64d81e20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0a7a30 tx=0x7fbf8c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2d4351da891a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbf83fff640 1 -- 172.21.15.78:0/3353609558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c01d070 con 0x55bc649a0f60 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbf83fff640 1 -- 172.21.15.78:0/3353609558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c0a7180 con 0x55bc649a0f60 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbf83fff640 1 -- 172.21.15.78:0/3353609558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0a2af0 con 0x55bc649a0f60 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3353609558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649a0f60 msgr2=0x55bc64d81e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3353609558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649a0f60 0x55bc64d81e20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0a7a30 tx=0x7fbf8c007f60 comp rx=0 tx=0).stop 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3353609558 shutdown_connections 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3353609558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649c16b0 0x55bc64d92b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3353609558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649b94d0 0x55bc64d925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/3353609558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649a0f60 0x55bc64d81e20 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3353609558 >> 172.21.15.78:0/3353609558 conn(0x55bc649d64e0 msgr2=0x55bc64d91030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3353609558 shutdown_connections 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.089+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/3353609558 wait complete. 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 -- start start 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 Processor -- start 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649a0f60 0x55bc64d96e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649b94d0 0x55bc64d97360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649c16b0 0x55bc64d98240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a77d0 con 0x55bc649c16b0 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a8c40 con 0x55bc649a0f60 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1a7ea0 con 0x55bc649b94d0 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbfa7d8a740 10 client.0 _ll_register_callbacks cb 0x7fbf94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649c16b0 0x55bc64d98240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649c16b0 0x55bc64d98240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55290/0 (socket says 172.21.15.78:55290) 2024-04-25T14:06:51.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 -- 172.21.15.78:0/2694776604 learned_addr learned my addr 172.21.15.78:0/2694776604 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649a0f60 0x55bc64d96e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf99d9b640 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649b94d0 0x55bc64d97360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 -- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649b94d0 msgr2=0x55bc64d97360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649b94d0 0x55bc64d97360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 -- 172.21.15.78:0/2694776604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649a0f60 msgr2=0x55bc64d96e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649a0f60 0x55bc64d96e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 -- 172.21.15.78:0/2694776604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc64d98800 con 0x55bc649c16b0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf9ad9d640 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649c16b0 0x55bc64d98240 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbf94004520 tx=0x7fbf94009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a044f8519f2648f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.090+0000 7fbf81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbf81ffb640 1 -- 172.21.15.78:0/2694776604 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94035040 con 0x55bc649c16b0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbf81ffb640 1 -- 172.21.15.78:0/2694776604 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbf94005250 con 0x55bc649c16b0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8c0a7180 con 0x55bc649c16b0 2024-04-25T14:06:51.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbf81ffb640 1 -- 172.21.15.78:0/2694776604 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbf94005450 con 0x55bc649c16b0 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 10 client.17664 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bc64e08570 con 0x55bc649c16b0 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbf81ffb640 1 -- 172.21.15.78:0/2694776604 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c0a7180 con 0x55bc649c16b0 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbf81ffb640 1 -- 172.21.15.78:0/2694776604 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbf9403a070 con 0x55bc649c16b0 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 10 client.17664 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 10 client.17664 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 20 client.17664 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649e2390 0x55bc64e0b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 4 client.17664 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.091+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc64d86520 con 0x55bc649e2390 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649e2390 0x55bc64e0b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbf81ffb640 10 client.17664 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:51.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbf9a59c640 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649e2390 0x55bc64e0b010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9cd4d7d6402fe44 server_cookie=348711424ce0bde7 in_seq=0 out_seq=0 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbf81ffb640 1 -- 172.21.15.78:0/2694776604 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc64d86520 con 0x55bc649e2390 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbf81ffb640 10 client.17664 handle_command_reply: tid=0 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 client.17664 shutdown 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649c16b0 msgr2=0x55bc64d98240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649c16b0 0x55bc64d98240 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fbf94004520 tx=0x7fbf94009260 comp rx=0 tx=0).stop 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 shutdown_connections 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55bc649e2390 0x55bc64e0b010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc649c16b0 0x55bc64d98240 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbf81ffb640 0 client.17664 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc649b94d0 0x55bc64d97360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 --2- 172.21.15.78:0/2694776604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc649a0f60 0x55bc64d96e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 >> 172.21.15.78:0/2694776604 conn(0x55bc649d64e0 msgr2=0x55bc64d925c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 shutdown_connections 2024-04-25T14:06:51.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 1 -- 172.21.15.78:0/2694776604 wait complete. 2024-04-25T14:06:51.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.092+0000 7fbfa7d8a740 20 client.17664 trim_cache size 0 max 16384 2024-04-25T14:06:51.099 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbf840afb70 msgr2=0x7fbf840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbf840afb70 0x7fbf840b2080 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbf9402a930 tx=0x7fbf94005ea0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c1029f0 msgr2=0x7fbf9c199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c1029f0 0x7fbf9c199160 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00cce0 tx=0x7fbf8c00a620 comp rx=0 tx=0).stop 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 shutdown_connections 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf9c108930 0x7fbf9c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf9c103c30 0x7fbf9c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbf840afb70 0x7fbf840b2080 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 --2- 172.21.15.78:0/2855483950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf9c1029f0 0x7fbf9c199160 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 >> 172.21.15.78:0/2855483950 conn(0x7fbf9c0fdbe0 msgr2=0x7fbf9c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 shutdown_connections 2024-04-25T14:06:51.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.093+0000 7fbfa0ee0640 1 -- 172.21.15.78:0/2855483950 wait complete. 2024-04-25T14:06:51.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:06:51.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/1576975150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4104d30 msgr2=0x7f6bf4105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/1576975150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4104d30 0x7f6bf4105140 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/1576975150 shutdown_connections 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/1576975150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf4109a30 0x7f6bf410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/1576975150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4104d30 0x7f6bf4105140 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/1576975150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf4103af0 0x7f6bf41047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/1576975150 >> 172.21.15.78:0/1576975150 conn(0x7f6bf40fece0 msgr2=0x7f6bf4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/1576975150 shutdown_connections 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.213+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/1576975150 wait complete. 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 Processor -- start 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 -- start start 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4103af0 0x7f6bf419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf4104d30 0x7f6bf419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf4109a30 0x7f6bf41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410f5f0 con 0x7f6bf4103af0 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410f470 con 0x7f6bf4109a30 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf8fc5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf410f770 con 0x7f6bf4104d30 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4103af0 0x7f6bf419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4103af0 0x7f6bf419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55306/0 (socket says 172.21.15.78:55306) 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf259c640 1 -- 172.21.15.78:0/2873840002 learned_addr learned my addr 172.21.15.78:0/2873840002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.214+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf4109a30 0x7f6bf41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bebfff640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf4104d30 0x7f6bf419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf259c640 1 -- 172.21.15.78:0/2873840002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf4104d30 msgr2=0x7f6bf419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf4104d30 0x7f6bf419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf259c640 1 -- 172.21.15.78:0/2873840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf4109a30 msgr2=0x7f6bf41a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf4109a30 0x7f6bf41a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf259c640 1 -- 172.21.15.78:0/2873840002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf41a1d60 con 0x7f6bf4103af0 2024-04-25T14:06:51.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4103af0 0x7f6bf419a220 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005fd0 tx=0x7f6be400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6beb7fe640 1 -- 172.21.15.78:0/2873840002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4017070 con 0x7f6bf4103af0 2024-04-25T14:06:51.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf41a1ff0 con 0x7f6bf4103af0 2024-04-25T14:06:51.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6beb7fe640 1 -- 172.21.15.78:0/2873840002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be4009c30 con 0x7f6bf4103af0 2024-04-25T14:06:51.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6beb7fe640 1 -- 172.21.15.78:0/2873840002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4003d10 con 0x7f6bf4103af0 2024-04-25T14:06:51.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.215+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf41a2500 con 0x7f6bf4103af0 2024-04-25T14:06:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6beb7fe640 1 -- 172.21.15.78:0/2873840002 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6be4003eb0 con 0x7f6bf4103af0 2024-04-25T14:06:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 Processor -- start 2024-04-25T14:06:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 -- start start 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d4e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4949834d0 0x55f494d4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f494cb2d80 con 0x55f4949834d0 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f494c6ca20 con 0x55f49498b6b0 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bffe77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f494caed40 con 0x55f4949ac390 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bf259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4949834d0 0x55f494d4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bf259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4949834d0 0x55f494d4ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55314/0 (socket says 172.21.15.78:55314) 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bf259c640 1 -- 172.21.15.78:0/2454711194 learned_addr learned my addr 172.21.15.78:0/2454711194 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d4e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6beb7fe640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be00afaf0 0x7f6be00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.218+0000 7f6bebfff640 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf259c640 1 -- 172.21.15.78:0/2454711194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 msgr2=0x55f494d4e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be00afaf0 0x7f6be00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d4e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf259c640 1 -- 172.21.15.78:0/2454711194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 msgr2=0x55f494d4fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d4fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf259c640 1 -- 172.21.15.78:0/2454711194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f494c77c30 con 0x55f4949834d0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bebfff640 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d4fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6beb7fe640 1 -- 172.21.15.78:0/2873840002 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6be40de940 con 0x7f6bf4103af0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4949834d0 0x55f494d4ed90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6be4060e80 tx=0x7f6be4061b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c909f7c236ce9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be00afaf0 0x7f6be00b2000 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55f494d4f860 tx=0x7f6bec004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6be8ff9640 1 -- 172.21.15.78:0/2454711194 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be40a6760 con 0x55f4949834d0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6be8ff9640 1 -- 172.21.15.78:0/2454711194 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be40e0e50 con 0x55f4949834d0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.219+0000 7f6be8ff9640 1 -- 172.21.15.78:0/2454711194 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4010500 con 0x55f4949834d0 2024-04-25T14:06:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 -- 172.21.15.78:0/2454711194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4949834d0 msgr2=0x55f494d4ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4949834d0 0x55f494d4ed90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6be4060e80 tx=0x7f6be4061b80 comp rx=0 tx=0).stop 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 -- 172.21.15.78:0/2454711194 shutdown_connections 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d4fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4949834d0 0x55f494d4ed90 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2454711194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d4e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 -- 172.21.15.78:0/2454711194 >> 172.21.15.78:0/2454711194 conn(0x55f4949a04e0 msgr2=0x55f494d4a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 -- 172.21.15.78:0/2454711194 shutdown_connections 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.220+0000 7f6bffe77740 1 -- 172.21.15.78:0/2454711194 wait complete. 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 -- start start 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 Processor -- start 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4949834d0 0x55f494d584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494d58a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f494d504d0 con 0x55f49498b6b0 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f494c39dc0 con 0x55f4949834d0 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f494d50650 con 0x55f4949ac390 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bffe77740 10 client.0 _ll_register_callbacks cb 0x55f494c55600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494d58a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494d58a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55330/0 (socket says 172.21.15.78:55330) 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 -- 172.21.15.78:0/2944863116 learned_addr learned my addr 172.21.15.78:0/2944863116 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4949834d0 0x55f494d584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 -- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 msgr2=0x55f494d5f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d5f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 -- 172.21.15.78:0/2944863116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4949834d0 msgr2=0x55f494d584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4949834d0 0x55f494d584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 -- 172.21.15.78:0/2944863116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4949c43f0 con 0x55f49498b6b0 2024-04-25T14:06:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bebfff640 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494d58a30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6bf419b2c0 tx=0x7f6bdc031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c2ff22a6ef1055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.221+0000 7f6bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2944863116 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc03f070 con 0x55f49498b6b0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2944863116 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6bdc004450 con 0x55f49498b6b0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be40e0e50 con 0x55f49498b6b0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2944863116 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6bdc035070 con 0x55f49498b6b0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 10 client.17676 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f494a8e7d0 con 0x55f49498b6b0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2944863116 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be40e0e50 con 0x55f49498b6b0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2944863116 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6bdc004b50 con 0x55f49498b6b0 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 10 client.17676 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 10 client.17676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 20 client.17676 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f49496af60 0x55f494d66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 4 client.17676 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f494cc0990 con 0x55f49496af60 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.222+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f49496af60 0x55f494d66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.223+0000 7f6bd9ffb640 10 client.17676 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.223+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f49496af60 0x55f494d66260 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8501b89053f35d99 server_cookie=8e6ec5f3db53722d in_seq=0 out_seq=0 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2944863116 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f494cc0990 con 0x55f49496af60 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bd9ffb640 10 client.17676 handle_command_reply: tid=0 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 client.17676 shutdown 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 msgr2=0x55f494d58a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494d58a30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6bf419b2c0 tx=0x7f6bdc031620 comp rx=0 tx=0).stop 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 shutdown_connections 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f49496af60 0x55f494d66260 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949ac390 0x55f494d5f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494d58a30 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bd9ffb640 0 client.17676 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2944863116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4949834d0 0x55f494d584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 >> 172.21.15.78:0/2944863116 conn(0x55f4949a04e0 msgr2=0x55f494d4cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 shutdown_connections 2024-04-25T14:06:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 1 -- 172.21.15.78:0/2944863116 wait complete. 2024-04-25T14:06:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.224+0000 7f6bffe77740 20 client.17676 trim_cache size 0 max 16384 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.237+0000 7f6bffe77740 1 Processor -- start 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.237+0000 7f6bffe77740 1 -- start start 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.237+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49496af60 0x55f494d56cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 0x55f494d53290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bffe77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41adbc0 con 0x55f49496af60 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bffe77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41ab490 con 0x55f49498b6b0 2024-04-25T14:06:51.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bffe77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41a92e0 con 0x55f4949834d0 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49496af60 0x55f494d56cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49496af60 0x55f494d56cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55334/0 (socket says 172.21.15.78:55334) 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 -- 172.21.15.78:0/1442484105 learned_addr learned my addr 172.21.15.78:0/1442484105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 -- 172.21.15.78:0/1442484105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 msgr2=0x55f494d53290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 --2- 172.21.15.78:0/1442484105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 0x55f494d53290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 -- 172.21.15.78:0/1442484105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 msgr2=0x55f494d537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 --2- 172.21.15.78:0/1442484105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 -- 172.21.15.78:0/1442484105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f494d4eea0 con 0x55f49496af60 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bf259c640 1 --2- 172.21.15.78:0/1442484105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49496af60 0x55f494d56cb0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6be40058c0 tx=0x7f6be400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731a4e043b88991d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bdbfff640 1 -- 172.21.15.78:0/1442484105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be40a6580 con 0x55f49496af60 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bdbfff640 1 -- 172.21.15.78:0/1442484105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be40e43a0 con 0x55f49496af60 2024-04-25T14:06:51.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.238+0000 7f6bdbfff640 1 -- 172.21.15.78:0/1442484105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be40a29a0 con 0x55f49496af60 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 -- 172.21.15.78:0/1442484105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49496af60 msgr2=0x55f494d56cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 --2- 172.21.15.78:0/1442484105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49496af60 0x55f494d56cb0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6be40058c0 tx=0x7f6be400bf60 comp rx=0 tx=0).stop 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 -- 172.21.15.78:0/1442484105 shutdown_connections 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 --2- 172.21.15.78:0/1442484105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49498b6b0 0x55f494d537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 --2- 172.21.15.78:0/1442484105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 0x55f494d53290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 --2- 172.21.15.78:0/1442484105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49496af60 0x55f494d56cb0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 -- 172.21.15.78:0/1442484105 >> 172.21.15.78:0/1442484105 conn(0x55f4949a04e0 msgr2=0x55f494d4e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 -- 172.21.15.78:0/1442484105 shutdown_connections 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 -- 172.21.15.78:0/1442484105 wait complete. 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 -- start start 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 Processor -- start 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.239+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49496af60 0x55f494d5c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 0x55f494d5c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494dcc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41a9950 con 0x55f49498b6b0 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41aadc0 con 0x55f49496af60 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41aa020 con 0x55f4949834d0 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 10 client.0 _ll_register_callbacks cb 0x7f6bdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494dcc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494dcc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55350/0 (socket says 172.21.15.78:55350) 2024-04-25T14:06:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 -- 172.21.15.78:0/2976642567 learned_addr learned my addr 172.21.15.78:0/2976642567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49496af60 0x55f494d5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 -- 172.21.15.78:0/2976642567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 msgr2=0x55f494d5c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 0x55f494d5c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 -- 172.21.15.78:0/2976642567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49496af60 msgr2=0x55f494d5c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49496af60 0x55f494d5c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 -- 172.21.15.78:0/2976642567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f494dccb00 con 0x55f49498b6b0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bf2d9d640 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494dcc5c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f6bec003dc0 tx=0x7f6bec009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74fc76e501ffaaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2976642567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec01a040 con 0x55f49498b6b0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2976642567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6bec012cf0 con 0x55f49498b6b0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be40e43a0 con 0x55f49498b6b0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2976642567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6bec014030 con 0x55f49498b6b0 2024-04-25T14:06:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 10 client.17682 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.240+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f494dcde10 con 0x55f49498b6b0 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2976642567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be40e43a0 con 0x55f49498b6b0 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2976642567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6bec020040 con 0x55f49498b6b0 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bffe77740 10 client.17682 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bffe77740 10 client.17682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bffe77740 20 client.17682 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f4949ac390 0x55f494dd08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bffe77740 4 client.17682 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f494cba800 con 0x55f4949ac390 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f4949ac390 0x55f494dd08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bd9ffb640 10 client.17682 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.241+0000 7f6bf259c640 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f4949ac390 0x55f494dd08b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45cd681c49a8c0f server_cookie=b2061ff0982010d1 in_seq=0 out_seq=0 2024-04-25T14:06:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bd9ffb640 1 -- 172.21.15.78:0/2976642567 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f494cba800 con 0x55f4949ac390 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bd9ffb640 10 client.17682 handle_command_reply: tid=0 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 client.17682 shutdown 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 msgr2=0x55f494dcc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494dcc5c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f6bec003dc0 tx=0x7f6bec009df0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 shutdown_connections 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f49498b6b0 0x55f494dcc5c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f4949ac390 0x55f494dd08b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4949834d0 0x55f494d5c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bd9ffb640 0 client.17682 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 --2- 172.21.15.78:0/2976642567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f49496af60 0x55f494d5c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 >> 172.21.15.78:0/2976642567 conn(0x55f4949a04e0 msgr2=0x55f494d4e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 shutdown_connections 2024-04-25T14:06:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 1 -- 172.21.15.78:0/2976642567 wait complete. 2024-04-25T14:06:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bffe77740 20 client.17682 trim_cache size 0 max 16384 2024-04-25T14:06:51.249 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be00afaf0 msgr2=0x7f6be00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be00afaf0 0x7f6be00b2000 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55f494d4f860 tx=0x7f6bec004ab0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4103af0 msgr2=0x7f6bf419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4103af0 0x7f6bf419a220 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005fd0 tx=0x7f6be400dde0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 shutdown_connections 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf4109a30 0x7f6bf41a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bf4104d30 0x7f6bf419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6be00afaf0 0x7f6be00b2000 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 --2- 172.21.15.78:0/2873840002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bf4103af0 0x7f6bf419a220 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.242+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 >> 172.21.15.78:0/2873840002 conn(0x7f6bf40fece0 msgr2=0x7f6bf4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.243+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 shutdown_connections 2024-04-25T14:06:51.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.243+0000 7f6bf8fc5640 1 -- 172.21.15.78:0/2873840002 wait complete. 2024-04-25T14:06:51.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 -- 172.21.15.78:0/860590528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 msgr2=0x7fe504102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 --2- 172.21.15.78:0/860590528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe504102e80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009ca0 tx=0x7fe4fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 -- 172.21.15.78:0/860590528 shutdown_connections 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 --2- 172.21.15.78:0/860590528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5041080c0 0x7fe50410a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 --2- 172.21.15.78:0/860590528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5041033c0 0x7fe5041037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 --2- 172.21.15.78:0/860590528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe504102e80 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 -- 172.21.15.78:0/860590528 >> 172.21.15.78:0/860590528 conn(0x7fe5040fd350 msgr2=0x7fe5040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 -- 172.21.15.78:0/860590528 shutdown_connections 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.364+0000 7fe50be77640 1 -- 172.21.15.78:0/860590528 wait complete. 2024-04-25T14:06:51.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 Processor -- start 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 -- start start 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe50419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5041033c0 0x7fe50419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5041080c0 0x7fe5041a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410e520 con 0x7fe504102180 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410e3a0 con 0x7fe5041080c0 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe50be77640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410e6a0 con 0x7fe5041033c0 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe50419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe50419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55356/0 (socket says 172.21.15.78:55356) 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 -- 172.21.15.78:0/4076345965 learned_addr learned my addr 172.21.15.78:0/4076345965 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509412640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5041033c0 0x7fe50419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 -- 172.21.15.78:0/4076345965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5041033c0 msgr2=0x7fe50419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5041033c0 0x7fe50419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 -- 172.21.15.78:0/4076345965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5041080c0 msgr2=0x7fe5041a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5041080c0 0x7fe5041a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 -- 172.21.15.78:0/4076345965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5041a1ce0 con 0x7fe504102180 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.365+0000 7fe509c13640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe50419a2c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0098d0 tx=0x7fe4fc031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.366+0000 7fe502ffd640 1 -- 172.21.15.78:0/4076345965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc003cb0 con 0x7fe504102180 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.366+0000 7fe502ffd640 1 -- 172.21.15.78:0/4076345965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4fc003e50 con 0x7fe504102180 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.366+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5041a1f70 con 0x7fe504102180 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.366+0000 7fe502ffd640 1 -- 172.21.15.78:0/4076345965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc02fc60 con 0x7fe504102180 2024-04-25T14:06:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.366+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041a23d0 con 0x7fe504102180 2024-04-25T14:06:51.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.368+0000 7fe512e05740 1 Processor -- start 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe502ffd640 1 -- 172.21.15.78:0/4076345965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe4fc004040 con 0x7fe504102180 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe512e05740 1 -- start start 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe512e05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55740617d390 0x55740651f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe512e05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061544d0 0x55740651fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe512e05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740615c6b0 0x557406520a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe512e05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557406483d80 con 0x55740617d390 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe512e05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55740643da20 con 0x5574061544d0 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe512e05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55740647fd40 con 0x55740615c6b0 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55740617d390 0x55740651f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55740617d390 0x55740651f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55370/0 (socket says 172.21.15.78:55370) 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 -- 172.21.15.78:0/2363071278 learned_addr learned my addr 172.21.15.78:0/2363071278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe509412640 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740615c6b0 0x557406520a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe509c13640 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061544d0 0x55740651fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 -- 172.21.15.78:0/2363071278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740615c6b0 msgr2=0x557406520a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740615c6b0 0x557406520a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 -- 172.21.15.78:0/2363071278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061544d0 msgr2=0x55740651fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061544d0 0x55740651fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 -- 172.21.15.78:0/2363071278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557406448c30 con 0x55740617d390 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe509412640 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740615c6b0 0x557406520a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe509c13640 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061544d0 0x55740651fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe502ffd640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4ec0afaf0 0x7fe4ec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe50a414640 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55740617d390 0x55740651f810 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400a9f0 tx=0x7fe4f400aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160aff237fbd4306 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.369+0000 7fe509c13640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4ec0afaf0 0x7fe4ec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.370+0000 7fe4ebfff640 1 -- 172.21.15.78:0/2363071278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f400cde0 con 0x55740617d390 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.370+0000 7fe502ffd640 1 -- 172.21.15.78:0/4076345965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe4fc100320 con 0x7fe504102180 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.370+0000 7fe4ebfff640 1 -- 172.21.15.78:0/2363071278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f400f900 con 0x55740617d390 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.370+0000 7fe4ebfff640 1 -- 172.21.15.78:0/2363071278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40189a0 con 0x55740617d390 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.370+0000 7fe512e05740 1 -- 172.21.15.78:0/2363071278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55740617d390 msgr2=0x55740651f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.370+0000 7fe509c13640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4ec0afaf0 0x7fe4ec0b2000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55740651f530 tx=0x7fe4fc009460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.370+0000 7fe512e05740 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55740617d390 0x55740651f810 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400a9f0 tx=0x7fe4f400aeb0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 -- 172.21.15.78:0/2363071278 shutdown_connections 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740615c6b0 0x557406520a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061544d0 0x55740651fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 --2- 172.21.15.78:0/2363071278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55740617d390 0x55740651f810 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 -- 172.21.15.78:0/2363071278 >> 172.21.15.78:0/2363071278 conn(0x5574061714e0 msgr2=0x55740651b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 -- 172.21.15.78:0/2363071278 shutdown_connections 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 -- 172.21.15.78:0/2363071278 wait complete. 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 -- start start 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 Processor -- start 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x5574065295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740615c6b0 0x557406529b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740617d390 0x557406530a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.371+0000 7fe512e05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557406521530 con 0x5574061544d0 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe512e05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55740640adc0 con 0x55740615c6b0 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe512e05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574065216b0 con 0x55740617d390 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe512e05740 10 client.0 _ll_register_callbacks cb 0x557406426600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x5574065295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x5574065295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55382/0 (socket says 172.21.15.78:55382) 2024-04-25T14:06:51.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 -- 172.21.15.78:0/201394621 learned_addr learned my addr 172.21.15.78:0/201394621 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740617d390 0x557406530a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe509412640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740615c6b0 0x557406529b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 -- 172.21.15.78:0/201394621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740617d390 msgr2=0x557406530a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740617d390 0x557406530a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 -- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740615c6b0 msgr2=0x557406529b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740615c6b0 0x557406529b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 -- 172.21.15.78:0/201394621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574061953f0 con 0x5574061544d0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe509412640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740615c6b0 0x557406529b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe50a414640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x5574065295d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4005b50 tx=0x7fe4f400a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240c3e882f8a5396 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe4e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe4e97fa640 1 -- 172.21.15.78:0/201394621 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4004020 con 0x5574061544d0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe4e97fa640 1 -- 172.21.15.78:0/201394621 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe4f40041c0 con 0x5574061544d0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f400f900 con 0x5574061544d0 2024-04-25T14:06:51.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe4e97fa640 1 -- 172.21.15.78:0/201394621 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe4f40043c0 con 0x5574061544d0 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe512e05740 10 client.17694 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.372+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55740625f7d0 con 0x5574061544d0 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe4e97fa640 1 -- 172.21.15.78:0/201394621 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f400f900 con 0x5574061544d0 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe4e97fa640 1 -- 172.21.15.78:0/201394621 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe4f4026030 con 0x5574061544d0 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe512e05740 10 client.17694 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe512e05740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe512e05740 20 client.17694 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe512e05740 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740613bf60 0x557406536f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe512e05740 4 client.17694 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557406491990 con 0x55740613bf60 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe509412640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740613bf60 0x557406536f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe4e97fa640 10 client.17694 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.373+0000 7fe509412640 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740613bf60 0x557406536f40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0f1177e64de25c server_cookie=3993d8deccb78bd2 in_seq=0 out_seq=0 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.374+0000 7fe4e97fa640 1 -- 172.21.15.78:0/201394621 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557406491990 con 0x55740613bf60 2024-04-25T14:06:51.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.374+0000 7fe4e97fa640 10 client.17694 handle_command_reply: tid=0 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.374+0000 7fe512e05740 1 client.17694 shutdown 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.374+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 msgr2=0x5574065295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.374+0000 7fe512e05740 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x5574065295d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4005b50 tx=0x7fe4f400a500 comp rx=0 tx=0).stop 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 shutdown_connections 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740613bf60 0x557406536f40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740617d390 0x557406530a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe4e97fa640 0 client.17694 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740615c6b0 0x557406529b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 --2- 172.21.15.78:0/201394621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x5574065295d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 >> 172.21.15.78:0/201394621 conn(0x5574061714e0 msgr2=0x55740651a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 shutdown_connections 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 1 -- 172.21.15.78:0/201394621 wait complete. 2024-04-25T14:06:51.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.375+0000 7fe512e05740 20 client.17694 trim_cache size 0 max 16384 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 Processor -- start 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 -- start start 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061714e0 0x5574065298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740613bf60 0x557406530a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574061544d0 0x55740653b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041adab0 con 0x5574061714e0 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041ab380 con 0x55740613bf60 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe512e05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041a91d0 con 0x5574061544d0 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.388+0000 7fe50a414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061714e0 0x5574065298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061714e0 0x5574065298b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55398/0 (socket says 172.21.15.78:55398) 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 -- 172.21.15.78:0/1300435078 learned_addr learned my addr 172.21.15.78:0/1300435078 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe509412640 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740613bf60 0x557406530a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 -- 172.21.15.78:0/1300435078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574061544d0 msgr2=0x55740653b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574061544d0 0x55740653b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 -- 172.21.15.78:0/1300435078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740613bf60 msgr2=0x557406530a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740613bf60 0x557406530a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 -- 172.21.15.78:0/1300435078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557406527670 con 0x5574061714e0 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe50a414640 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061714e0 0x5574065298b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40046f0 tx=0x7fe4f4003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815e97a64cf4183a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/1300435078 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40035f0 con 0x5574061714e0 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/1300435078 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f400f7d0 con 0x5574061714e0 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.389+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/1300435078 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4035b50 con 0x5574061714e0 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 -- 172.21.15.78:0/1300435078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061714e0 msgr2=0x5574065298b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061714e0 0x5574065298b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40046f0 tx=0x7fe4f4003ad0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 -- 172.21.15.78:0/1300435078 shutdown_connections 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574061544d0 0x55740653b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55740613bf60 0x557406530a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 --2- 172.21.15.78:0/1300435078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061714e0 0x5574065298b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 -- 172.21.15.78:0/1300435078 >> 172.21.15.78:0/1300435078 conn(0x557406537270 msgr2=0x5574065291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 -- 172.21.15.78:0/1300435078 shutdown_connections 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 -- 172.21.15.78:0/1300435078 wait complete. 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 -- start start 2024-04-25T14:06:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.390+0000 7fe512e05740 1 Processor -- start 2024-04-25T14:06:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740613bf60 0x55740652d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x55740652dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061714e0 0x55740659d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041a9840 con 0x5574061544d0 2024-04-25T14:06:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041aacb0 con 0x5574061714e0 2024-04-25T14:06:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041a9f10 con 0x55740613bf60 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 10 client.0 _ll_register_callbacks cb 0x7fe4f400a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x55740652dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x55740652dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55410/0 (socket says 172.21.15.78:55410) 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 -- 172.21.15.78:0/3424597599 learned_addr learned my addr 172.21.15.78:0/3424597599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe50a414640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740613bf60 0x55740652d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe50a414640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061714e0 0x55740659d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 -- 172.21.15.78:0/3424597599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740613bf60 msgr2=0x55740652d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740613bf60 0x55740652d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 -- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061714e0 msgr2=0x55740659d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061714e0 0x55740659d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 -- 172.21.15.78:0/3424597599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55740652e3a0 con 0x5574061544d0 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe50a414640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740613bf60 0x55740652d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:51.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe509412640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x55740652dde0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x55740651bdb0 tx=0x7fe4f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c02a84e150cbee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe4e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe4e97fa640 1 -- 172.21.15.78:0/3424597599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f801b6b0 con 0x5574061544d0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe4e97fa640 1 -- 172.21.15.78:0/3424597599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe4f801b850 con 0x5574061544d0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.391+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f400f7d0 con 0x5574061544d0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe4e97fa640 1 -- 172.21.15.78:0/3424597599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe4f801ba50 con 0x5574061544d0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 10 client.17700 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55740659e930 con 0x5574061544d0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe4e97fa640 1 -- 172.21.15.78:0/3424597599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f400f7d0 con 0x5574061544d0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe4e97fa640 1 -- 172.21.15.78:0/3424597599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe4f80104c0 con 0x5574061544d0 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 10 client.17700 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 20 client.17700 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:51.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740615c6b0 0x5574065a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 4 client.17700 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55740648b800 con 0x55740615c6b0 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.392+0000 7fe50a414640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740615c6b0 0x5574065a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe50a414640 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740615c6b0 0x5574065a1380 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ff8655bf9e3fc8 server_cookie=17aad7a7c3bda87e in_seq=0 out_seq=0 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe4e97fa640 10 client.17700 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe4e97fa640 1 -- 172.21.15.78:0/3424597599 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55740648b800 con 0x55740615c6b0 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe4e97fa640 10 client.17700 handle_command_reply: tid=0 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 client.17700 shutdown 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 msgr2=0x55740652dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x55740652dde0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x55740651bdb0 tx=0x7fe4f8011810 comp rx=0 tx=0).stop 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 shutdown_connections 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5574061714e0 0x55740659d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574061544d0 0x55740652dde0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55740615c6b0 0x5574065a1380 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 --2- 172.21.15.78:0/3424597599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55740613bf60 0x55740652d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 >> 172.21.15.78:0/3424597599 conn(0x557406537270 msgr2=0x55740651de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe4e97fa640 0 client.17700 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 shutdown_connections 2024-04-25T14:06:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 1 -- 172.21.15.78:0/3424597599 wait complete. 2024-04-25T14:06:51.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.393+0000 7fe512e05740 20 client.17700 trim_cache size 0 max 16384 2024-04-25T14:06:51.400 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4ec0afaf0 msgr2=0x7fe4ec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4ec0afaf0 0x7fe4ec0b2000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55740651f530 tx=0x7fe4fc009460 comp rx=0 tx=0).stop 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 msgr2=0x7fe50419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe50419a2c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0098d0 tx=0x7fe4fc031840 comp rx=0 tx=0).stop 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 shutdown_connections 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5041080c0 0x7fe5041a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5041033c0 0x7fe50419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe4ec0afaf0 0x7fe4ec0b2000 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 --2- 172.21.15.78:0/4076345965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe504102180 0x7fe50419a2c0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 >> 172.21.15.78:0/4076345965 conn(0x7fe5040fd350 msgr2=0x7fe504109590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 shutdown_connections 2024-04-25T14:06:51.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:51.394+0000 7fe50be77640 1 -- 172.21.15.78:0/4076345965 wait complete. 2024-04-25T14:06:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:51 smithi129 ceph-mon[32569]: pgmap v568: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:06:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:51 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3451372052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:51 smithi078 ceph-mon[29163]: pgmap v568: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:06:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:51 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3451372052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:51.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:51 smithi112 ceph-mon[32617]: pgmap v568: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:06:51.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:51 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3451372052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:06:52.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:06:52.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:06:52.406 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:06:52.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 -- 172.21.15.78:0/2415247425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 msgr2=0x7f948810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 --2- 172.21.15.78:0/2415247425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f948810be60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9480009900 tx=0x7f948002a410 comp rx=0 tx=0).stop 2024-04-25T14:06:52.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 -- 172.21.15.78:0/2415247425 shutdown_connections 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 --2- 172.21.15.78:0/2415247425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f948810be60 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 --2- 172.21.15.78:0/2415247425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488104d40 0x7f9488105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 --2- 172.21.15.78:0/2415247425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488103b00 0x7f9488104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 -- 172.21.15.78:0/2415247425 >> 172.21.15.78:0/2415247425 conn(0x7f94880fed10 msgr2=0x7f94881011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 -- 172.21.15.78:0/2415247425 shutdown_connections 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 -- 172.21.15.78:0/2415247425 wait complete. 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.535+0000 7f948fd72640 1 Processor -- start 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948fd72640 1 -- start start 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948fd72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488103b00 0x7f948819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948fd72640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488104d40 0x7f948819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948fd72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f94881a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948fd72640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f948810f630 con 0x7f9488109a40 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948fd72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f948810f4b0 con 0x7f9488103b00 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948fd72640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f948810f7b0 con 0x7f9488104d40 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f94881a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f94881a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55426/0 (socket says 172.21.15.78:55426) 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 -- 172.21.15.78:0/2559362698 learned_addr learned my addr 172.21.15.78:0/2559362698 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948d30d640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488104d40 0x7f948819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948db0e640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488103b00 0x7f948819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 -- 172.21.15.78:0/2559362698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488104d40 msgr2=0x7f948819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488104d40 0x7f948819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 -- 172.21.15.78:0/2559362698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488103b00 msgr2=0x7f948819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488103b00 0x7f948819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 -- 172.21.15.78:0/2559362698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94881a1d50 con 0x7f9488109a40 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948db0e640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488103b00 0x7f948819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:52.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f948e30f640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f94881a16e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9480009230 tx=0x7f9480031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.536+0000 7f947effd640 1 -- 172.21.15.78:0/2559362698 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948003f030 con 0x7f9488109a40 2024-04-25T14:06:52.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.537+0000 7f947effd640 1 -- 172.21.15.78:0/2559362698 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9480003aa0 con 0x7f9488109a40 2024-04-25T14:06:52.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.537+0000 7f947effd640 1 -- 172.21.15.78:0/2559362698 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480035070 con 0x7f9488109a40 2024-04-25T14:06:52.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.537+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94881a1fe0 con 0x7f9488109a40 2024-04-25T14:06:52.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.537+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94881a23f0 con 0x7f9488109a40 2024-04-25T14:06:52.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.539+0000 7f947effd640 1 -- 172.21.15.78:0/2559362698 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9480042030 con 0x7f9488109a40 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.539+0000 7f9496c1e740 1 Processor -- start 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.539+0000 7f9496c1e740 1 -- start start 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8832410 0x563cb8bd34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bd3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8bda920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f9496c1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563cb8b38b00 con 0x563cb8809550 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f9496c1e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cb8ad4920 con 0x563cb8832410 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f9496c1e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563cb8b34ac0 con 0x563cb8811730 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bd3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bd3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55434/0 (socket says 172.21.15.78:55434) 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 -- 172.21.15.78:0/1157347219 learned_addr learned my addr 172.21.15.78:0/1157347219 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f947effd640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94740afb70 0x7f94740b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948d30d640 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8832410 0x563cb8bd34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948d30d640 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8bda920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 -- 172.21.15.78:0/1157347219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 msgr2=0x563cb8bda920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8bda920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 -- 172.21.15.78:0/1157347219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8832410 msgr2=0x563cb8bd34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8832410 0x563cb8bd34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948db0e640 1 -- 172.21.15.78:0/1157347219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb89127e0 con 0x563cb8809550 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948d30d640 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8832410 0x563cb8bd34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.540+0000 7f948e30f640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94740afb70 0x7f94740b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.541+0000 7f947effd640 1 -- 172.21.15.78:0/2559362698 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f94800fee40 con 0x7f9488109a40 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.541+0000 7f948db0e640 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bd3a20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9484009ee0 tx=0x7f948400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d18d94d808eea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.541+0000 7f9473fff640 1 -- 172.21.15.78:0/1157347219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948401a680 con 0x563cb8809550 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.541+0000 7f9473fff640 1 -- 172.21.15.78:0/1157347219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94840118f0 con 0x563cb8809550 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.541+0000 7f9473fff640 1 -- 172.21.15.78:0/1157347219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484010870 con 0x563cb8809550 2024-04-25T14:06:52.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 -- 172.21.15.78:0/1157347219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 msgr2=0x563cb8bd3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f948e30f640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94740afb70 0x7f94740b2080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f94800c3370 tx=0x7f94800f5a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bd3a20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9484009ee0 tx=0x7f948400f730 comp rx=0 tx=0).stop 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 -- 172.21.15.78:0/1157347219 shutdown_connections 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8bda920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bd3a20 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1157347219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8832410 0x563cb8bd34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 -- 172.21.15.78:0/1157347219 >> 172.21.15.78:0/1157347219 conn(0x563cb8826560 msgr2=0x563cb8bcee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 -- 172.21.15.78:0/1157347219 shutdown_connections 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 -- 172.21.15.78:0/1157347219 wait complete. 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 -- start start 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.542+0000 7f9496c1e740 1 Processor -- start 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bdd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8811730 0x563cb8bdd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563cb8bdb370 con 0x563cb8809550 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563cb8ad25d0 con 0x563cb8811730 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563cb8bdb4f0 con 0x563cb8832410 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 10 client.0 _ll_register_callbacks cb 0x563cb8ac0080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bdd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bdd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55438/0 (socket says 172.21.15.78:55438) 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 -- 172.21.15.78:0/2131081463 learned_addr learned my addr 172.21.15.78:0/2131081463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948db0e640 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8811730 0x563cb8bdd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 -- 172.21.15.78:0/2131081463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 msgr2=0x563cb8be4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 -- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8811730 msgr2=0x563cb8bdd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8811730 0x563cb8bdd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 -- 172.21.15.78:0/2131081463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb8972f70 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f948d30d640 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bdd2a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f947800bd10 tx=0x7f94780041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e400c34a4c9c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f94717fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f94717fa640 1 -- 172.21.15.78:0/2131081463 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478005220 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f94717fa640 1 -- 172.21.15.78:0/2131081463 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f94780053c0 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.543+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94840118f0 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f94717fa640 1 -- 172.21.15.78:0/2131081463 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9478025050 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 10 client.17712 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563cb880a170 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f94717fa640 1 -- 172.21.15.78:0/2131081463 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94840118f0 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f94717fa640 1 -- 172.21.15.78:0/2131081463 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9478011770 con 0x563cb8809550 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 10 client.17712 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 20 client.17712 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:52.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87f11f0 0x563cb8beab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 4 client.17712 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cb8b46710 con 0x563cb87f11f0 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.544+0000 7f948db0e640 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87f11f0 0x563cb8beab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.545+0000 7f94717fa640 10 client.17712 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.545+0000 7f948db0e640 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87f11f0 0x563cb8beab90 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31f479bc8b3529d server_cookie=440b9bc69c12e746 in_seq=0 out_seq=0 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f94717fa640 1 -- 172.21.15.78:0/2131081463 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563cb8b46710 con 0x563cb87f11f0 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f94717fa640 10 client.17712 handle_command_reply: tid=0 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 client.17712 shutdown 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 msgr2=0x563cb8bdd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bdd2a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f947800bd10 tx=0x7f94780041f0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 shutdown_connections 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87f11f0 0x563cb8beab90 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8811730 0x563cb8bdd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f94717fa640 0 client.17712 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 --2- 172.21.15.78:0/2131081463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8809550 0x563cb8bdd2a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 >> 172.21.15.78:0/2131081463 conn(0x563cb8826560 msgr2=0x563cb8bcdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 shutdown_connections 2024-04-25T14:06:52.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 1 -- 172.21.15.78:0/2131081463 wait complete. 2024-04-25T14:06:52.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.546+0000 7f9496c1e740 20 client.17712 trim_cache size 0 max 16384 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 Processor -- start 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 -- start start 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8bf0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8811730 0x563cb8bf08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94881adb20 con 0x563cb8811730 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94881ab3f0 con 0x563cb8809550 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.564+0000 7f9496c1e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94881a9240 con 0x563cb8832410 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8811730 0x563cb8bf08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8811730 0x563cb8bf08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55442/0 (socket says 172.21.15.78:55442) 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 -- 172.21.15.78:0/4215010336 learned_addr learned my addr 172.21.15.78:0/4215010336 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948d30d640 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948d30d640 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8bf0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 -- 172.21.15.78:0/4215010336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 msgr2=0x563cb8be46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 -- 172.21.15.78:0/4215010336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 msgr2=0x563cb8bf0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8bf0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 -- 172.21.15.78:0/4215010336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb8bdb670 con 0x563cb8811730 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948d30d640 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8bf0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f948db0e640 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8811730 0x563cb8bf08d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x563cb8bd4880 tx=0x7f9484009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07459d56b9daef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f94737fe640 1 -- 172.21.15.78:0/4215010336 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948402a020 con 0x563cb8811730 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f94737fe640 1 -- 172.21.15.78:0/4215010336 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94840268e0 con 0x563cb8811730 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.565+0000 7f94737fe640 1 -- 172.21.15.78:0/4215010336 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484011a60 con 0x563cb8811730 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 -- 172.21.15.78:0/4215010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8811730 msgr2=0x563cb8bf08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8811730 0x563cb8bf08d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x563cb8bd4880 tx=0x7f9484009c20 comp rx=0 tx=0).stop 2024-04-25T14:06:52.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 -- 172.21.15.78:0/4215010336 shutdown_connections 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8832410 0x563cb8be46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8811730 0x563cb8bf08d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 --2- 172.21.15.78:0/4215010336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8bf0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 -- 172.21.15.78:0/4215010336 >> 172.21.15.78:0/4215010336 conn(0x563cb8826560 msgr2=0x563cb8bdc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 -- 172.21.15.78:0/4215010336 shutdown_connections 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 -- 172.21.15.78:0/4215010336 wait complete. 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 -- start start 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.566+0000 7f9496c1e740 1 Processor -- start 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8be2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8be3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f9496c1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8832410 0x563cb8be4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f9496c1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94881a98b0 con 0x563cb8832410 2024-04-25T14:06:52.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f9496c1e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94881aad20 con 0x563cb8809550 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f9496c1e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94881a9f80 con 0x563cb8811730 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f9496c1e740 10 client.0 _ll_register_callbacks cb 0x7f947800b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8832410 0x563cb8be4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8832410 0x563cb8be4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55456/0 (socket says 172.21.15.78:55456) 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 -- 172.21.15.78:0/1385317004 learned_addr learned my addr 172.21.15.78:0/1385317004 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948db0e640 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8be3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8be2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 -- 172.21.15.78:0/1385317004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 msgr2=0x563cb8be3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8be3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 -- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 msgr2=0x563cb8be2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8be2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 -- 172.21.15.78:0/1385317004 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb8be4710 con 0x563cb8832410 2024-04-25T14:06:52.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f948d30d640 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8832410 0x563cb8be4150 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f947800b6b0 tx=0x7f9478000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84629b16b9dfab57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.567+0000 7f94717fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f94717fa640 1 -- 172.21.15.78:0/1385317004 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478011850 con 0x563cb8832410 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f94717fa640 1 -- 172.21.15.78:0/1385317004 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9478010550 con 0x563cb8832410 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94840268e0 con 0x563cb8832410 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f94717fa640 1 -- 172.21.15.78:0/1385317004 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9478010750 con 0x563cb8832410 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 10 client.17718 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563cb8c534a0 con 0x563cb8832410 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f94717fa640 1 -- 172.21.15.78:0/1385317004 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94840268e0 con 0x563cb8832410 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f94717fa640 1 -- 172.21.15.78:0/1385317004 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f947801e7d0 con 0x563cb8832410 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 10 client.17718 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 20 client.17718 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87c6630 0x563cb8c55e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 4 client.17718 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:06:52.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563cb8b40580 con 0x563cb87c6630 2024-04-25T14:06:52.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.568+0000 7f948db0e640 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87c6630 0x563cb8c55e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.569+0000 7f94717fa640 10 client.17718 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.569+0000 7f948db0e640 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87c6630 0x563cb8c55e00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1aebc7026f27b69 server_cookie=91e2880e971d20e0 in_seq=0 out_seq=0 2024-04-25T14:06:52.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f94717fa640 1 -- 172.21.15.78:0/1385317004 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563cb8b40580 con 0x563cb87c6630 2024-04-25T14:06:52.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f94717fa640 10 client.17718 handle_command_reply: tid=0 2024-04-25T14:06:52.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 client.17718 shutdown 2024-04-25T14:06:52.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8832410 msgr2=0x563cb8be4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8832410 0x563cb8be4150 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f947800b6b0 tx=0x7f9478000bc0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 shutdown_connections 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563cb8832410 0x563cb8be4150 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563cb87c6630 0x563cb8c55e00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563cb8811730 0x563cb8be3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 --2- 172.21.15.78:0/1385317004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563cb8809550 0x563cb8be2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f94717fa640 0 client.17718 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 >> 172.21.15.78:0/1385317004 conn(0x563cb8826560 msgr2=0x563cb8bef380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.570+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 shutdown_connections 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f9496c1e740 1 -- 172.21.15.78:0/1385317004 wait complete. 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f9496c1e740 20 client.17718 trim_cache size 0 max 16384 2024-04-25T14:06:52.576 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:06:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:06:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "93e22f09-cf36-4805-9d3d-5b4d770f4e08", 2024-04-25T14:06:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:06:52.577 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:06:52.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94740afb70 msgr2=0x7f94740b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94740afb70 0x7f94740b2080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f94800c3370 tx=0x7f94800f5a40 comp rx=0 tx=0).stop 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 msgr2=0x7f94881a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f94881a16e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9480009230 tx=0x7f9480031a50 comp rx=0 tx=0).stop 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 shutdown_connections 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f94740afb70 0x7f94740b2080 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9488109a40 0x7f94881a16e0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9488104d40 0x7f948819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 --2- 172.21.15.78:0/2559362698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9488103b00 0x7f948819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 >> 172.21.15.78:0/2559362698 conn(0x7f94880fed10 msgr2=0x7f9488105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 shutdown_connections 2024-04-25T14:06:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.571+0000 7f948fd72640 1 -- 172.21.15.78:0/2559362698 wait complete. 2024-04-25T14:06:52.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:06:52.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:06:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 -- 172.21.15.78:0/345815932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 msgr2=0x7fe7e8105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 --2- 172.21.15.78:0/345815932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e8105140 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 -- 172.21.15.78:0/345815932 shutdown_connections 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 --2- 172.21.15.78:0/345815932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7e8109a30 0x7fe7e810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 --2- 172.21.15.78:0/345815932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e8105140 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 --2- 172.21.15.78:0/345815932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e8103af0 0x7fe7e81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 -- 172.21.15.78:0/345815932 >> 172.21.15.78:0/345815932 conn(0x7fe7e80fece0 msgr2=0x7fe7e8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 -- 172.21.15.78:0/345815932 shutdown_connections 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.692+0000 7fe7edf64640 1 -- 172.21.15.78:0/345815932 wait complete. 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 Processor -- start 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 -- start start 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e8103af0 0x7fe7e819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7e8109a30 0x7fe7e81a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e810f5f0 con 0x7fe7e8104d30 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e810f470 con 0x7fe7e8109a30 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7edf64640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e810f770 con 0x7fe7e8103af0 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55480/0 (socket says 172.21.15.78:55480) 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/398151573 learned_addr learned my addr 172.21.15.78:0/398151573 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7e8109a30 0x7fe7e81a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.693+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e8103af0 0x7fe7e819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/398151573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e8103af0 msgr2=0x7fe7e819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e8103af0 0x7fe7e819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/398151573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7e8109a30 msgr2=0x7fe7e81a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7e8109a30 0x7fe7e81a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/398151573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e81a1d30 con 0x7fe7e8104d30 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e819a730 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006010 tx=0x7fe7dc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e4ff9640 1 -- 172.21.15.78:0/398151573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc03f030 con 0x7fe7e8104d30 2024-04-25T14:06:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e4ff9640 1 -- 172.21.15.78:0/398151573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7dc003cb0 con 0x7fe7e8104d30 2024-04-25T14:06:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e81a1fc0 con 0x7fe7e8104d30 2024-04-25T14:06:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7e4ff9640 1 -- 172.21.15.78:0/398151573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc035070 con 0x7fe7e8104d30 2024-04-25T14:06:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.694+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e81a2420 con 0x7fe7e8104d30 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.696+0000 7fe7e4ff9640 1 -- 172.21.15.78:0/398151573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe7dc042030 con 0x7fe7e8104d30 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.696+0000 7fe7f4e11740 1 Processor -- start 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.696+0000 7fe7f4e11740 1 -- start start 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05ac4390 0x559b05e65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05a9b4d0 0x559b05e65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b05dcad80 con 0x559b05a9b4d0 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b05d848b0 con 0x559b05aa36b0 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559b05dc6d40 con 0x559b05ac4390 2024-04-25T14:06:52.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05a9b4d0 0x559b05e65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05a9b4d0 0x559b05e65fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55492/0 (socket says 172.21.15.78:55492) 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 -- 172.21.15.78:0/265890008 learned_addr learned my addr 172.21.15.78:0/265890008 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05ac4390 0x559b05e65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e4ff9640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe7d00afb70 0x7fe7d00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 -- 172.21.15.78:0/265890008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05ac4390 msgr2=0x559b05e65a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05ac4390 0x559b05e65a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 -- 172.21.15.78:0/265890008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 msgr2=0x559b05e66c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e66c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 -- 172.21.15.78:0/265890008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b05d55a40 con 0x559b05a9b4d0 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe7d00afb70 0x7fe7d00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e4ff9640 1 -- 172.21.15.78:0/398151573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe7dc0ff280 con 0x7fe7e8104d30 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05a9b4d0 0x559b05e65fe0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800bcd0 tx=0x7fe7d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da12a6d694ea8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.697+0000 7fe7cdffb640 1 -- 172.21.15.78:0/265890008 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8015070 con 0x559b05a9b4d0 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7cdffb640 1 -- 172.21.15.78:0/265890008 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d80054d0 con 0x559b05a9b4d0 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7cdffb640 1 -- 172.21.15.78:0/265890008 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8011440 con 0x559b05a9b4d0 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 -- 172.21.15.78:0/265890008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05a9b4d0 msgr2=0x559b05e65fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe7d00afb70 0x7fe7d00b2080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x559b05e66ab0 tx=0x7fe7e0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05a9b4d0 0x559b05e65fe0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800bcd0 tx=0x7fe7d800f730 comp rx=0 tx=0).stop 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 -- 172.21.15.78:0/265890008 shutdown_connections 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05a9b4d0 0x559b05e65fe0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/265890008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05ac4390 0x559b05e65a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 -- 172.21.15.78:0/265890008 >> 172.21.15.78:0/265890008 conn(0x559b05ab84e0 msgr2=0x559b05e61270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 -- 172.21.15.78:0/265890008 shutdown_connections 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.698+0000 7fe7f4e11740 1 -- 172.21.15.78:0/265890008 wait complete. 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 -- start start 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 Processor -- start 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e6f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e6fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e76da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b05e67780 con 0x559b05aa36b0 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559b05d51d10 con 0x559b05ac4390 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559b05e67900 con 0x559b05a9b4d0 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7f4e11740 10 client.0 _ll_register_callbacks cb 0x559b05d7e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e6fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e6fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55494/0 (socket says 172.21.15.78:55494) 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/2974808931 learned_addr learned my addr 172.21.15.78:0/2974808931 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.699+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e6f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/2974808931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 msgr2=0x559b05e6f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e6f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e76da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 msgr2=0x559b05e76da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e76da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/2974808931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b05ba67d0 con 0x559b05aa36b0 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e76da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e6fe40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0c37d0 tx=0x7fe7dc0c2f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef4efff29a0417c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/2974808931 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc049070 con 0x559b05aa36b0 2024-04-25T14:06:52.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/2974808931 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe7dc0b00f0 con 0x559b05aa36b0 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d80054d0 con 0x559b05aa36b0 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/2974808931 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc0c6ec0 con 0x559b05aa36b0 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7f4e11740 10 client.17730 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559b05adc3f0 con 0x559b05aa36b0 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/2974808931 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d80054d0 con 0x559b05aa36b0 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.700+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/2974808931 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe7dc03bc20 con 0x559b05aa36b0 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7f4e11740 10 client.17730 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7f4e11740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7f4e11740 20 client.17730 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a82f60 0x559b05e7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7f4e11740 4 client.17730 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b05dd8990 con 0x559b05a82f60 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a82f60 0x559b05e7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7c6ffd640 10 client.17730 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.701+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a82f60 0x559b05e7d240 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda0cb01185f42a3 server_cookie=8f8aac29f431a0ec in_seq=0 out_seq=0 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/2974808931 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b05dd8990 con 0x559b05a82f60 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7c6ffd640 10 client.17730 handle_command_reply: tid=0 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 client.17730 shutdown 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 msgr2=0x559b05e6fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e6fe40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0c37d0 tx=0x7fe7dc0c2f70 comp rx=0 tx=0).stop 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 shutdown_connections 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e76da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e6fe40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a82f60 0x559b05e7d240 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/2974808931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e6f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 >> 172.21.15.78:0/2974808931 conn(0x559b05ab84e0 msgr2=0x559b05e64180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7c6ffd640 0 client.17730 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 shutdown_connections 2024-04-25T14:06:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 1 -- 172.21.15.78:0/2974808931 wait complete. 2024-04-25T14:06:52.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.702+0000 7fe7f4e11740 20 client.17730 trim_cache size 0 max 16384 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 Processor -- start 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 -- start start 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e60ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05ac4390 0x559b05e6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e81adae0 con 0x559b05ac4390 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e81ab3b0 con 0x559b05aa36b0 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e81a9200 con 0x559b05a9b4d0 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05ac4390 0x559b05e6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05ac4390 0x559b05e6df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55508/0 (socket says 172.21.15.78:55508) 2024-04-25T14:06:52.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.720+0000 7fe7e7fff640 1 -- 172.21.15.78:0/3671452945 learned_addr learned my addr 172.21.15.78:0/3671452945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e60ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e7fff640 1 -- 172.21.15.78:0/3671452945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 msgr2=0x559b05e60ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e60ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e7fff640 1 -- 172.21.15.78:0/3671452945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 msgr2=0x559b05e611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e7fff640 1 -- 172.21.15.78:0/3671452945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b05a9c0f0 con 0x559b05ac4390 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05ac4390 0x559b05e6df70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7e001e7b0 tx=0x7fe7e0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1183fb1f9a4601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7cd7fa640 1 -- 172.21.15.78:0/3671452945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e001ee60 con 0x559b05ac4390 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7cd7fa640 1 -- 172.21.15.78:0/3671452945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e001e220 con 0x559b05ac4390 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.721+0000 7fe7cd7fa640 1 -- 172.21.15.78:0/3671452945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e001da70 con 0x559b05ac4390 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 -- 172.21.15.78:0/3671452945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05ac4390 msgr2=0x559b05e6df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05ac4390 0x559b05e6df70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7e001e7b0 tx=0x7fe7e0015000 comp rx=0 tx=0).stop 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 -- 172.21.15.78:0/3671452945 shutdown_connections 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05ac4390 0x559b05e6df70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05aa36b0 0x559b05e611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/3671452945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e60ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 -- 172.21.15.78:0/3671452945 >> 172.21.15.78:0/3671452945 conn(0x559b05ab84e0 msgr2=0x559b05e659e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 -- 172.21.15.78:0/3671452945 shutdown_connections 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 -- 172.21.15.78:0/3671452945 wait complete. 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 -- start start 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.722+0000 7fe7f4e11740 1 Processor -- start 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e73880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e73dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e74cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e81a9870 con 0x559b05aa36b0 2024-04-25T14:06:52.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e81aace0 con 0x559b05ac4390 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e81a9f40 con 0x559b05a9b4d0 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 10 client.0 _ll_register_callbacks cb 0x7fe7dc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e73dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e73dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55522/0 (socket says 172.21.15.78:55522) 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/828469198 learned_addr learned my addr 172.21.15.78:0/828469198 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e7fff640 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e74cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/828469198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 msgr2=0x559b05e73880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e73880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 msgr2=0x559b05e74cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e74cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 -- 172.21.15.78:0/828469198 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b05d8b4b0 con 0x559b05aa36b0 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7e6ffd640 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e73dc0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0c49c0 tx=0x7fe7dc0b0320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8d4fb8fca5135d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:52.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/828469198 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc049040 con 0x559b05aa36b0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/828469198 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe7dc0d0120 con 0x559b05aa36b0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e001e220 con 0x559b05aa36b0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/828469198 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc0ea0b0 con 0x559b05aa36b0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 10 client.17736 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.723+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559b05e76410 con 0x559b05aa36b0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/828469198 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e001e220 con 0x559b05aa36b0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/828469198 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe7dc0dcba0 con 0x559b05aa36b0 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7f4e11740 10 client.17736 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7f4e11740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7f4e11740 20 client.17736 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a583a0 0x559b05ee78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7f4e11740 4 client.17736 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b05dd2800 con 0x559b05a583a0 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a583a0 0x559b05ee78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7c6ffd640 10 client.17736 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.724+0000 7fe7e77fe640 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a583a0 0x559b05ee78f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85eafdddf5d85115 server_cookie=a936ffc70ecb7f49 in_seq=0 out_seq=0 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7c6ffd640 1 -- 172.21.15.78:0/828469198 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559b05dd2800 con 0x559b05a583a0 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7c6ffd640 10 client.17736 handle_command_reply: tid=0 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 client.17736 shutdown 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 msgr2=0x559b05e73dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e73dc0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0c49c0 tx=0x7fe7dc0b0320 comp rx=0 tx=0).stop 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 shutdown_connections 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559b05ac4390 0x559b05e74cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b05aa36b0 0x559b05e73dc0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559b05a583a0 0x559b05ee78f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 --2- 172.21.15.78:0/828469198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559b05a9b4d0 0x559b05e73880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7c6ffd640 0 client.17736 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 >> 172.21.15.78:0/828469198 conn(0x559b05ab84e0 msgr2=0x559b05e60c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 shutdown_connections 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 1 -- 172.21.15.78:0/828469198 wait complete. 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7f4e11740 20 client.17736 trim_cache size 0 max 16384 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stdout: "93e22f09-cf36-4805-9d3d-5b4d770f4e08": { 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "93e22f09-cf36-4805-9d3d-5b4d770f4e08", 2024-04-25T14:06:52.732 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:06:52.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:06:52.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:06:52.733 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:06:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe7d00afb70 msgr2=0x7fe7d00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7edf64640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe7d00afb70 0x7fe7d00b2080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x559b05e66ab0 tx=0x7fe7e0004a40 comp rx=0 tx=0).stop 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 msgr2=0x7fe7e819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.725+0000 7fe7edf64640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e819a730 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006010 tx=0x7fe7dc004340 comp rx=0 tx=0).stop 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 shutdown_connections 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7e8109a30 0x7fe7e81a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe7d00afb70 0x7fe7d00b2080 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e8104d30 0x7fe7e819a730 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 --2- 172.21.15.78:0/398151573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e8103af0 0x7fe7e819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 >> 172.21.15.78:0/398151573 conn(0x7fe7e80fece0 msgr2=0x7fe7e8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 shutdown_connections 2024-04-25T14:06:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:06:52.726+0000 7fe7edf64640 1 -- 172.21.15.78:0/398151573 wait complete. 2024-04-25T14:06:52.736 INFO:tasks.cephfs.filesystem:scrub status for tag:93e22f09-cf36-4805-9d3d-5b4d770f4e08 - {'path': '/', 'tag': '93e22f09-cf36-4805-9d3d-5b4d770f4e08', 'options': 'recursive,force'} 2024-04-25T14:06:53.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:53 smithi129 ceph-mon[32569]: pgmap v569: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:06:53.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:53 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:06:53.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:53 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:53.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:53 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:06:53.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:53 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:53 smithi078 ceph-mon[29163]: pgmap v569: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:06:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:53 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:06:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:53 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:53 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:06:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:53 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:53.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:53 smithi112 ceph-mon[32617]: pgmap v569: 97 pgs: 97 active+clean; 1.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:06:53.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:53 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:06:53.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:53 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:53.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:53 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:06:53.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:53 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:06:55.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:55 smithi129 ceph-mon[32569]: pgmap v570: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:06:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:55 smithi078 ceph-mon[29163]: pgmap v570: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:06:55.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:55 smithi112 ceph-mon[32617]: pgmap v570: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:06:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:56 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:06:56.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:56 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:06:56.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:56 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:06:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:57 smithi078 ceph-mon[29163]: pgmap v571: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:57.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:57 smithi112 ceph-mon[32617]: pgmap v571: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 -- 172.21.15.78:0/79838782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101e90 msgr2=0x7fc08c106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 --2- 172.21.15.78:0/79838782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101e90 0x7fc08c106b50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009900 tx=0x7fc07c02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 -- 172.21.15.78:0/79838782 shutdown_connections 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 --2- 172.21.15.78:0/79838782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc08c1097b0 0x7fc08c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 --2- 172.21.15.78:0/79838782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc08c102390 0x7fc08c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 --2- 172.21.15.78:0/79838782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101e90 0x7fc08c106b50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 -- 172.21.15.78:0/79838782 >> 172.21.15.78:0/79838782 conn(0x7fc08c0fd3a0 msgr2=0x7fc08c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 -- 172.21.15.78:0/79838782 shutdown_connections 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.838+0000 7fc093814640 1 -- 172.21.15.78:0/79838782 wait complete. 2024-04-25T14:06:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 Processor -- start 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 -- start start 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc08c101e90 0x7fc08c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc08c102390 0x7fc08c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1097b0 0x7fc08c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10f610 con 0x7fc08c1097b0 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10f490 con 0x7fc08c102390 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc093814640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10f790 con 0x7fc08c101e90 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc091db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1097b0 0x7fc08c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc091db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1097b0 0x7fc08c1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49262/0 (socket says 172.21.15.78:49262) 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc091db1640 1 -- 172.21.15.78:0/2234920159 learned_addr learned my addr 172.21.15.78:0/2234920159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc0915b0640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc08c101e90 0x7fc08c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.839+0000 7fc090daf640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc08c102390 0x7fc08c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc091db1640 1 -- 172.21.15.78:0/2234920159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc08c101e90 msgr2=0x7fc08c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc091db1640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc08c101e90 0x7fc08c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc091db1640 1 -- 172.21.15.78:0/2234920159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc08c102390 msgr2=0x7fc08c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc091db1640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc08c102390 0x7fc08c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc091db1640 1 -- 172.21.15.78:0/2234920159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c1a1d50 con 0x7fc08c1097b0 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc091db1640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1097b0 0x7fc08c1a16e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc08400eaa0 tx=0x7fc08400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc0827fc640 1 -- 172.21.15.78:0/2234920159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08400cdb0 con 0x7fc08c1097b0 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc0827fc640 1 -- 172.21.15.78:0/2234920159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084005020 con 0x7fc08c1097b0 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c1a1fe0 con 0x7fc08c1097b0 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc0827fc640 1 -- 172.21.15.78:0/2234920159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840116d0 con 0x7fc08c1097b0 2024-04-25T14:06:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.840+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a2450 con 0x7fc08c1097b0 2024-04-25T14:06:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.843+0000 7fc09a79c740 1 Processor -- start 2024-04-25T14:06:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.843+0000 7fc0827fc640 1 -- 172.21.15.78:0/2234920159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc0840051d0 con 0x7fc08c1097b0 2024-04-25T14:06:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.843+0000 7fc09a79c740 1 -- start start 2024-04-25T14:06:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc86970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc8ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc09a79c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2bbbec460 con 0x55a2bb8e61b0 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc09a79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2bbba6590 con 0x55a2bb8c54d0 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc09a79c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a2bbbe8420 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc86970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc86970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49274/0 (socket says 172.21.15.78:49274) 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 -- 172.21.15.78:0/1017815798 learned_addr learned my addr 172.21.15.78:0/1017815798 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc0915b0640 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc0827fc640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc0780afae0 0x7fc0780b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 -- 172.21.15.78:0/1017815798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8bd2f0 msgr2=0x55a2bbc86ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc86ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 -- 172.21.15.78:0/1017815798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 msgr2=0x55a2bbc8ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc8ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 -- 172.21.15.78:0/1017815798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bbbb30d0 con 0x55a2bb8e61b0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc091db1640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc0780afae0 0x7fc0780b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc0827fc640 1 -- 172.21.15.78:0/2234920159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc0840ded40 con 0x7fc08c1097b0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc090daf640 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc86970 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc088009d40 tx=0x7fc088013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7516ad20f327aee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.844+0000 7fc0777fe640 1 -- 172.21.15.78:0/1017815798 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088004050 con 0x55a2bb8e61b0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc0777fe640 1 -- 172.21.15.78:0/1017815798 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc088017030 con 0x55a2bb8e61b0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc0777fe640 1 -- 172.21.15.78:0/1017815798 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088010430 con 0x55a2bb8e61b0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 -- 172.21.15.78:0/1017815798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8e61b0 msgr2=0x55a2bbc86970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc091db1640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc0780afae0 0x7fc0780b1ff0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0840a3340 tx=0x7fc08409bc00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc86970 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc088009d40 tx=0x7fc088013830 comp rx=0 tx=0).stop 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 -- 172.21.15.78:0/1017815798 shutdown_connections 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc8ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc86ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1017815798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc86970 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 -- 172.21.15.78:0/1017815798 >> 172.21.15.78:0/1017815798 conn(0x55a2bb8da300 msgr2=0x55a2bbc82300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 -- 172.21.15.78:0/1017815798 shutdown_connections 2024-04-25T14:06:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.845+0000 7fc09a79c740 1 -- 172.21.15.78:0/1017815798 wait complete. 2024-04-25T14:06:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 -- start start 2024-04-25T14:06:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 Processor -- start 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc90be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2bbc8e840 con 0x55a2bb8c54d0 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a2bbb73b60 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a2bbc8e9c0 con 0x55a2bb8e61b0 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc09a79c740 10 client.0 _ll_register_callbacks cb 0x55a2bbb982b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc0915b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc90be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc0915b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc90be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49282/0 (socket says 172.21.15.78:49282) 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.846+0000 7fc0915b0640 1 -- 172.21.15.78:0/3112461715 learned_addr learned my addr 172.21.15.78:0/3112461715 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc090daf640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc090daf640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc0915b0640 1 -- 172.21.15.78:0/3112461715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8e61b0 msgr2=0x55a2bbc97ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc0915b0640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc97ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc0915b0640 1 -- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8bd2f0 msgr2=0x55a2bbc906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc0915b0640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc906a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc0915b0640 1 -- 172.21.15.78:0/3112461715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bb8fe210 con 0x55a2bb8c54d0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc090daf640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc0915b0640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc90be0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc07c02a930 tx=0x7fc07c031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca2f062881d392e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc074ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc074ff9640 1 -- 172.21.15.78:0/3112461715 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c035040 con 0x55a2bb8c54d0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc074ff9640 1 -- 172.21.15.78:0/3112461715 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc07c004240 con 0x55a2bb8c54d0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc088017030 con 0x55a2bb8c54d0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc074ff9640 1 -- 172.21.15.78:0/3112461715 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc07c0318f0 con 0x55a2bb8c54d0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc09a79c740 10 client.17748 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a2bb9ca620 con 0x55a2bb8c54d0 2024-04-25T14:06:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.847+0000 7fc074ff9640 1 -- 172.21.15.78:0/3112461715 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc088017030 con 0x55a2bb8c54d0 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc074ff9640 1 -- 172.21.15.78:0/3112461715 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc07c044020 con 0x55a2bb8c54d0 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc09a79c740 10 client.17748 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc09a79c740 10 client.17748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc09a79c740 20 client.17748 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc09a79c740 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8a4d80 0x55a2bbc9e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc09a79c740 4 client.17748 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2bbbfa070 con 0x55a2bb8a4d80 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc090daf640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8a4d80 0x55a2bbc9e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc074ff9640 10 client.17748 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.848+0000 7fc090daf640 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8a4d80 0x55a2bbc9e320 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96623763bae5d0f server_cookie=9d74480a047c5df1 in_seq=0 out_seq=0 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc074ff9640 1 -- 172.21.15.78:0/3112461715 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2bbbfa070 con 0x55a2bb8a4d80 2024-04-25T14:06:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc074ff9640 10 client.17748 handle_command_reply: tid=0 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 client.17748 shutdown 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8c54d0 msgr2=0x55a2bbc90be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc90be0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc07c02a930 tx=0x7fc07c031e80 comp rx=0 tx=0).stop 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 shutdown_connections 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8a4d80 0x55a2bbc9e320 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8e61b0 0x55a2bbc97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc90be0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc074ff9640 0 client.17748 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 --2- 172.21.15.78:0/3112461715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 >> 172.21.15.78:0/3112461715 conn(0x55a2bb8da300 msgr2=0x55a2bbc85210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 shutdown_connections 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.849+0000 7fc09a79c740 1 -- 172.21.15.78:0/3112461715 wait complete. 2024-04-25T14:06:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.850+0000 7fc09a79c740 20 client.17748 trim_cache size 0 max 16384 2024-04-25T14:06:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.864+0000 7fc09a79c740 1 Processor -- start 2024-04-25T14:06:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.864+0000 7fc09a79c740 1 -- start start 2024-04-25T14:06:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc8ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc8f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc09a79c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1adb70 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc09a79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1ab440 con 0x55a2bb8c54d0 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc09a79c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1a9290 con 0x55a2bb8a4d80 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc8f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc8f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49292/0 (socket says 172.21.15.78:49292) 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 -- 172.21.15.78:0/1392000738 learned_addr learned my addr 172.21.15.78:0/1392000738 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc090daf640 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 -- 172.21.15.78:0/1392000738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8a4d80 msgr2=0x55a2bbc8ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc090daf640 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc8ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc8ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 -- 172.21.15.78:0/1392000738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 msgr2=0x55a2bbc979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 -- 172.21.15.78:0/1392000738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bbc97f30 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc090daf640 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc8ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc0915b0640 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc8f2a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc07c046740 tx=0x7fc07c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112422b9f2378b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc076ffd640 1 -- 172.21.15.78:0/1392000738 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c035040 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc076ffd640 1 -- 172.21.15.78:0/1392000738 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c046440 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.865+0000 7fc076ffd640 1 -- 172.21.15.78:0/1392000738 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c000e60 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 -- 172.21.15.78:0/1392000738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 msgr2=0x55a2bbc8f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc8f2a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc07c046740 tx=0x7fc07c02a410 comp rx=0 tx=0).stop 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 -- 172.21.15.78:0/1392000738 shutdown_connections 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8c54d0 0x55a2bbc979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc8f2a0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 --2- 172.21.15.78:0/1392000738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc8ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 -- 172.21.15.78:0/1392000738 >> 172.21.15.78:0/1392000738 conn(0x55a2bb8da300 msgr2=0x55a2bbc86930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 -- 172.21.15.78:0/1392000738 shutdown_connections 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.866+0000 7fc09a79c740 1 -- 172.21.15.78:0/1392000738 wait complete. 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 -- start start 2024-04-25T14:06:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 Processor -- start 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc96b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc97090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8c54d0 0x55a2bbd047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1a9900 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1aad70 con 0x55a2bb8a4d80 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1a9fd0 con 0x55a2bb8c54d0 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc09a79c740 10 client.0 _ll_register_callbacks cb 0x7fc07c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc0915b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc97090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc0915b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc97090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49300/0 (socket says 172.21.15.78:49300) 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc0915b0640 1 -- 172.21.15.78:0/664313006 learned_addr learned my addr 172.21.15.78:0/664313006 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc090daf640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8c54d0 0x55a2bbd047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc090daf640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc96b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc0915b0640 1 -- 172.21.15.78:0/664313006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8c54d0 msgr2=0x55a2bbd047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc0915b0640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8c54d0 0x55a2bbd047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc0915b0640 1 -- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8a4d80 msgr2=0x55a2bbc96b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.867+0000 7fc0915b0640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc96b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc0915b0640 1 -- 172.21.15.78:0/664313006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bbd04d60 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc090daf640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc96b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc0915b0640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc97090 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009850 tx=0x7fc07c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23edc5498eb04be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc074ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc074ff9640 1 -- 172.21.15.78:0/664313006 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c035070 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc074ff9640 1 -- 172.21.15.78:0/664313006 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc07c051670 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2bbbf3ee0 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc074ff9640 1 -- 172.21.15.78:0/664313006 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc07c051870 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 10 client.17754 fetch_fsmap learned FSMap version 22 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a2bbd05f60 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc074ff9640 1 -- 172.21.15.78:0/664313006 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a2bbbf3ee0 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc074ff9640 1 -- 172.21.15.78:0/664313006 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc07c044040 con 0x55a2bb8bd2f0 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 10 client.17754 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 10 client.17754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 20 client.17754 populate_metadata read hostname 'smithi078' 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8e61b0 0x55a2bbd08890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 4 client.17754 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.868+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2bbb6f5f0 con 0x55a2bb8e61b0 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.869+0000 7fc090daf640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8e61b0 0x55a2bbd08890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.869+0000 7fc090daf640 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8e61b0 0x55a2bbd08890 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b83ad10d5d5f20 server_cookie=4906abe005e2a041 in_seq=0 out_seq=0 2024-04-25T14:06:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.869+0000 7fc074ff9640 10 client.17754 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc074ff9640 1 -- 172.21.15.78:0/664313006 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2bbb6f5f0 con 0x55a2bb8e61b0 2024-04-25T14:06:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc074ff9640 10 client.17754 handle_command_reply: tid=0 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 client.17754 shutdown 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 msgr2=0x55a2bbc97090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc97090 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009850 tx=0x7fc07c005fd0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 shutdown_connections 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a2bb8e61b0 0x55a2bbd08890 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a2bb8c54d0 0x55a2bbd047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2bb8bd2f0 0x55a2bbc97090 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 --2- 172.21.15.78:0/664313006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a2bb8a4d80 0x55a2bbc96b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc074ff9640 0 client.17754 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 >> 172.21.15.78:0/664313006 conn(0x55a2bb8da300 msgr2=0x55a2bbc868f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 shutdown_connections 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 1 -- 172.21.15.78:0/664313006 wait complete. 2024-04-25T14:06:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc09a79c740 20 client.17754 trim_cache size 0 max 16384 2024-04-25T14:06:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:06:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:06:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:06:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc0780afae0 msgr2=0x7fc0780b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc0780afae0 0x7fc0780b1ff0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc0840a3340 tx=0x7fc08409bc00 comp rx=0 tx=0).stop 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1097b0 msgr2=0x7fc08c1a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1097b0 0x7fc08c1a16e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc08400eaa0 tx=0x7fc08400ef60 comp rx=0 tx=0).stop 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 shutdown_connections 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc0780afae0 0x7fc0780b1ff0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1097b0 0x7fc08c1a16e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc08c102390 0x7fc08c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 --2- 172.21.15.78:0/2234920159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc08c101e90 0x7fc08c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:06:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 >> 172.21.15.78:0/2234920159 conn(0x7fc08c0fd3a0 msgr2=0x7fc08c106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:06:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 shutdown_connections 2024-04-25T14:06:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:06:57.884+0000 7fc093814640 1 -- 172.21.15.78:0/2234920159 wait complete. 2024-04-25T14:06:57.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:57 smithi129 ceph-mon[32569]: pgmap v571: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:06:58.475 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.475 INFO:tasks.workunit.client.0.smithi078.stdout:313/600: open fd = 3 2024-04-25T14:06:58.475 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.475 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.475 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.476 INFO:tasks.workunit.client.0.smithi078.stdout:314/600: open fd = 3 2024-04-25T14:06:58.476 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.476 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.476 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.476 INFO:tasks.workunit.client.0.smithi078.stdout:315/600: open fd = 3 2024-04-25T14:06:58.476 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.477 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.477 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.477 INFO:tasks.workunit.client.0.smithi078.stdout:316/600: open fd = 3 2024-04-25T14:06:58.477 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.477 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.477 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.478 INFO:tasks.workunit.client.0.smithi078.stdout:317/600: open fd = 3 2024-04-25T14:06:58.478 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.478 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.478 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.478 INFO:tasks.workunit.client.0.smithi078.stdout:318/600: open fd = 3 2024-04-25T14:06:58.478 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.479 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.479 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.479 INFO:tasks.workunit.client.0.smithi078.stdout:319/600: open fd = 3 2024-04-25T14:06:58.479 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.479 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.479 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.480 INFO:tasks.workunit.client.0.smithi078.stdout:320/600: open fd = 3 2024-04-25T14:06:58.480 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.480 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.480 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.480 INFO:tasks.workunit.client.0.smithi078.stdout:321/600: open fd = 3 2024-04-25T14:06:58.481 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.481 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.481 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.481 INFO:tasks.workunit.client.0.smithi078.stdout:322/600: open fd = 3 2024-04-25T14:06:58.481 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.481 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.481 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.482 INFO:tasks.workunit.client.0.smithi078.stdout:323/600: open fd = 3 2024-04-25T14:06:58.482 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.482 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.482 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.482 INFO:tasks.workunit.client.0.smithi078.stdout:324/600: open fd = 3 2024-04-25T14:06:58.482 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.483 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.483 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.483 INFO:tasks.workunit.client.0.smithi078.stdout:325/600: open fd = 3 2024-04-25T14:06:58.483 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.483 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.483 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.484 INFO:tasks.workunit.client.0.smithi078.stdout:326/600: open fd = 3 2024-04-25T14:06:58.484 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.484 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.484 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.484 INFO:tasks.workunit.client.0.smithi078.stdout:327/600: open fd = 3 2024-04-25T14:06:58.485 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.485 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.485 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.485 INFO:tasks.workunit.client.0.smithi078.stdout:328/600: open fd = 3 2024-04-25T14:06:58.485 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.485 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.486 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.486 INFO:tasks.workunit.client.0.smithi078.stdout:329/600: open fd = 3 2024-04-25T14:06:58.486 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.486 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.486 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.486 INFO:tasks.workunit.client.0.smithi078.stdout:330/600: open fd = 3 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:331/600: open fd = 3 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.487 INFO:tasks.workunit.client.0.smithi078.stdout:332/600: open fd = 3 2024-04-25T14:06:58.488 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.488 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.488 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.488 INFO:tasks.workunit.client.0.smithi078.stdout:333/600: open fd = 3 2024-04-25T14:06:58.488 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.488 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.488 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.489 INFO:tasks.workunit.client.0.smithi078.stdout:334/600: open fd = 3 2024-04-25T14:06:58.489 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.489 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.489 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.489 INFO:tasks.workunit.client.0.smithi078.stdout:335/600: open fd = 3 2024-04-25T14:06:58.489 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.490 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.490 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.490 INFO:tasks.workunit.client.0.smithi078.stdout:336/600: open fd = 3 2024-04-25T14:06:58.490 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.490 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.490 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.491 INFO:tasks.workunit.client.0.smithi078.stdout:337/600: open fd = 3 2024-04-25T14:06:58.491 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.491 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.491 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.491 INFO:tasks.workunit.client.0.smithi078.stdout:338/600: open fd = 3 2024-04-25T14:06:58.491 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.492 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.492 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.492 INFO:tasks.workunit.client.0.smithi078.stdout:339/600: open fd = 3 2024-04-25T14:06:58.492 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.492 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.492 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.493 INFO:tasks.workunit.client.0.smithi078.stdout:340/600: open fd = 3 2024-04-25T14:06:58.493 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.493 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.493 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.493 INFO:tasks.workunit.client.0.smithi078.stdout:341/600: open fd = 3 2024-04-25T14:06:58.493 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.494 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.494 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.494 INFO:tasks.workunit.client.0.smithi078.stdout:342/600: open fd = 3 2024-04-25T14:06:58.494 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.494 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.494 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.495 INFO:tasks.workunit.client.0.smithi078.stdout:343/600: open fd = 3 2024-04-25T14:06:58.495 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.495 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.495 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.495 INFO:tasks.workunit.client.0.smithi078.stdout:344/600: open fd = 3 2024-04-25T14:06:58.495 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.496 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.496 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.496 INFO:tasks.workunit.client.0.smithi078.stdout:345/600: open fd = 3 2024-04-25T14:06:58.496 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.496 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.496 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.497 INFO:tasks.workunit.client.0.smithi078.stdout:346/600: open fd = 3 2024-04-25T14:06:58.497 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.497 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.497 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.497 INFO:tasks.workunit.client.0.smithi078.stdout:347/600: open fd = 3 2024-04-25T14:06:58.497 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.498 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.498 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.498 INFO:tasks.workunit.client.0.smithi078.stdout:348/600: open fd = 3 2024-04-25T14:06:58.498 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.498 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.499 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.499 INFO:tasks.workunit.client.0.smithi078.stdout:349/600: open fd = 3 2024-04-25T14:06:58.499 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.499 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.499 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.499 INFO:tasks.workunit.client.0.smithi078.stdout:350/600: open fd = 3 2024-04-25T14:06:58.500 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.500 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.500 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.500 INFO:tasks.workunit.client.0.smithi078.stdout:351/600: open fd = 3 2024-04-25T14:06:58.500 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.500 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.501 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.501 INFO:tasks.workunit.client.0.smithi078.stdout:352/600: open fd = 3 2024-04-25T14:06:58.501 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.501 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.501 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.501 INFO:tasks.workunit.client.0.smithi078.stdout:353/600: open fd = 3 2024-04-25T14:06:58.501 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.502 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.502 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.502 INFO:tasks.workunit.client.0.smithi078.stdout:354/600: open fd = 3 2024-04-25T14:06:58.502 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.502 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:355/600: open fd = 3 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:356/600: open fd = 3 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.503 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.504 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.504 INFO:tasks.workunit.client.0.smithi078.stdout:357/600: open fd = 3 2024-04-25T14:06:58.504 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.504 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.504 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.504 INFO:tasks.workunit.client.0.smithi078.stdout:358/600: open fd = 3 2024-04-25T14:06:58.504 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:359/600: open fd = 3 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:360/600: open fd = 3 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.505 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:361/600: open fd = 3 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:362/600: open fd = 3 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:363/600: open fd = 3 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.506 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:364/600: open fd = 3 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:365/600: open fd = 3 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:366/600: open fd = 3 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:367/600: open fd = 3 2024-04-25T14:06:58.507 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:368/600: open fd = 3 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:369/600: open fd = 3 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.508 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.509 INFO:tasks.workunit.client.0.smithi078.stdout:370/600: open fd = 3 2024-04-25T14:06:58.509 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.509 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.509 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.509 INFO:tasks.workunit.client.0.smithi078.stdout:371/600: open fd = 3 2024-04-25T14:06:58.509 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.509 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.510 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.510 INFO:tasks.workunit.client.0.smithi078.stdout:372/600: open fd = 3 2024-04-25T14:06:58.510 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.510 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.510 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.510 INFO:tasks.workunit.client.0.smithi078.stdout:373/600: open fd = 3 2024-04-25T14:06:58.510 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.511 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.511 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:58.511 INFO:tasks.workunit.client.0.smithi078.stdout:374/600: open fd = 3 2024-04-25T14:06:58.511 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.511 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:06:58.511 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:06:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:06:59 smithi078 ceph-mon[29163]: pgmap v572: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:06:59.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:06:59 smithi112 ceph-mon[32617]: pgmap v572: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:06:59.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:06:59 smithi129 ceph-mon[32569]: pgmap v572: 97 pgs: 97 active+clean; 1.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:01 smithi078 ceph-mon[29163]: pgmap v573: 97 pgs: 97 active+clean; 1.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:01.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:01 smithi112 ceph-mon[32617]: pgmap v573: 97 pgs: 97 active+clean; 1.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:01 smithi129 ceph-mon[32569]: pgmap v573: 97 pgs: 97 active+clean; 1.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:03 smithi078 ceph-mon[29163]: pgmap v574: 97 pgs: 97 active+clean; 1.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:03.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:03 smithi112 ceph-mon[32617]: pgmap v574: 97 pgs: 97 active+clean; 1.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:03.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:03 smithi129 ceph-mon[32569]: pgmap v574: 97 pgs: 97 active+clean; 1.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:05 smithi078 ceph-mon[29163]: pgmap v575: 97 pgs: 97 active+clean; 1.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:07:05.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:05 smithi112 ceph-mon[32617]: pgmap v575: 97 pgs: 97 active+clean; 1.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:07:05.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:05 smithi129 ceph-mon[32569]: pgmap v575: 97 pgs: 97 active+clean; 1.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:07:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:07 smithi078 ceph-mon[29163]: pgmap v576: 97 pgs: 97 active+clean; 1.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:07:07.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:07 smithi112 ceph-mon[32617]: pgmap v576: 97 pgs: 97 active+clean; 1.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:07:07.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:07 smithi129 ceph-mon[32569]: pgmap v576: 97 pgs: 97 active+clean; 1.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:07:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:09 smithi078 ceph-mon[29163]: pgmap v577: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:07:09.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:09 smithi112 ceph-mon[32617]: pgmap v577: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:07:09.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:09 smithi129 ceph-mon[32569]: pgmap v577: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:07:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 -- 172.21.15.78:0/3743754783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c102a30 msgr2=0x7f971c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 --2- 172.21.15.78:0/3743754783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c102a30 0x7f971c102f00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f970c009ca0 tx=0x7f970c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 -- 172.21.15.78:0/3743754783 shutdown_connections 2024-04-25T14:07:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 --2- 172.21.15.78:0/3743754783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f971c103590 0x7f971c103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 --2- 172.21.15.78:0/3743754783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c102a30 0x7f971c102f00 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 --2- 172.21.15.78:0/3743754783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f971c102050 0x7f971c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 -- 172.21.15.78:0/3743754783 >> 172.21.15.78:0/3743754783 conn(0x7f971c0fdc20 msgr2=0x7f971c1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 -- 172.21.15.78:0/3743754783 shutdown_connections 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.994+0000 7f972310e640 1 -- 172.21.15.78:0/3743754783 wait complete. 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 Processor -- start 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 -- start start 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f971c102050 0x7f971c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f971c102a30 0x7f971c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c103590 0x7f971c1a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f971c111830 con 0x7f971c103590 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1116b0 con 0x7f971c102a30 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f972310e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1119b0 con 0x7f971c102050 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c103590 0x7f971c1a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c103590 0x7f971c1a3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35180/0 (socket says 172.21.15.78:35180) 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 -- 172.21.15.78:0/4210459170 learned_addr learned my addr 172.21.15.78:0/4210459170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f9720eaa640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f971c102050 0x7f971c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f971bfff640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f971c102a30 0x7f971c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 -- 172.21.15.78:0/4210459170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f971c102050 msgr2=0x7f971c19c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f971c102050 0x7f971c19c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 -- 172.21.15.78:0/4210459170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f971c102a30 msgr2=0x7f971c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f971c102a30 0x7f971c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f97216ab640 1 -- 172.21.15.78:0/4210459170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c1a3f70 con 0x7f971c103590 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.995+0000 7f971bfff640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f971c102a30 0x7f971c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.996+0000 7f97216ab640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c103590 0x7f971c1a3900 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f971400d750 tx=0x7f971400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.996+0000 7f9719ffb640 1 -- 172.21.15.78:0/4210459170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714017070 con 0x7f971c103590 2024-04-25T14:07:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.996+0000 7f9719ffb640 1 -- 172.21.15.78:0/4210459170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140054d0 con 0x7f971c103590 2024-04-25T14:07:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.996+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c1a4260 con 0x7f971c103590 2024-04-25T14:07:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.996+0000 7f9719ffb640 1 -- 172.21.15.78:0/4210459170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714004c60 con 0x7f971c103590 2024-04-25T14:07:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.996+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c1a4760 con 0x7f971c103590 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.998+0000 7f9719ffb640 1 -- 172.21.15.78:0/4210459170 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9714009bd0 con 0x7f971c103590 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 Processor -- start 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 -- start start 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f19949a1b0 0x55f19983a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f19983aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9719ffb640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97080afae0 0x7f97080b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1994794d0 0x55f199841df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f1997a0460 con 0x55f1994712f0 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f19975a590 con 0x55f19949a1b0 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9729fbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f19979c420 con 0x55f1994794d0 2024-04-25T14:07:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f9720eaa640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97080afae0 0x7f97080b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f971bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f19983aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f97216ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1994794d0 0x55f199841df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f971bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f19983aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35192/0 (socket says 172.21.15.78:35192) 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f971bfff640 1 -- 172.21.15.78:0/3699379372 learned_addr learned my addr 172.21.15.78:0/3699379372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:10.999+0000 7f971bfff640 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f19949a1b0 0x55f19983a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f9719ffb640 1 -- 172.21.15.78:0/4210459170 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f97140de610 con 0x7f971c103590 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f97216ab640 1 -- 172.21.15.78:0/3699379372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f19949a1b0 msgr2=0x55f19983a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f97216ab640 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f19949a1b0 0x55f19983a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f97216ab640 1 -- 172.21.15.78:0/3699379372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 msgr2=0x55f19983aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f97216ab640 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f19983aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f97216ab640 1 -- 172.21.15.78:0/3699379372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1997670d0 con 0x55f1994794d0 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f971bfff640 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f19983aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f9720eaa640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97080afae0 0x7f97080b1ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9704005fd0 tx=0x7f970400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f97216ab640 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1994794d0 0x55f199841df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97140dc150 tx=0x7f971409b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af2f1ba9becee5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f9712ffd640 1 -- 172.21.15.78:0/3699379372 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714020070 con 0x55f1994794d0 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f9712ffd640 1 -- 172.21.15.78:0/3699379372 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140af290 con 0x55f1994794d0 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.000+0000 7f9712ffd640 1 -- 172.21.15.78:0/3699379372 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140aae90 con 0x55f1994794d0 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 -- 172.21.15.78:0/3699379372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1994794d0 msgr2=0x55f199841df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1994794d0 0x55f199841df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97140dc150 tx=0x7f971409b420 comp rx=0 tx=0).stop 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 -- 172.21.15.78:0/3699379372 shutdown_connections 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1994794d0 0x55f199841df0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f19983aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 --2- 172.21.15.78:0/3699379372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f19949a1b0 0x55f19983a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 -- 172.21.15.78:0/3699379372 >> 172.21.15.78:0/3699379372 conn(0x55f19948e300 msgr2=0x55f199836300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 -- 172.21.15.78:0/3699379372 shutdown_connections 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.001+0000 7f9729fbd740 1 -- 172.21.15.78:0/3699379372 wait complete. 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.002+0000 7f9729fbd740 1 -- start start 2024-04-25T14:07:11.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.002+0000 7f9729fbd740 1 Processor -- start 2024-04-25T14:07:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f199844770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994794d0 0x55f199844cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f19984bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9729fbd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f199842840 con 0x55f1994712f0 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9729fbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f199727b60 con 0x55f1994794d0 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9729fbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f1998429c0 con 0x55f19949a1b0 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9729fbd740 10 client.0 _ll_register_callbacks cb 0x55f19974c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f199844770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f199844770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35198/0 (socket says 172.21.15.78:35198) 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 -- 172.21.15.78:0/2650973690 learned_addr learned my addr 172.21.15.78:0/2650973690 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f97216ab640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f19984bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f971bfff640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994794d0 0x55f199844cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 -- 172.21.15.78:0/2650973690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 msgr2=0x55f19984bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f19984bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 -- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994794d0 msgr2=0x55f199844cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994794d0 0x55f199844cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 -- 172.21.15.78:0/2650973690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1994b2210 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f97216ab640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f19984bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f971bfff640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994794d0 0x55f199844cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f9720eaa640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f199844770 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f97040116c0 tx=0x7f9704004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f570c57727a8c74b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f96fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.003+0000 7f96fbfff640 1 -- 172.21.15.78:0/2650973690 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970401e8f0 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f96fbfff640 1 -- 172.21.15.78:0/2650973690 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f970401ea90 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97140af290 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f96fbfff640 1 -- 172.21.15.78:0/2650973690 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f970401ec90 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 10 client.17766 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f19957e620 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f96fbfff640 1 -- 172.21.15.78:0/2650973690 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140af290 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f96fbfff640 1 -- 172.21.15.78:0/2650973690 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9704026400 con 0x55f1994712f0 2024-04-25T14:07:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 10 client.17766 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 20 client.17766 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f199458d80 0x55f1998524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 4 client.17766 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1997ae070 con 0x55f199458d80 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.004+0000 7f971bfff640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f199458d80 0x55f1998524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.005+0000 7f971bfff640 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f199458d80 0x55f1998524e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f2fec2869ee8b4 server_cookie=b7df559cef60e02e in_seq=0 out_seq=0 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.005+0000 7f96fbfff640 10 client.17766 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f96fbfff640 1 -- 172.21.15.78:0/2650973690 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1997ae070 con 0x55f199458d80 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f96fbfff640 10 client.17766 handle_command_reply: tid=0 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 client.17766 shutdown 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 msgr2=0x55f199844770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f199844770 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f97040116c0 tx=0x7f9704004730 comp rx=0 tx=0).stop 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 shutdown_connections 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f199458d80 0x55f1998524e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f19984bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994794d0 0x55f199844cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2650973690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994712f0 0x55f199844770 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f96fbfff640 0 client.17766 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 >> 172.21.15.78:0/2650973690 conn(0x55f19948e300 msgr2=0x55f199835250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 shutdown_connections 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 1 -- 172.21.15.78:0/2650973690 wait complete. 2024-04-25T14:07:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.006+0000 7f9729fbd740 20 client.17766 trim_cache size 0 max 16384 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 Processor -- start 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 -- start start 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f1998434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f19984bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f199835be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1afd90 con 0x55f1994794d0 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1ad660 con 0x55f1994712f0 2024-04-25T14:07:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9729fbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1ab4b0 con 0x55f19949a1b0 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f971bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f19984bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f971bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f19984bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35200/0 (socket says 172.21.15.78:35200) 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f971bfff640 1 -- 172.21.15.78:0/1255272105 learned_addr learned my addr 172.21.15.78:0/1255272105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.021+0000 7f9720eaa640 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f1998434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f97216ab640 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f199835be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f971bfff640 1 -- 172.21.15.78:0/1255272105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 msgr2=0x55f199835be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f971bfff640 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f199835be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f971bfff640 1 -- 172.21.15.78:0/1255272105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 msgr2=0x55f1998434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f971bfff640 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f1998434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f971bfff640 1 -- 172.21.15.78:0/1255272105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19983f5f0 con 0x55f1994794d0 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f971bfff640 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f19984bba0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55f19983bd50 tx=0x7f970c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559ec6b0ff936096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f97127fc640 1 -- 172.21.15.78:0/1255272105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c04ae60 con 0x55f1994794d0 2024-04-25T14:07:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f97127fc640 1 -- 172.21.15.78:0/1255272105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970c04f070 con 0x55f1994794d0 2024-04-25T14:07:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.022+0000 7f97127fc640 1 -- 172.21.15.78:0/1255272105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c04a910 con 0x55f1994794d0 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 -- 172.21.15.78:0/1255272105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 msgr2=0x55f19984bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f19984bba0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55f19983bd50 tx=0x7f970c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 -- 172.21.15.78:0/1255272105 shutdown_connections 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f199835be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f19984bba0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 --2- 172.21.15.78:0/1255272105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f1998434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 -- 172.21.15.78:0/1255272105 >> 172.21.15.78:0/1255272105 conn(0x55f19948e300 msgr2=0x55f199842d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 -- 172.21.15.78:0/1255272105 shutdown_connections 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 -- 172.21.15.78:0/1255272105 wait complete. 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 -- start start 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.023+0000 7f9729fbd740 1 Processor -- start 2024-04-25T14:07:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f199848a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f199848fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9729fbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f1998b8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9729fbd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1abb20 con 0x55f1994794d0 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9729fbd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1acf90 con 0x55f1994712f0 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9729fbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1ac1f0 con 0x55f19949a1b0 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9729fbd740 10 client.0 _ll_register_callbacks cb 0x7f970400b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f199848fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f199848fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35202/0 (socket says 172.21.15.78:35202) 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 -- 172.21.15.78:0/2306712288 learned_addr learned my addr 172.21.15.78:0/2306712288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f97216ab640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f1998b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9720eaa640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f199848a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 -- 172.21.15.78:0/2306712288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 msgr2=0x55f1998b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f1998b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 -- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 msgr2=0x55f199848a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f199848a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 -- 172.21.15.78:0/2306712288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f199849610 con 0x55f1994794d0 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f9720eaa640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f199848a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f971bfff640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f199848fa0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f970c009ca0 tx=0x7f970c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c452d203252172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.024+0000 7f96fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f96fbfff640 1 -- 172.21.15.78:0/2306712288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c058060 con 0x55f1994794d0 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f96fbfff640 1 -- 172.21.15.78:0/2306712288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f970c04a690 con 0x55f1994794d0 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970c04f070 con 0x55f1994794d0 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f96fbfff640 1 -- 172.21.15.78:0/2306712288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f970c04a890 con 0x55f1994794d0 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 10 client.17772 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f1998b9eb0 con 0x55f1994794d0 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f96fbfff640 1 -- 172.21.15.78:0/2306712288 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970c04f070 con 0x55f1994794d0 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f96fbfff640 1 -- 172.21.15.78:0/2306712288 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f970c004310 con 0x55f1994794d0 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 10 client.17772 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 20 client.17772 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f19942e1c0 0x55f1998bc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 4 client.17772 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:07:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1997a7ee0 con 0x55f19942e1c0 2024-04-25T14:07:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.025+0000 7f9720eaa640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f19942e1c0 0x55f1998bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.026+0000 7f9720eaa640 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f19942e1c0 0x55f1998bc880 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6bb534c0dc8b02f server_cookie=cade1b44cc2b4fe1 in_seq=0 out_seq=0 2024-04-25T14:07:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.026+0000 7f96fbfff640 10 client.17772 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f96fbfff640 1 -- 172.21.15.78:0/2306712288 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1997a7ee0 con 0x55f19942e1c0 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f96fbfff640 10 client.17772 handle_command_reply: tid=0 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 client.17772 shutdown 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 msgr2=0x55f199848fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f199848fa0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f970c009ca0 tx=0x7f970c0098b0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 shutdown_connections 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f19942e1c0 0x55f1998bc880 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19949a1b0 0x55f1998b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f96fbfff640 0 client.17772 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1994794d0 0x55f199848fa0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 --2- 172.21.15.78:0/2306712288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1994712f0 0x55f199848a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 >> 172.21.15.78:0/2306712288 conn(0x55f19948e300 msgr2=0x55f199835be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 shutdown_connections 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 1 -- 172.21.15.78:0/2306712288 wait complete. 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.041+0000 7f9729fbd740 20 client.17772 trim_cache size 0 max 16384 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:07:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:07:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:07:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97080afae0 msgr2=0x7f97080b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97080afae0 0x7f97080b1ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9704005fd0 tx=0x7f970400b390 comp rx=0 tx=0).stop 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c103590 msgr2=0x7f971c1a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c103590 0x7f971c1a3900 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f971400d750 tx=0x7f971400dc10 comp rx=0 tx=0).stop 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 shutdown_connections 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f97080afae0 0x7f97080b1ff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f971c103590 0x7f971c1a3900 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f971c102a30 0x7f971c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 --2- 172.21.15.78:0/4210459170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f971c102050 0x7f971c19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 >> 172.21.15.78:0/4210459170 conn(0x7f971c0fdc20 msgr2=0x7f971c10a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 shutdown_connections 2024-04-25T14:07:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:11.042+0000 7f972310e640 1 -- 172.21.15.78:0/4210459170 wait complete. 2024-04-25T14:07:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:11 smithi078 ceph-mon[29163]: pgmap v578: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:11.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:11 smithi112 ceph-mon[32617]: pgmap v578: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:11.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:11 smithi129 ceph-mon[32569]: pgmap v578: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:13.716 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:13 smithi129 ceph-mon[32569]: pgmap v579: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:13.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:13 smithi078 ceph-mon[29163]: pgmap v579: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:13 smithi112 ceph-mon[32617]: pgmap v579: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:15.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:15 smithi078 ceph-mon[29163]: pgmap v580: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:07:15.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:15 smithi112 ceph-mon[32617]: pgmap v580: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:07:15.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:15 smithi129 ceph-mon[32569]: pgmap v580: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:07:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:17 smithi078 ceph-mon[29163]: pgmap v581: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:17.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:17 smithi112 ceph-mon[32617]: pgmap v581: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:17.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:17 smithi129 ceph-mon[32569]: pgmap v581: 97 pgs: 97 active+clean; 1.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:19 smithi078 ceph-mon[29163]: pgmap v582: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:19.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:19 smithi112 ceph-mon[32617]: pgmap v582: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:19.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:19 smithi129 ceph-mon[32569]: pgmap v582: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:21.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:21 smithi078 ceph-mon[29163]: pgmap v583: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:07:21.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:21 smithi112 ceph-mon[32617]: pgmap v583: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:07:21.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:21 smithi129 ceph-mon[32569]: pgmap v583: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:07:22.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:07:22.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:07:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 -- 172.21.15.78:0/3921734875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8109870 msgr2=0x7f16e810bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3921734875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8109870 0x7f16e810bc90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009ca0 tx=0x7f16e002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 -- 172.21.15.78:0/3921734875 shutdown_connections 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3921734875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8109870 0x7f16e810bc90 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3921734875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e8104b70 0x7f16e8104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3921734875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16e8103930 0x7f16e8104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 -- 172.21.15.78:0/3921734875 >> 172.21.15.78:0/3921734875 conn(0x7f16e80feb20 msgr2=0x7f16e8100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 -- 172.21.15.78:0/3921734875 shutdown_connections 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 -- 172.21.15.78:0/3921734875 wait complete. 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.861+0000 7f16ef19d640 1 Processor -- start 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ef19d640 1 -- start start 2024-04-25T14:07:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ef19d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8103930 0x7f16e819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ef19d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16e8104b70 0x7f16e819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ef19d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e8109870 0x7f16e81a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ef19d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810f460 con 0x7f16e8103930 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ef19d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810f2e0 con 0x7f16e8109870 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ef19d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810f5e0 con 0x7f16e8104b70 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8103930 0x7f16e819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8103930 0x7f16e819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38652/0 (socket says 172.21.15.78:38652) 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 -- 172.21.15.78:0/3646831412 learned_addr learned my addr 172.21.15.78:0/3646831412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ed99a640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16e8104b70 0x7f16e819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee99c640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e8109870 0x7f16e81a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 -- 172.21.15.78:0/3646831412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16e8104b70 msgr2=0x7f16e819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16e8104b70 0x7f16e819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 -- 172.21.15.78:0/3646831412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e8109870 msgr2=0x7f16e81a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e8109870 0x7f16e81a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 -- 172.21.15.78:0/3646831412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e81a1b40 con 0x7f16e8103930 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee99c640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e8109870 0x7f16e81a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16ee19b640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8103930 0x7f16e819a000 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f16e400c970 tx=0x7f16e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.862+0000 7f16df7fe640 1 -- 172.21.15.78:0/3646831412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4004220 con 0x7f16e8103930 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.863+0000 7f16df7fe640 1 -- 172.21.15.78:0/3646831412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e40043c0 con 0x7f16e8103930 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.863+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e81a1e30 con 0x7f16e8103930 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.863+0000 7f16df7fe640 1 -- 172.21.15.78:0/3646831412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4004a00 con 0x7f16e8103930 2024-04-25T14:07:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.863+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a2290 con 0x7f16e8103930 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.865+0000 7f16f738f740 1 Processor -- start 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.865+0000 7f16df7fe640 1 -- 172.21.15.78:0/3646831412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f16e40079b0 con 0x7f16e8103930 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.865+0000 7f16f738f740 1 -- start start 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16f738f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16f738f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16f738f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16f738f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561e54048d80 con 0x561e53d194d0 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16f738f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e540028b0 con 0x561e53d216b0 2024-04-25T14:07:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16f738f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561e54044d40 con 0x561e53d42390 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38666/0 (socket says 172.21.15.78:38666) 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 -- 172.21.15.78:0/3308332576 learned_addr learned my addr 172.21.15.78:0/3308332576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16df7fe640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f16d40afb60 0x7f16d40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ed99a640 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee99c640 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee99c640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f16d40afb60 0x7f16d40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16df7fe640 1 -- 172.21.15.78:0/3646831412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f16e40dea40 con 0x7f16e8103930 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 -- 172.21.15.78:0/3308332576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 msgr2=0x561e540e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540e3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 -- 172.21.15.78:0/3308332576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 msgr2=0x561e540e4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540e4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.866+0000 7f16ee19b640 1 -- 172.21.15.78:0/3308332576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e53fd3a40 con 0x561e53d194d0 2024-04-25T14:07:22.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.867+0000 7f16ee19b640 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e41d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f16e40a4180 tx=0x7f16e40052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2280a6984e7096a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.867+0000 7f16ee99c640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f16d40afb60 0x7f16d40b2070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f16e0005cc0 tx=0x7f16e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.867+0000 7f16dcbfe640 1 -- 172.21.15.78:0/3308332576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e401d070 con 0x561e53d194d0 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.867+0000 7f16dcbfe640 1 -- 172.21.15.78:0/3308332576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e40a6a20 con 0x561e53d194d0 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.867+0000 7f16dcbfe640 1 -- 172.21.15.78:0/3308332576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e40afd70 con 0x561e53d194d0 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 -- 172.21.15.78:0/3308332576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 msgr2=0x561e540e41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e41d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f16e40a4180 tx=0x7f16e40052e0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 -- 172.21.15.78:0/3308332576 shutdown_connections 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540e4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e41d0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 --2- 172.21.15.78:0/3308332576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 -- 172.21.15.78:0/3308332576 >> 172.21.15.78:0/3308332576 conn(0x561e53d364e0 msgr2=0x561e540df460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 -- 172.21.15.78:0/3308332576 shutdown_connections 2024-04-25T14:07:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 -- 172.21.15.78:0/3308332576 wait complete. 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 -- start start 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 Processor -- start 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d216b0 0x561e540ee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.868+0000 7f16f738f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d42390 0x561e540f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16f738f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561e540e5910 con 0x561e53d216b0 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16f738f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561e53fcfd10 con 0x561e53d42390 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16f738f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561e540e5a90 con 0x561e53d194d0 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16f738f740 10 client.0 _ll_register_callbacks cb 0x561e53ffc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d216b0 0x561e540ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d216b0 0x561e540ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38674/0 (socket says 172.21.15.78:38674) 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 -- 172.21.15.78:0/1562438235 learned_addr learned my addr 172.21.15.78:0/1562438235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:22.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ee19b640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ee99c640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d42390 0x561e540f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 -- 172.21.15.78:0/1562438235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 msgr2=0x561e540edaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540edaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 -- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d42390 msgr2=0x561e540f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d42390 0x561e540f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 -- 172.21.15.78:0/1562438235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e53e247d0 con 0x561e53d216b0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ee99c640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d42390 0x561e540f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ee19b640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16ed99a640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d216b0 0x561e540ee030 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f16d800bec0 tx=0x7f16d8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aad603fae7352a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16d1ffb640 1 -- 172.21.15.78:0/1562438235 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8010960 con 0x561e53d216b0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16d1ffb640 1 -- 172.21.15.78:0/1562438235 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f16d8010b00 con 0x561e53d216b0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e40a6a20 con 0x561e53d216b0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16d1ffb640 1 -- 172.21.15.78:0/1562438235 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f16d8010d00 con 0x561e53d216b0 2024-04-25T14:07:22.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16f738f740 10 client.17784 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.869+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561e53d5a3f0 con 0x561e53d216b0 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16d1ffb640 1 -- 172.21.15.78:0/1562438235 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e40a6a20 con 0x561e53d216b0 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16d1ffb640 1 -- 172.21.15.78:0/1562438235 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f16d8024050 con 0x561e53d216b0 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16f738f740 10 client.17784 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16f738f740 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16f738f740 20 client.17784 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16f738f740 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53d00f60 0x561e540fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16f738f740 4 client.17784 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e54056990 con 0x561e53d00f60 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16ee19b640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53d00f60 0x561e540fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16d1ffb640 10 client.17784 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.870+0000 7f16ee19b640 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53d00f60 0x561e540fb520 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d191f667c54bd985 server_cookie=5b6eb34013bb3688 in_seq=0 out_seq=0 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16d1ffb640 1 -- 172.21.15.78:0/1562438235 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e54056990 con 0x561e53d00f60 2024-04-25T14:07:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16d1ffb640 10 client.17784 handle_command_reply: tid=0 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 client.17784 shutdown 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d216b0 msgr2=0x561e540ee030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d216b0 0x561e540ee030 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f16d800bec0 tx=0x7f16d8012820 comp rx=0 tx=0).stop 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 shutdown_connections 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d42390 0x561e540f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d216b0 0x561e540ee030 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53d00f60 0x561e540fb520 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 --2- 172.21.15.78:0/1562438235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16d1ffb640 0 client.17784 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.871+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 >> 172.21.15.78:0/1562438235 conn(0x561e53d364e0 msgr2=0x561e540e2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.872+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 shutdown_connections 2024-04-25T14:07:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.872+0000 7f16f738f740 1 -- 172.21.15.78:0/1562438235 wait complete. 2024-04-25T14:07:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.872+0000 7f16f738f740 20 client.17784 trim_cache size 0 max 16384 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 Processor -- start 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 -- start start 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81ad970 con 0x561e53d194d0 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81ab240 con 0x561e53d216b0 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16f738f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a9090 con 0x561e53d42390 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38678/0 (socket says 172.21.15.78:38678) 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 -- 172.21.15.78:0/882531640 learned_addr learned my addr 172.21.15.78:0/882531640 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:22.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ed99a640 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee99c640 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 -- 172.21.15.78:0/882531640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 msgr2=0x561e540edaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540edaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 -- 172.21.15.78:0/882531640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 msgr2=0x561e540f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 -- 172.21.15.78:0/882531640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e540e8340 con 0x561e53d194d0 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee99c640 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.890+0000 7f16ee19b640 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e8ec0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f16e40dc580 tx=0x7f16e40a3690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66527e18ca60adc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16d3fff640 1 -- 172.21.15.78:0/882531640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e401d040 con 0x561e53d194d0 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16d3fff640 1 -- 172.21.15.78:0/882531640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e40a6a20 con 0x561e53d194d0 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16d3fff640 1 -- 172.21.15.78:0/882531640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e40a7170 con 0x561e53d194d0 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 -- 172.21.15.78:0/882531640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 msgr2=0x561e540e8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e8ec0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f16e40dc580 tx=0x7f16e40a3690 comp rx=0 tx=0).stop 2024-04-25T14:07:22.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 -- 172.21.15.78:0/882531640 shutdown_connections 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d42390 0x561e540edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 --2- 172.21.15.78:0/882531640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d194d0 0x561e540e8ec0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 -- 172.21.15.78:0/882531640 >> 172.21.15.78:0/882531640 conn(0x561e53d364e0 msgr2=0x561e540e2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 -- 172.21.15.78:0/882531640 shutdown_connections 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.891+0000 7f16f738f740 1 -- 172.21.15.78:0/882531640 wait complete. 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 -- start start 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 Processor -- start 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d42390 0x561e540f3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a9700 con 0x561e53d42390 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81aab70 con 0x561e53d216b0 2024-04-25T14:07:22.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a9dd0 con 0x561e53d194d0 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16f738f740 10 client.0 _ll_register_callbacks cb 0x7f16d800b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d42390 0x561e540f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d42390 0x561e540f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38690/0 (socket says 172.21.15.78:38690) 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 -- 172.21.15.78:0/2692722055 learned_addr learned my addr 172.21.15.78:0/2692722055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee19b640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ed99a640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 -- 172.21.15.78:0/2692722055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 msgr2=0x561e540f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 -- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 msgr2=0x561e540f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ee99c640 1 -- 172.21.15.78:0/2692722055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e53d1a0f0 con 0x561e53d42390 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.892+0000 7f16ed99a640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16ee99c640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d42390 0x561e540f3520 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f16e0035000 tx=0x7f16e00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59be4326cd5dc0ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:22.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16d1ffb640 1 -- 172.21.15.78:0/2692722055 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0058060 con 0x561e53d42390 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16d1ffb640 1 -- 172.21.15.78:0/2692722055 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f16e00045c0 con 0x561e53d42390 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e40a6a20 con 0x561e53d42390 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16d1ffb640 1 -- 172.21.15.78:0/2692722055 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f16e00047c0 con 0x561e53d42390 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 10 client.17790 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561e540094b0 con 0x561e53d42390 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16d1ffb640 1 -- 172.21.15.78:0/2692722055 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e40a6a20 con 0x561e53d42390 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16d1ffb640 1 -- 172.21.15.78:0/2692722055 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f16e00306e0 con 0x561e53d42390 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 10 client.17790 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 20 client.17790 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53cd63a0 0x561e541658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:22.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 4 client.17790 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e54050800 con 0x561e53cd63a0 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.893+0000 7f16ee19b640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53cd63a0 0x561e541658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16d1ffb640 10 client.17790 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16ee19b640 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53cd63a0 0x561e541658e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2c0c1d6cf0f76e server_cookie=facc63ee963c037b in_seq=0 out_seq=0 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16d1ffb640 1 -- 172.21.15.78:0/2692722055 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561e54050800 con 0x561e53cd63a0 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16d1ffb640 10 client.17790 handle_command_reply: tid=0 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 client.17790 shutdown 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d42390 msgr2=0x561e540f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d42390 0x561e540f3520 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f16e0035000 tx=0x7f16e00095c0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 shutdown_connections 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e53d42390 0x561e540f3520 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561e53d216b0 0x561e540f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561e53cd63a0 0x561e541658e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 --2- 172.21.15.78:0/2692722055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e53d194d0 0x561e540f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16d1ffb640 0 client.17790 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 >> 172.21.15.78:0/2692722055 conn(0x561e53d364e0 msgr2=0x561e540ecb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 shutdown_connections 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 1 -- 172.21.15.78:0/2692722055 wait complete. 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.894+0000 7f16f738f740 20 client.17790 trim_cache size 0 max 16384 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:07:22.902 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:07:22.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f16d40afb60 msgr2=0x7f16d40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f16d40afb60 0x7f16d40b2070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f16e0005cc0 tx=0x7f16e0005fd0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8103930 msgr2=0x7f16e819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:22.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8103930 0x7f16e819a000 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f16e400c970 tx=0x7f16e400ce30 comp rx=0 tx=0).stop 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 shutdown_connections 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16e8109870 0x7f16e81a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16e8104b70 0x7f16e819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f16d40afb60 0x7f16d40b2070 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 --2- 172.21.15.78:0/3646831412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16e8103930 0x7f16e819a000 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 >> 172.21.15.78:0/3646831412 conn(0x7f16e80feb20 msgr2=0x7f16e8105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 shutdown_connections 2024-04-25T14:07:22.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:22.895+0000 7f16ef19d640 1 -- 172.21.15.78:0/3646831412 wait complete. 2024-04-25T14:07:22.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:07:22.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:23.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2351415803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88104d40 msgr2=0x7fae88105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2351415803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88104d40 0x7fae88105150 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fae70009900 tx=0x7fae7002a410 comp rx=0 tx=0).stop 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2351415803 shutdown_connections 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2351415803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae88109a40 0x7fae8810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2351415803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88104d40 0x7fae88105150 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2351415803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae88103b00 0x7fae88104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2351415803 >> 172.21.15.78:0/2351415803 conn(0x7fae880fed10 msgr2=0x7fae881011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.014+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2351415803 shutdown_connections 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.015+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2351415803 wait complete. 2024-04-25T14:07:23.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.015+0000 7fae8d0a8640 1 Processor -- start 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.015+0000 7fae8d0a8640 1 -- start start 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.015+0000 7fae8d0a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88103b00 0x7fae8819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.015+0000 7fae8d0a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae88104d40 0x7fae8819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae8d0a8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae88109a40 0x7fae881a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae8d0a8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810f600 con 0x7fae88103b00 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae8d0a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810f480 con 0x7fae88104d40 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae8d0a8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae8810f780 con 0x7fae88109a40 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88103b00 0x7fae8819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88103b00 0x7fae8819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38712/0 (socket says 172.21.15.78:38712) 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 -- 172.21.15.78:0/2471905132 learned_addr learned my addr 172.21.15.78:0/2471905132 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae7ffff640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae88104d40 0x7fae8819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae8759e640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae88109a40 0x7fae881a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 -- 172.21.15.78:0/2471905132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae88109a40 msgr2=0x7fae881a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae88109a40 0x7fae881a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 -- 172.21.15.78:0/2471905132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae88104d40 msgr2=0x7fae8819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae88104d40 0x7fae8819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 -- 172.21.15.78:0/2471905132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae881a1cf0 con 0x7fae88103b00 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae7ffff640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae88104d40 0x7fae8819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae86d9d640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88103b00 0x7fae8819a260 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fae7800d740 tx=0x7fae7800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae84d99640 1 -- 172.21.15.78:0/2471905132 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78017070 con 0x7fae88103b00 2024-04-25T14:07:23.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881a1fe0 con 0x7fae88103b00 2024-04-25T14:07:23.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae84d99640 1 -- 172.21.15.78:0/2471905132 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7800de50 con 0x7fae88103b00 2024-04-25T14:07:23.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae84d99640 1 -- 172.21.15.78:0/2471905132 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78003c30 con 0x7fae88103b00 2024-04-25T14:07:23.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.016+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881a2440 con 0x7fae88103b00 2024-04-25T14:07:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.018+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8806a1c0 con 0x7fae88103b00 2024-04-25T14:07:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.019+0000 7fae84d99640 1 -- 172.21.15.78:0/2471905132 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fae78003dd0 con 0x7fae88103b00 2024-04-25T14:07:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.020+0000 7fae84d99640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fae740afaf0 0x7fae740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.020+0000 7fae7ffff640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fae740afaf0 0x7fae740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.020+0000 7fae84d99640 1 -- 172.21.15.78:0/2471905132 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fae780deb20 con 0x7fae88103b00 2024-04-25T14:07:23.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.020+0000 7fae7ffff640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fae740afaf0 0x7fae740b2000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fae70005f70 tx=0x7fae70031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.023+0000 7fae84d99640 1 -- 172.21.15.78:0/2471905132 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae8806a1c0 con 0x7fae88103b00 2024-04-25T14:07:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.242+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae8802dc30 con 0x7fae88103b00 2024-04-25T14:07:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.242+0000 7fae84d99640 1 -- 172.21.15.78:0/2471905132 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fae780a76f0 con 0x7fae88103b00 2024-04-25T14:07:23.242 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:07:23.242 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:07:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:07:23.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fae740afaf0 msgr2=0x7fae740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fae740afaf0 0x7fae740b2000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fae70005f70 tx=0x7fae70031400 comp rx=0 tx=0).stop 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88103b00 msgr2=0x7fae8819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88103b00 0x7fae8819a260 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fae7800d740 tx=0x7fae7800dc00 comp rx=0 tx=0).stop 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 shutdown_connections 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae88109a40 0x7fae881a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae88104d40 0x7fae8819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fae740afaf0 0x7fae740b2000 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.244+0000 7fae8d0a8640 1 --2- 172.21.15.78:0/2471905132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae88103b00 0x7fae8819a260 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.245+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 >> 172.21.15.78:0/2471905132 conn(0x7fae880fed10 msgr2=0x7fae88105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.245+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 shutdown_connections 2024-04-25T14:07:23.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.245+0000 7fae8d0a8640 1 -- 172.21.15.78:0/2471905132 wait complete. 2024-04-25T14:07:23.256 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:07:23.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 -- 172.21.15.78:0/2379609216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76381050a0 msgr2=0x7f76381054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 --2- 172.21.15.78:0/2379609216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76381050a0 0x7f76381054b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7628009900 tx=0x7f762802a410 comp rx=0 tx=0).stop 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 -- 172.21.15.78:0/2379609216 shutdown_connections 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 --2- 172.21.15.78:0/2379609216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7638102250 0x7f763810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 --2- 172.21.15.78:0/2379609216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638101750 0x7f7638101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 --2- 172.21.15.78:0/2379609216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76381050a0 0x7f76381054b0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 -- 172.21.15.78:0/2379609216 >> 172.21.15.78:0/2379609216 conn(0x7f76380fd350 msgr2=0x7f76380ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 -- 172.21.15.78:0/2379609216 shutdown_connections 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.365+0000 7f763e94d640 1 -- 172.21.15.78:0/2379609216 wait complete. 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 Processor -- start 2024-04-25T14:07:23.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 -- start start 2024-04-25T14:07:23.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638101750 0x7f763819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7638102250 0x7f763819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76381050a0 0x7f76381a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f763810f5f0 con 0x7f7638102250 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f763810f470 con 0x7f7638101750 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f763e94d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f763810f770 con 0x7f76381050a0 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f76377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7638102250 0x7f763819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f76377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7638102250 0x7f763819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38720/0 (socket says 172.21.15.78:38720) 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.366+0000 7f76377fe640 1 -- 172.21.15.78:0/857892252 learned_addr learned my addr 172.21.15.78:0/857892252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f763ceea640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76381050a0 0x7f76381a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f7637fff640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638101750 0x7f763819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76377fe640 1 -- 172.21.15.78:0/857892252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76381050a0 msgr2=0x7f76381a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76377fe640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76381050a0 0x7f76381a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76377fe640 1 -- 172.21.15.78:0/857892252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638101750 msgr2=0x7f763819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76377fe640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638101750 0x7f763819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76377fe640 1 -- 172.21.15.78:0/857892252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76381a1d90 con 0x7f7638102250 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f763ceea640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76381050a0 0x7f76381a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76377fe640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7638102250 0x7f763819a790 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f762c00d950 tx=0x7f762c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76357fa640 1 -- 172.21.15.78:0/857892252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c017070 con 0x7f7638102250 2024-04-25T14:07:23.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76357fa640 1 -- 172.21.15.78:0/857892252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f762c009c30 con 0x7f7638102250 2024-04-25T14:07:23.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76381a2080 con 0x7f7638102250 2024-04-25T14:07:23.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f76357fa640 1 -- 172.21.15.78:0/857892252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c00f620 con 0x7f7638102250 2024-04-25T14:07:23.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.367+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76381a25d0 con 0x7f7638102250 2024-04-25T14:07:23.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.369+0000 7f76357fa640 1 -- 172.21.15.78:0/857892252 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f762c010460 con 0x7f7638102250 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 Processor -- start 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 -- start start 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9380810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9381a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b1e92e4d80 con 0x55b1e8fb54d0 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1e929ea20 con 0x55b1e8fde390 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7645804740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b1e92e0d40 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38736/0 (socket says 172.21.15.78:38736) 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 -- 172.21.15.78:0/322315701 learned_addr learned my addr 172.21.15.78:0/322315701 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f76357fa640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76200afaf0 0x7f76200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f76377fe640 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9381a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f763ceea640 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9380810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f763ceea640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76200afaf0 0x7f76200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f76357fa640 1 -- 172.21.15.78:0/857892252 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f762c0de950 con 0x7f7638102250 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 -- 172.21.15.78:0/322315701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 msgr2=0x55b1e9381a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9381a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 -- 172.21.15.78:0/322315701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 msgr2=0x55b1e9380810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9380810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.370+0000 7f7637fff640 1 -- 172.21.15.78:0/322315701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e92a9c30 con 0x55b1e8fb54d0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.371+0000 7f7637fff640 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380d90 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7628000c00 tx=0x7f7628032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9d32d3967440d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.371+0000 7f763ceea640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76200afaf0 0x7f76200b2000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55b1e9381860 tx=0x7f7630009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.371+0000 7f761e7fc640 1 -- 172.21.15.78:0/322315701 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762803f070 con 0x55b1e8fb54d0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.371+0000 7f761e7fc640 1 -- 172.21.15.78:0/322315701 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7628004040 con 0x55b1e8fb54d0 2024-04-25T14:07:23.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.371+0000 7f761e7fc640 1 -- 172.21.15.78:0/322315701 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628035070 con 0x55b1e8fb54d0 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.371+0000 7f7645804740 1 -- 172.21.15.78:0/322315701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 msgr2=0x55b1e9380d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.371+0000 7f7645804740 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380d90 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7628000c00 tx=0x7f7628032b20 comp rx=0 tx=0).stop 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- 172.21.15.78:0/322315701 shutdown_connections 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9381a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380d90 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 --2- 172.21.15.78:0/322315701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9380810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- 172.21.15.78:0/322315701 >> 172.21.15.78:0/322315701 conn(0x55b1e8fd24e0 msgr2=0x55b1e937c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- 172.21.15.78:0/322315701 shutdown_connections 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- 172.21.15.78:0/322315701 wait complete. 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- start start 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 Processor -- start 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fb54d0 0x55b1e938a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fbd6b0 0x55b1e938aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9391a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b1e93824d0 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b1e926bdc0 con 0x55b1e8fde390 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.372+0000 7f7645804740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b1e9382650 con 0x55b1e8fb54d0 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f7645804740 10 client.0 _ll_register_callbacks cb 0x55b1e9287600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fbd6b0 0x55b1e938aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fbd6b0 0x55b1e938aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38742/0 (socket says 172.21.15.78:38742) 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 -- 172.21.15.78:0/1173749655 learned_addr learned my addr 172.21.15.78:0/1173749655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f763ceea640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9391a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f7637fff640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fb54d0 0x55b1e938a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 -- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fb54d0 msgr2=0x55b1e938a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fb54d0 0x55b1e938a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 -- 172.21.15.78:0/1173749655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 msgr2=0x55b1e9391a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9391a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 -- 172.21.15.78:0/1173749655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e8ff63f0 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f763ceea640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9391a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76377fe640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fbd6b0 0x55b1e938aad0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f762c000c00 tx=0x7f762c020e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc6eb59b1f235e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76177fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76177fe640 1 -- 172.21.15.78:0/1173749655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c020070 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76177fe640 1 -- 172.21.15.78:0/1173749655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f762c0a3410 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7628004040 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f76177fe640 1 -- 172.21.15.78:0/1173749655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f762c0a6930 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f7645804740 10 client.17808 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.373+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b1e90c07d0 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f76177fe640 1 -- 172.21.15.78:0/1173749655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7628004040 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f76177fe640 1 -- 172.21.15.78:0/1173749655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f762c010750 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7645804740 10 client.17808 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7645804740 10 client.17808 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7645804740 20 client.17808 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7645804740 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f9cf60 0x55b1e9398340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7645804740 4 client.17808 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1e92f2990 con 0x55b1e8f9cf60 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7637fff640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f9cf60 0x55b1e9398340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f76177fe640 10 client.17808 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.374+0000 7f7637fff640 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f9cf60 0x55b1e9398340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afe9df4886df6617 server_cookie=11d5acf57f45f369 in_seq=0 out_seq=0 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.375+0000 7f76177fe640 1 -- 172.21.15.78:0/1173749655 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1e92f2990 con 0x55b1e8f9cf60 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.375+0000 7f76177fe640 10 client.17808 handle_command_reply: tid=0 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.375+0000 7f7645804740 1 client.17808 shutdown 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.375+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fbd6b0 msgr2=0x55b1e938aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.375+0000 7f7645804740 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fbd6b0 0x55b1e938aad0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f762c000c00 tx=0x7f762c020e80 comp rx=0 tx=0).stop 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.375+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 shutdown_connections 2024-04-25T14:07:23.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f9cf60 0x55b1e9398340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fde390 0x55b1e9391a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fbd6b0 0x55b1e938aad0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f76177fe640 0 client.17808 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 1 --2- 172.21.15.78:0/1173749655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fb54d0 0x55b1e938a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 >> 172.21.15.78:0/1173749655 conn(0x55b1e8fd24e0 msgr2=0x55b1e937ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 shutdown_connections 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 1 -- 172.21.15.78:0/1173749655 wait complete. 2024-04-25T14:07:23.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.376+0000 7f7645804740 20 client.17808 trim_cache size 0 max 16384 2024-04-25T14:07:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 Processor -- start 2024-04-25T14:07:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 -- start start 2024-04-25T14:07:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9380cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fde390 0x55b1e9385310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76381adb40 con 0x55b1e8fb54d0 2024-04-25T14:07:23.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76381ab410 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7645804740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76381a9260 con 0x55b1e8fde390 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7637fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7637fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38758/0 (socket says 172.21.15.78:38758) 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f7637fff640 1 -- 172.21.15.78:0/949480250 learned_addr learned my addr 172.21.15.78:0/949480250 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f76377fe640 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9380cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.389+0000 7f763ceea640 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fde390 0x55b1e9385310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f7637fff640 1 -- 172.21.15.78:0/949480250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fde390 msgr2=0x55b1e9385310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f7637fff640 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fde390 0x55b1e9385310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f7637fff640 1 -- 172.21.15.78:0/949480250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fbd6b0 msgr2=0x55b1e9380cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f7637fff640 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9380cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f7637fff640 1 -- 172.21.15.78:0/949480250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e93960b0 con 0x55b1e8fb54d0 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f7637fff640 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380790 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f7628005f70 tx=0x7f7628031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72d2d39d1372df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f761dffb640 1 -- 172.21.15.78:0/949480250 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628054070 con 0x55b1e8fb54d0 2024-04-25T14:07:23.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f761dffb640 1 -- 172.21.15.78:0/949480250 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7628004040 con 0x55b1e8fb54d0 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.390+0000 7f761dffb640 1 -- 172.21.15.78:0/949480250 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762803f030 con 0x55b1e8fb54d0 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- 172.21.15.78:0/949480250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 msgr2=0x55b1e9380790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380790 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f7628005f70 tx=0x7f7628031400 comp rx=0 tx=0).stop 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- 172.21.15.78:0/949480250 shutdown_connections 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fde390 0x55b1e9385310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9380cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 --2- 172.21.15.78:0/949480250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fb54d0 0x55b1e9380790 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- 172.21.15.78:0/949480250 >> 172.21.15.78:0/949480250 conn(0x55b1e8fd24e0 msgr2=0x55b1e9388cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- 172.21.15.78:0/949480250 shutdown_connections 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- 172.21.15.78:0/949480250 wait complete. 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- start start 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 Processor -- start 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fb54d0 0x55b1e9390350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9390890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fde390 0x55b1e93fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76381a98d0 con 0x55b1e8fde390 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76381aad40 con 0x55b1e8fb54d0 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76381a9fa0 con 0x55b1e8fbd6b0 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.391+0000 7f7645804740 10 client.0 _ll_register_callbacks cb 0x7f762c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fde390 0x55b1e93fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fde390 0x55b1e93fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38774/0 (socket says 172.21.15.78:38774) 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 -- 172.21.15.78:0/1566076631 learned_addr learned my addr 172.21.15.78:0/1566076631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f7637fff640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fb54d0 0x55b1e9390350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f76377fe640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9390890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 -- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 msgr2=0x55b1e9390890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9390890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 -- 172.21.15.78:0/1566076631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fb54d0 msgr2=0x55b1e9390350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fb54d0 0x55b1e9390350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 -- 172.21.15.78:0/1566076631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e93fedf0 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f7637fff640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fb54d0 0x55b1e9390350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f763ceea640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fde390 0x55b1e93fe800 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7630010e30 tx=0x7f7630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ab3252c03dbc44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f76177fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f76177fe640 1 -- 172.21.15.78:0/1566076631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630018070 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f76177fe640 1 -- 172.21.15.78:0/1566076631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7630004b20 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7628004040 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f76177fe640 1 -- 172.21.15.78:0/1566076631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7630004d20 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f7645804740 10 client.17814 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.392+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b1e94000e0 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f76177fe640 1 -- 172.21.15.78:0/1566076631 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7628004040 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f76177fe640 1 -- 172.21.15.78:0/1566076631 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7630005330 con 0x55b1e8fde390 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7645804740 10 client.17814 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:23.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7645804740 10 client.17814 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7645804740 20 client.17814 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7645804740 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f723a0 0x55b1e9402b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7645804740 4 client.17814 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1e92ec800 con 0x55b1e8f723a0 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7637fff640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f723a0 0x55b1e9402b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f76177fe640 10 client.17814 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.393+0000 7f7637fff640 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f723a0 0x55b1e9402b30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da515578ba431068 server_cookie=dfa8bdca7fe8f1f1 in_seq=0 out_seq=0 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f76177fe640 1 -- 172.21.15.78:0/1566076631 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1e92ec800 con 0x55b1e8f723a0 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f76177fe640 10 client.17814 handle_command_reply: tid=0 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 client.17814 shutdown 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fde390 msgr2=0x55b1e93fe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fde390 0x55b1e93fe800 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7630010e30 tx=0x7f7630004a40 comp rx=0 tx=0).stop 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 shutdown_connections 2024-04-25T14:07:23.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x55b1e8f723a0 0x55b1e9402b30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1e8fde390 0x55b1e93fe800 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b1e8fbd6b0 0x55b1e9390890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f76177fe640 0 client.17814 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 --2- 172.21.15.78:0/1566076631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b1e8fb54d0 0x55b1e9390350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 >> 172.21.15.78:0/1566076631 conn(0x55b1e8fd24e0 msgr2=0x55b1e9380790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 shutdown_connections 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 1 -- 172.21.15.78:0/1566076631 wait complete. 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.394+0000 7f7645804740 20 client.17814 trim_cache size 0 max 16384 2024-04-25T14:07:23.400 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:07:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76200afaf0 msgr2=0x7f76200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76200afaf0 0x7f76200b2000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55b1e9381860 tx=0x7f7630009500 comp rx=0 tx=0).stop 2024-04-25T14:07:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7638102250 msgr2=0x7f763819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7638102250 0x7f763819a790 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f762c00d950 tx=0x7f762c00de10 comp rx=0 tx=0).stop 2024-04-25T14:07:23.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 shutdown_connections 2024-04-25T14:07:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76381050a0 0x7f76381a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f76200afaf0 0x7f76200b2000 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7638102250 0x7f763819a790 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 --2- 172.21.15.78:0/857892252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7638101750 0x7f763819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 >> 172.21.15.78:0/857892252 conn(0x7f76380fd350 msgr2=0x7f7638105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 shutdown_connections 2024-04-25T14:07:23.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.395+0000 7f763e94d640 1 -- 172.21.15.78:0/857892252 wait complete. 2024-04-25T14:07:23.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 -- 172.21.15.78:0/3936698438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 msgr2=0x7ff87c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 --2- 172.21.15.78:0/3936698438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c1036e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff870009900 tx=0x7ff87002a410 comp rx=0 tx=0).stop 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 -- 172.21.15.78:0/3936698438 shutdown_connections 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 --2- 172.21.15.78:0/3936698438 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87c108920 0x7ff87c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 --2- 172.21.15.78:0/3936698438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff87c103c20 0x7ff87c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 --2- 172.21.15.78:0/3936698438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c1036e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 -- 172.21.15.78:0/3936698438 >> 172.21.15.78:0/3936698438 conn(0x7ff87c0fdbd0 msgr2=0x7ff87c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 -- 172.21.15.78:0/3936698438 shutdown_connections 2024-04-25T14:07:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.516+0000 7ff882bba640 1 -- 172.21.15.78:0/3936698438 wait complete. 2024-04-25T14:07:23.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 Processor -- start 2024-04-25T14:07:23.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 -- start start 2024-04-25T14:07:23.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87c103c20 0x7ff87c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff87c108920 0x7ff87c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c10e4c0 con 0x7ff87c1029e0 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c10e340 con 0x7ff87c103c20 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff882bba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c10e640 con 0x7ff87c108920 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38794/0 (socket says 172.21.15.78:38794) 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 -- 172.21.15.78:0/1888326202 learned_addr learned my addr 172.21.15.78:0/1888326202 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 -- 172.21.15.78:0/1888326202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff87c108920 msgr2=0x7ff87c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff87c108920 0x7ff87c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 -- 172.21.15.78:0/1888326202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87c103c20 msgr2=0x7ff87c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87c103c20 0x7ff87c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 -- 172.21.15.78:0/1888326202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c1a20c0 con 0x7ff87c1029e0 2024-04-25T14:07:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.517+0000 7ff880956640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c19a580 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff870032fd0 tx=0x7ff8700042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.518+0000 7ff879ffb640 1 -- 172.21.15.78:0/1888326202 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87003f070 con 0x7ff87c1029e0 2024-04-25T14:07:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.518+0000 7ff879ffb640 1 -- 172.21.15.78:0/1888326202 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8700044d0 con 0x7ff87c1029e0 2024-04-25T14:07:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.518+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c1a2350 con 0x7ff87c1029e0 2024-04-25T14:07:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.518+0000 7ff879ffb640 1 -- 172.21.15.78:0/1888326202 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870035070 con 0x7ff87c1029e0 2024-04-25T14:07:23.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.518+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c1a2890 con 0x7ff87c1029e0 2024-04-25T14:07:23.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.520+0000 7ff889a60740 1 Processor -- start 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.520+0000 7ff879ffb640 1 -- 172.21.15.78:0/1888326202 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff87003b5f0 con 0x7ff87c1029e0 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.520+0000 7ff889a60740 1 -- start start 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff889a60740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1e8390 0x56454e58a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff889a60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1bf4d0 0x56454e58ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff889a60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1c76b0 0x56454e58ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff889a60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56454e4eed80 con 0x56454e1bf4d0 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff889a60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56454e4a8a20 con 0x56454e1c76b0 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff889a60740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56454e4ead40 con 0x56454e1e8390 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1bf4d0 0x56454e58ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1bf4d0 0x56454e58ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38804/0 (socket says 172.21.15.78:38804) 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 -- 172.21.15.78:0/1157594233 learned_addr learned my addr 172.21.15.78:0/1157594233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff881157640 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1e8390 0x56454e58a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff879ffb640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff86c0afb70 0x7ff86c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff87bfff640 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1c76b0 0x56454e58ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 -- 172.21.15.78:0/1157594233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1e8390 msgr2=0x56454e58a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1e8390 0x56454e58a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 -- 172.21.15.78:0/1157594233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1c76b0 msgr2=0x56454e58ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1c76b0 0x56454e58ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 -- 172.21.15.78:0/1157594233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454e4b3c30 con 0x56454e1bf4d0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff881157640 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1e8390 0x56454e58a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff881157640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff86c0afb70 0x7ff86c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff879ffb640 1 -- 172.21.15.78:0/1888326202 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff8700ff280 con 0x7ff87c1029e0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.521+0000 7ff880956640 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1bf4d0 0x56454e58ad90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ebc50 tx=0x7ff8700c33e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423ce4544e7246d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff866ffd640 1 -- 172.21.15.78:0/1157594233 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870048070 con 0x56454e1bf4d0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff866ffd640 1 -- 172.21.15.78:0/1157594233 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8700eb8a0 con 0x56454e1bf4d0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff866ffd640 1 -- 172.21.15.78:0/1157594233 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8700c7040 con 0x56454e1bf4d0 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff889a60740 1 -- 172.21.15.78:0/1157594233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1bf4d0 msgr2=0x56454e58ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff889a60740 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1bf4d0 0x56454e58ad90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ebc50 tx=0x7ff8700c33e0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff881157640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff86c0afb70 0x7ff86c0b2080 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x56454e58b860 tx=0x7ff87400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff889a60740 1 -- 172.21.15.78:0/1157594233 shutdown_connections 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff889a60740 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1c76b0 0x56454e58ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff889a60740 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1bf4d0 0x56454e58ad90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff889a60740 1 --2- 172.21.15.78:0/1157594233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1e8390 0x56454e58a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.522+0000 7ff889a60740 1 -- 172.21.15.78:0/1157594233 >> 172.21.15.78:0/1157594233 conn(0x56454e1dc4e0 msgr2=0x56454e586020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 -- 172.21.15.78:0/1157594233 shutdown_connections 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 -- 172.21.15.78:0/1157594233 wait complete. 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 -- start start 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 Processor -- start 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1bf4d0 0x56454e594250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1c76b0 0x56454e594790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1e8390 0x56454e59b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56454e58c2f0 con 0x56454e1c76b0 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56454e475dc0 con 0x56454e1e8390 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56454e58c470 con 0x56454e1bf4d0 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff889a60740 10 client.0 _ll_register_callbacks cb 0x56454e491600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff87bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1c76b0 0x56454e594790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.523+0000 7ff87bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1c76b0 0x56454e594790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38808/0 (socket says 172.21.15.78:38808) 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff87bfff640 1 -- 172.21.15.78:0/1806085595 learned_addr learned my addr 172.21.15.78:0/1806085595 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff881157640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1e8390 0x56454e59b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff880956640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1bf4d0 0x56454e594250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff87bfff640 1 -- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1bf4d0 msgr2=0x56454e594250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff87bfff640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1bf4d0 0x56454e594250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff87bfff640 1 -- 172.21.15.78:0/1806085595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1e8390 msgr2=0x56454e59b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff87bfff640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1e8390 0x56454e59b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff87bfff640 1 -- 172.21.15.78:0/1806085595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454e2003f0 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff880956640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1bf4d0 0x56454e594250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff87bfff640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1c76b0 0x56454e594790 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff86800d440 tx=0x7ff86800d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94dc76cfd8bcb7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff85bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff85bfff640 1 -- 172.21.15.78:0/1806085595 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86800f8f0 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff85bfff640 1 -- 172.21.15.78:0/1806085595 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff86800fa90 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56454e4fc990 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff85bfff640 1 -- 172.21.15.78:0/1806085595 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff86800fc90 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff889a60740 10 client.17826 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56454e2ca7d0 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.524+0000 7ff85bfff640 1 -- 172.21.15.78:0/1806085595 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56454e4fc990 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff85bfff640 1 -- 172.21.15.78:0/1806085595 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff868017420 con 0x56454e1c76b0 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff889a60740 10 client.17826 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff889a60740 10 client.17826 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:07:23.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff889a60740 20 client.17826 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff889a60740 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e1a6f60 0x56454e5a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff889a60740 4 client.17826 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454e4f6800 con 0x56454e1a6f60 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.525+0000 7ff880956640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e1a6f60 0x56454e5a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff85bfff640 10 client.17826 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff880956640 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e1a6f60 0x56454e5a1f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa8e0c9f9d895965 server_cookie=52f4e95dce37eadd in_seq=0 out_seq=0 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff85bfff640 1 -- 172.21.15.78:0/1806085595 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56454e4f6800 con 0x56454e1a6f60 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff85bfff640 10 client.17826 handle_command_reply: tid=0 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 client.17826 shutdown 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1c76b0 msgr2=0x56454e594790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1c76b0 0x56454e594790 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff86800d440 tx=0x7ff86800d900 comp rx=0 tx=0).stop 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 shutdown_connections 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1e8390 0x56454e59b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1c76b0 0x56454e594790 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e1a6f60 0x56454e5a1f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 --2- 172.21.15.78:0/1806085595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1bf4d0 0x56454e594250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff85bfff640 0 client.17826 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:23.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.526+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 >> 172.21.15.78:0/1806085595 conn(0x56454e1dc4e0 msgr2=0x56454e588f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.527+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 shutdown_connections 2024-04-25T14:07:23.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.527+0000 7ff889a60740 1 -- 172.21.15.78:0/1806085595 wait complete. 2024-04-25T14:07:23.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.527+0000 7ff889a60740 20 client.17826 trim_cache size 0 max 16384 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 Processor -- start 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 -- start start 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e5a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e5a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1adc30 con 0x56454e1e8390 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1ab500 con 0x56454e1bf4d0 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff889a60740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1a9350 con 0x56454e1c76b0 2024-04-25T14:07:23.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff881157640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e5a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.540+0000 7ff881157640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e5a6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38814/0 (socket says 172.21.15.78:38814) 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff881157640 1 -- 172.21.15.78:0/2504348135 learned_addr learned my addr 172.21.15.78:0/2504348135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff87bfff640 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff880956640 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e5a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff881157640 1 -- 172.21.15.78:0/2504348135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 msgr2=0x56454e59b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff881157640 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff881157640 1 -- 172.21.15.78:0/2504348135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 msgr2=0x56454e5a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff881157640 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e5a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff881157640 1 -- 172.21.15.78:0/2504348135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454e58fb60 con 0x56454e1e8390 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff87bfff640 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff881157640 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e5a6bc0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff87400f5b0 tx=0x7ff874012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ef0f3119e3cd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff8667fc640 1 -- 172.21.15.78:0/2504348135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874028070 con 0x56454e1e8390 2024-04-25T14:07:23.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff8667fc640 1 -- 172.21.15.78:0/2504348135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff874018450 con 0x56454e1e8390 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.541+0000 7ff8667fc640 1 -- 172.21.15.78:0/2504348135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874023680 con 0x56454e1e8390 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 -- 172.21.15.78:0/2504348135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 msgr2=0x56454e5a6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e5a6bc0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff87400f5b0 tx=0x7ff874012000 comp rx=0 tx=0).stop 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 -- 172.21.15.78:0/2504348135 shutdown_connections 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e5a6bc0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 --2- 172.21.15.78:0/2504348135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e5a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 -- 172.21.15.78:0/2504348135 >> 172.21.15.78:0/2504348135 conn(0x56454e1dc4e0 msgr2=0x56454e592870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 -- 172.21.15.78:0/2504348135 shutdown_connections 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 -- 172.21.15.78:0/2504348135 wait complete. 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 -- start start 2024-04-25T14:07:23.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.542+0000 7ff889a60740 1 Processor -- start 2024-04-25T14:07:23.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff889a60740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e59a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff889a60740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff889a60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e608370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff889a60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1a99c0 con 0x56454e1e8390 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff889a60740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1aae30 con 0x56454e1bf4d0 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff889a60740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1aa090 con 0x56454e1c76b0 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff889a60740 10 client.0 _ll_register_callbacks cb 0x7ff868005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e608370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e608370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38824/0 (socket says 172.21.15.78:38824) 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 -- 172.21.15.78:0/2364649522 learned_addr learned my addr 172.21.15.78:0/2364649522 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff87bfff640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff880956640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e59a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 -- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 msgr2=0x56454e59add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 -- 172.21.15.78:0/2364649522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 msgr2=0x56454e59a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e59a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 -- 172.21.15.78:0/2364649522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454e59bb50 con 0x56454e1e8390 2024-04-25T14:07:23.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff87bfff640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff880956640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e59a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff881157640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e608370 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff874018ae0 tx=0x7ff874018830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54f3b47143fd267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.543+0000 7ff85bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff85bfff640 1 -- 172.21.15.78:0/2364649522 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874028040 con 0x56454e1e8390 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff85bfff640 1 -- 172.21.15.78:0/2364649522 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff87401a9c0 con 0x56454e1e8390 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff874018450 con 0x56454e1e8390 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff85bfff640 1 -- 172.21.15.78:0/2364649522 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff87401abc0 con 0x56454e1e8390 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 10 client.17832 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56454e609950 con 0x56454e1e8390 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff85bfff640 1 -- 172.21.15.78:0/2364649522 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff874018450 con 0x56454e1e8390 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff85bfff640 1 -- 172.21.15.78:0/2364649522 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff87402b070 con 0x56454e1e8390 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 10 client.17832 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:23.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 10 client.17832 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 20 client.17832 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e17c3a0 0x56454e60c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 4 client.17832 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.544+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56454e471610 con 0x56454e17c3a0 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.545+0000 7ff880956640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e17c3a0 0x56454e60c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.545+0000 7ff85bfff640 10 client.17832 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.545+0000 7ff880956640 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e17c3a0 0x56454e60c260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa887d2fa3863a58 server_cookie=eadf3e0e214cdd1a in_seq=0 out_seq=0 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.545+0000 7ff85bfff640 1 -- 172.21.15.78:0/2364649522 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56454e471610 con 0x56454e17c3a0 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.545+0000 7ff85bfff640 10 client.17832 handle_command_reply: tid=0 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.545+0000 7ff889a60740 1 client.17832 shutdown 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 msgr2=0x56454e608370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e608370 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff874018ae0 tx=0x7ff874018830 comp rx=0 tx=0).stop 2024-04-25T14:07:23.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 shutdown_connections 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454e1e8390 0x56454e608370 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56454e17c3a0 0x56454e60c260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56454e1c76b0 0x56454e59add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 --2- 172.21.15.78:0/2364649522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56454e1bf4d0 0x56454e59a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff85bfff640 0 client.17832 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 >> 172.21.15.78:0/2364649522 conn(0x56454e1dc4e0 msgr2=0x56454e588e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 shutdown_connections 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 1 -- 172.21.15.78:0/2364649522 wait complete. 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff889a60740 20 client.17832 trim_cache size 0 max 16384 2024-04-25T14:07:23.552 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:07:23.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff86c0afb70 msgr2=0x7ff86c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff86c0afb70 0x7ff86c0b2080 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x56454e58b860 tx=0x7ff87400a380 comp rx=0 tx=0).stop 2024-04-25T14:07:23.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 msgr2=0x7ff87c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c19a580 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff870032fd0 tx=0x7ff8700042e0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 shutdown_connections 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff87c108920 0x7ff87c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff87c103c20 0x7ff87c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff86c0afb70 0x7ff86c0b2080 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 --2- 172.21.15.78:0/1888326202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff87c1029e0 0x7ff87c19a580 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.546+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 >> 172.21.15.78:0/1888326202 conn(0x7ff87c0fdbd0 msgr2=0x7ff87c109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.547+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 shutdown_connections 2024-04-25T14:07:23.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.547+0000 7ff882bba640 1 -- 172.21.15.78:0/1888326202 wait complete. 2024-04-25T14:07:23.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:23 smithi078 ceph-mon[29163]: pgmap v584: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:23.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:23 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2471905132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:23.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.666+0000 7f9fc1061640 1 -- 172.21.15.78:0/1194753291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1045e0 msgr2=0x7f9fbc106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.666+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1194753291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1045e0 0x7f9fbc106a00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009900 tx=0x7f9fac02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 -- 172.21.15.78:0/1194753291 shutdown_connections 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1194753291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc1098a0 0x7f9fbc10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1194753291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fbc106f40 0x7f9fbc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1194753291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1045e0 0x7f9fbc106a00 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 -- 172.21.15.78:0/1194753291 >> 172.21.15.78:0/1194753291 conn(0x7f9fbc0fdbe0 msgr2=0x7f9fbc100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 -- 172.21.15.78:0/1194753291 shutdown_connections 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 -- 172.21.15.78:0/1194753291 wait complete. 2024-04-25T14:07:23.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 Processor -- start 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.667+0000 7f9fc1061640 1 -- start start 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fbc1045e0 0x7f9fbc103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc106f40 0x7f9fbc1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1098a0 0x7f9fbc102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc10f5d0 con 0x7f9fbc1098a0 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc10f450 con 0x7f9fbc106f40 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc10f750 con 0x7f9fbc1045e0 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1098a0 0x7f9fbc102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1098a0 0x7f9fbc102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38856/0 (socket says 172.21.15.78:38856) 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 -- 172.21.15.78:0/1838244628 learned_addr learned my addr 172.21.15.78:0/1838244628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fbc1045e0 0x7f9fbc103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc106f40 0x7f9fbc1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 -- 172.21.15.78:0/1838244628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fbc1045e0 msgr2=0x7f9fbc103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fbc1045e0 0x7f9fbc103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 -- 172.21.15.78:0/1838244628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc106f40 msgr2=0x7f9fbc1020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc106f40 0x7f9fbc1020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 -- 172.21.15.78:0/1838244628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc102cb0 con 0x7f9fbc1098a0 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fbc1045e0 0x7f9fbc103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc106f40 0x7f9fbc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fbb59e640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1098a0 0x7f9fbc102610 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400eb40 tx=0x7f9fb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fb8d99640 1 -- 172.21.15.78:0/1838244628 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4004b20 con 0x7f9fbc1098a0 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc1a4350 con 0x7f9fbc1098a0 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fb8d99640 1 -- 172.21.15.78:0/1838244628 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb4004cc0 con 0x7f9fbc1098a0 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fb8d99640 1 -- 172.21.15.78:0/1838244628 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4010720 con 0x7f9fbc1098a0 2024-04-25T14:07:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.668+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc1a4760 con 0x7f9fbc1098a0 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 Processor -- start 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fb8d99640 1 -- 172.21.15.78:0/1838244628 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9fb4005560 con 0x7f9fbc1098a0 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 -- start start 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2b5f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b27944d0 0x55e7b2b5fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b279c6b0 0x55e7b2b60a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e7b2ac3d80 con 0x55e7b27944d0 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e7b2a7da20 con 0x55e7b27bd390 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.671+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7b2abfd40 con 0x55e7b279c6b0 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b27944d0 0x55e7b2b5fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b27944d0 0x55e7b2b5fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38858/0 (socket says 172.21.15.78:38858) 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 -- 172.21.15.78:0/758705933 learned_addr learned my addr 172.21.15.78:0/758705933 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2b5f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b279c6b0 0x55e7b2b60a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb8d99640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9fa80afaf0 0x7f9fa80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 -- 172.21.15.78:0/758705933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b279c6b0 msgr2=0x55e7b2b60a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b279c6b0 0x55e7b2b60a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 -- 172.21.15.78:0/758705933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 msgr2=0x55e7b2b5f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2b5f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 -- 172.21.15.78:0/758705933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b2a88c30 con 0x55e7b27944d0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b279c6b0 0x55e7b2b60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbb59e640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9fa80afaf0 0x7f9fa80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb8d99640 1 -- 172.21.15.78:0/1838244628 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9fb40df510 con 0x7f9fbc1098a0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b27944d0 0x55e7b2b5fd90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0096c0 tx=0x7f9fac032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37c807ef7c1dd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb17fa640 1 -- 172.21.15.78:0/758705933 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac03f070 con 0x55e7b27944d0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb17fa640 1 -- 172.21.15.78:0/758705933 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fac0317a0 con 0x55e7b27944d0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fbb59e640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9fa80afaf0 0x7f9fa80b2000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40a3a60 tx=0x7f9fb4076530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.672+0000 7f9fb17fa640 1 -- 172.21.15.78:0/758705933 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac035070 con 0x55e7b27944d0 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 -- 172.21.15.78:0/758705933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b27944d0 msgr2=0x55e7b2b5fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b27944d0 0x55e7b2b5fd90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0096c0 tx=0x7f9fac032de0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 -- 172.21.15.78:0/758705933 shutdown_connections 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b279c6b0 0x55e7b2b60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b27944d0 0x55e7b2b5fd90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/758705933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2b5f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 -- 172.21.15.78:0/758705933 >> 172.21.15.78:0/758705933 conn(0x55e7b27b14e0 msgr2=0x55e7b2b5b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 -- 172.21.15.78:0/758705933 shutdown_connections 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.673+0000 7f9fc7fee740 1 -- 172.21.15.78:0/758705933 wait complete. 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 -- start start 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 Processor -- start 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b69520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b69a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e7b2b614d0 con 0x55e7b279c6b0 2024-04-25T14:07:23.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e7b2a4adc0 con 0x55e7b27944d0 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7b2b61650 con 0x55e7b27bd390 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fc7fee740 10 client.0 _ll_register_callbacks cb 0x55e7b2a66600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b69a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b69a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38862/0 (socket says 172.21.15.78:38862) 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 -- 172.21.15.78:0/3530655856 learned_addr learned my addr 172.21.15.78:0/3530655856 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b69520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 -- 172.21.15.78:0/3530655856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 msgr2=0x55e7b2b709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 -- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 msgr2=0x55e7b2b69520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b69520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fbad9d640 1 -- 172.21.15.78:0/3530655856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b27d53f0 con 0x55e7b279c6b0 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.674+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b69a60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9fac02a930 tx=0x7f9fac004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319fa7498ac1adf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fa27fc640 1 -- 172.21.15.78:0/3530655856 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac04b060 con 0x55e7b279c6b0 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fa27fc640 1 -- 172.21.15.78:0/3530655856 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9fac031d20 con 0x55e7b279c6b0 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac0317a0 con 0x55e7b279c6b0 2024-04-25T14:07:23.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fa27fc640 1 -- 172.21.15.78:0/3530655856 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9fac03f030 con 0x55e7b279c6b0 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 10 client.17844 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e7b289f7d0 con 0x55e7b279c6b0 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fa27fc640 1 -- 172.21.15.78:0/3530655856 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fac0317a0 con 0x55e7b279c6b0 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fa27fc640 1 -- 172.21.15.78:0/3530655856 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9fac0035f0 con 0x55e7b279c6b0 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 10 client.17844 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 10 client.17844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 20 client.17844 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:23.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b277bf60 0x55e7b2b76e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 4 client.17844 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.675+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7b2ad1990 con 0x55e7b277bf60 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.676+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b277bf60 0x55e7b2b76e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.676+0000 7f9fa27fc640 10 client.17844 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.676+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b277bf60 0x55e7b2b76e10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3e526904304d6b server_cookie=8bb1ae17cd74b420 in_seq=0 out_seq=0 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fa27fc640 1 -- 172.21.15.78:0/3530655856 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7b2ad1990 con 0x55e7b277bf60 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fa27fc640 10 client.17844 handle_command_reply: tid=0 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 client.17844 shutdown 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 msgr2=0x55e7b2b69a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b69a60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9fac02a930 tx=0x7f9fac004970 comp rx=0 tx=0).stop 2024-04-25T14:07:23.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 shutdown_connections 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b277bf60 0x55e7b2b76e10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b69a60 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/3530655856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b69520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fa27fc640 0 client.17844 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 >> 172.21.15.78:0/3530655856 conn(0x55e7b27b14e0 msgr2=0x55e7b2b5df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 shutdown_connections 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 1 -- 172.21.15.78:0/3530655856 wait complete. 2024-04-25T14:07:23.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.677+0000 7f9fc7fee740 20 client.17844 trim_cache size 0 max 16384 2024-04-25T14:07:23.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 Processor -- start 2024-04-25T14:07:23.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 -- start start 2024-04-25T14:07:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b7c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1afc90 con 0x55e7b279c6b0 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ad560 con 0x55e7b27944d0 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ab3b0 con 0x55e7b27bd390 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38868/0 (socket says 172.21.15.78:38868) 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 -- 172.21.15.78:0/2884980186 learned_addr learned my addr 172.21.15.78:0/2884980186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b7c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 -- 172.21.15.78:0/2884980186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 msgr2=0x55e7b2b7c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b7c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 -- 172.21.15.78:0/2884980186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 msgr2=0x55e7b2b683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.691+0000 7f9fbad9d640 1 -- 172.21.15.78:0/2884980186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b2b617a0 con 0x55e7b279c6b0 2024-04-25T14:07:23.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.692+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b708e0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9fac003a10 tx=0x7f9fac032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273e7f285322e51d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.692+0000 7f9fb0ff9640 1 -- 172.21.15.78:0/2884980186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac035040 con 0x55e7b279c6b0 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.692+0000 7f9fb0ff9640 1 -- 172.21.15.78:0/2884980186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fac0037f0 con 0x55e7b279c6b0 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.692+0000 7f9fb0ff9640 1 -- 172.21.15.78:0/2884980186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac003f70 con 0x55e7b279c6b0 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- 172.21.15.78:0/2884980186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 msgr2=0x55e7b2b708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b708e0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9fac003a10 tx=0x7f9fac032de0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- 172.21.15.78:0/2884980186 shutdown_connections 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27bd390 0x55e7b2b7c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b708e0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/2884980186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27944d0 0x55e7b2b683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- 172.21.15.78:0/2884980186 >> 172.21.15.78:0/2884980186 conn(0x55e7b27b14e0 msgr2=0x55e7b2b67cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- 172.21.15.78:0/2884980186 shutdown_connections 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- 172.21.15.78:0/2884980186 wait complete. 2024-04-25T14:07:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- start start 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 Processor -- start 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27944d0 0x55e7b2b6ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2bdd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1aba20 con 0x55e7b279c6b0 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ace90 con 0x55e7b27bd390 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ac0f0 con 0x55e7b27944d0 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fc7fee740 10 client.0 _ll_register_callbacks cb 0x7f9fa400d980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38882/0 (socket says 172.21.15.78:38882) 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.693+0000 7f9fbad9d640 1 -- 172.21.15.78:0/4083554021 learned_addr learned my addr 172.21.15.78:0/4083554021 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2bdd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27944d0 0x55e7b2b6ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fbad9d640 1 -- 172.21.15.78:0/4083554021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27944d0 msgr2=0x55e7b2b6ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27944d0 0x55e7b2b6ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fbad9d640 1 -- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 msgr2=0x55e7b2bdd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2bdd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fbad9d640 1 -- 172.21.15.78:0/4083554021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b2b70e20 con 0x55e7b279c6b0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fbad9d640 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b704b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9fac003c00 tx=0x7f9fac004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bb46eeb5704958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fa27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fa27fc640 1 -- 172.21.15.78:0/4083554021 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac067060 con 0x55e7b279c6b0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac0037f0 con 0x55e7b279c6b0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fa27fc640 1 -- 172.21.15.78:0/4083554021 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9fac003cf0 con 0x55e7b279c6b0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fa27fc640 1 -- 172.21.15.78:0/4083554021 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9fac035030 con 0x55e7b279c6b0 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fc7fee740 10 client.17850 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:23.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e7b2b68900 con 0x55e7b279c6b0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.694+0000 7f9fa27fc640 1 -- 172.21.15.78:0/4083554021 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fac0037f0 con 0x55e7b279c6b0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fa27fc640 1 -- 172.21.15.78:0/4083554021 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9fac03b550 con 0x55e7b279c6b0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fc7fee740 10 client.17850 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fc7fee740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fc7fee740 20 client.17850 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b27513a0 0x55e7b2be14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fc7fee740 4 client.17850 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7b2acb800 con 0x55e7b27513a0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b27513a0 0x55e7b2be14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fb3fff640 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b27513a0 0x55e7b2be14b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5374fe9901d3aa server_cookie=49717020472c1b2e in_seq=0 out_seq=0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fa27fc640 10 client.17850 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fa27fc640 1 -- 172.21.15.78:0/4083554021 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7b2acb800 con 0x55e7b27513a0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fa27fc640 10 client.17850 handle_command_reply: tid=0 2024-04-25T14:07:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.695+0000 7f9fc7fee740 1 client.17850 shutdown 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 msgr2=0x55e7b2b704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b704b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9fac003c00 tx=0x7f9fac004b80 comp rx=0 tx=0).stop 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 shutdown_connections 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e7b27bd390 0x55e7b2bdd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7b279c6b0 0x55e7b2b704b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e7b27513a0 0x55e7b2be14b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 --2- 172.21.15.78:0/4083554021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7b27944d0 0x55e7b2b6ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 >> 172.21.15.78:0/4083554021 conn(0x55e7b27b14e0 msgr2=0x55e7b2b5de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fa27fc640 0 client.17850 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 shutdown_connections 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 1 -- 172.21.15.78:0/4083554021 wait complete. 2024-04-25T14:07:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc7fee740 20 client.17850 trim_cache size 0 max 16384 2024-04-25T14:07:23.703 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9fa80afaf0 msgr2=0x7f9fa80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9fa80afaf0 0x7f9fa80b2000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40a3a60 tx=0x7f9fb4076530 comp rx=0 tx=0).stop 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1098a0 msgr2=0x7f9fbc102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1098a0 0x7f9fbc102610 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400eb40 tx=0x7f9fb400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 shutdown_connections 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9fa80afaf0 0x7f9fa80b2000 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fbc1098a0 0x7f9fbc102610 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fbc106f40 0x7f9fbc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 --2- 172.21.15.78:0/1838244628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fbc1045e0 0x7f9fbc103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 >> 172.21.15.78:0/1838244628 conn(0x7f9fbc0fdbe0 msgr2=0x7f9fbc106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 shutdown_connections 2024-04-25T14:07:23.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:23.696+0000 7f9fc1061640 1 -- 172.21.15.78:0/1838244628 wait complete. 2024-04-25T14:07:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:23 smithi112 ceph-mon[32617]: pgmap v584: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:23.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:23 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2471905132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:23.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:23 smithi129 ceph-mon[32569]: pgmap v584: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:23.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:23 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2471905132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 -- 172.21.15.78:0/1315117822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 msgr2=0x7f1990104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 --2- 172.21.15.78:0/1315117822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f1990104070 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1984009900 tx=0x7f198402a410 comp rx=0 tx=0).stop 2024-04-25T14:07:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 -- 172.21.15.78:0/1315117822 shutdown_connections 2024-04-25T14:07:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 --2- 172.21.15.78:0/1315117822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1990108960 0x7f199010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 --2- 172.21.15.78:0/1315117822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f1990104070 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 --2- 172.21.15.78:0/1315117822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1990102a20 0x7f1990103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 -- 172.21.15.78:0/1315117822 >> 172.21.15.78:0/1315117822 conn(0x7f19900fdc10 msgr2=0x7f19901000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 -- 172.21.15.78:0/1315117822 shutdown_connections 2024-04-25T14:07:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.141+0000 7f1995eaf640 1 -- 172.21.15.78:0/1315117822 wait complete. 2024-04-25T14:07:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 Processor -- start 2024-04-25T14:07:24.142 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 -- start start 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1990102a20 0x7f199019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f199019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1990108960 0x7f19901a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f199010e520 con 0x7f1990103c60 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f199010e3a0 con 0x7f1990108960 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f1995eaf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f199010e6a0 con 0x7f1990102a20 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f199019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f199019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38902/0 (socket says 172.21.15.78:38902) 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 -- 172.21.15.78:0/2754492511 learned_addr learned my addr 172.21.15.78:0/2754492511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198f7fe640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1990102a20 0x7f199019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 -- 172.21.15.78:0/2754492511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1990102a20 msgr2=0x7f199019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1990102a20 0x7f199019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 -- 172.21.15.78:0/2754492511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1990108960 msgr2=0x7f19901a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1990108960 0x7f19901a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.142+0000 7f198effd640 1 -- 172.21.15.78:0/2754492511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19901a1d70 con 0x7f1990103c60 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.143+0000 7f198effd640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f199019a770 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1984006010 tx=0x7f1984004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.143+0000 7f198cff9640 1 -- 172.21.15.78:0/2754492511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198403f030 con 0x7f1990103c60 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.143+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19901a2000 con 0x7f1990103c60 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.143+0000 7f198cff9640 1 -- 172.21.15.78:0/2754492511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1984003cb0 con 0x7f1990103c60 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.143+0000 7f198cff9640 1 -- 172.21.15.78:0/2754492511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984035070 con 0x7f1990103c60 2024-04-25T14:07:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.143+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901a2540 con 0x7f1990103c60 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.145+0000 7f198cff9640 1 -- 172.21.15.78:0/2754492511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1984042030 con 0x7f1990103c60 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.145+0000 7f199cd63740 1 Processor -- start 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.145+0000 7f199cd63740 1 -- start start 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f199cd63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63f81b0 0x5637b6798970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f199cd63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63cf2f0 0x5637b6798ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f199cd63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b679fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f199cd63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637b66fe460 con 0x5637b63f81b0 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f199cd63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5637b66b8590 con 0x5637b63cf2f0 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f199cd63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637b66fa420 con 0x5637b63d74d0 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198cff9640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19780afae0 0x7f19780b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63f81b0 0x5637b6798970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63f81b0 0x5637b6798970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38904/0 (socket says 172.21.15.78:38904) 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 -- 172.21.15.78:0/1142532018 learned_addr learned my addr 172.21.15.78:0/1142532018 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19780afae0 0x7f19780b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198cff9640 1 -- 172.21.15.78:0/2754492511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f19840ff2b0 con 0x7f1990103c60 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198f7fe640 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63cf2f0 0x5637b6798ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198effd640 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b679fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 -- 172.21.15.78:0/1142532018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 msgr2=0x5637b679fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b679fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 -- 172.21.15.78:0/1142532018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63cf2f0 msgr2=0x5637b6798ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63cf2f0 0x5637b6798ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 -- 172.21.15.78:0/1142532018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b66c50d0 con 0x5637b63f81b0 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63f81b0 0x5637b6798970 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1988013c50 tx=0x7f1988011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311a12ce8929d0db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.146+0000 7f198ffff640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19780afae0 0x7f19780b1ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1988011c50 tx=0x7f198801a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f1975ffb640 1 -- 172.21.15.78:0/1142532018 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988020070 con 0x5637b63f81b0 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f1975ffb640 1 -- 172.21.15.78:0/1142532018 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1988009a10 con 0x5637b63f81b0 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f1975ffb640 1 -- 172.21.15.78:0/1142532018 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198800eb80 con 0x5637b63f81b0 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 -- 172.21.15.78:0/1142532018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63f81b0 msgr2=0x5637b6798970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63f81b0 0x5637b6798970 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1988013c50 tx=0x7f1988011750 comp rx=0 tx=0).stop 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 -- 172.21.15.78:0/1142532018 shutdown_connections 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b679fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63cf2f0 0x5637b6798ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 --2- 172.21.15.78:0/1142532018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63f81b0 0x5637b6798970 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 -- 172.21.15.78:0/1142532018 >> 172.21.15.78:0/1142532018 conn(0x5637b63ec300 msgr2=0x5637b6794300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 -- 172.21.15.78:0/1142532018 shutdown_connections 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.147+0000 7f199cd63740 1 -- 172.21.15.78:0/1142532018 wait complete. 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 -- start start 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 Processor -- start 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63cf2f0 0x5637b67a2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63d74d0 0x5637b67a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637b67a07e0 con 0x5637b63d74d0 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5637b6685b60 con 0x5637b63f81b0 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637b67a0960 con 0x5637b63cf2f0 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f199cd63740 10 client.0 _ll_register_callbacks cb 0x5637b66aa2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f198effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63d74d0 0x5637b67a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f198effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63d74d0 0x5637b67a2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38920/0 (socket says 172.21.15.78:38920) 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.148+0000 7f198effd640 1 -- 172.21.15.78:0/3915777907 learned_addr learned my addr 172.21.15.78:0/3915777907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198ffff640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198f7fe640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63cf2f0 0x5637b67a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198effd640 1 -- 172.21.15.78:0/3915777907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63cf2f0 msgr2=0x5637b67a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198effd640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63cf2f0 0x5637b67a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198effd640 1 -- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 msgr2=0x5637b67a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198effd640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198effd640 1 -- 172.21.15.78:0/3915777907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b6410210 con 0x5637b63d74d0 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198ffff640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f198effd640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63d74d0 0x5637b67a2c40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f19840fcda0 tx=0x7f19840c7190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1ddc71e8727aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f196effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f196effd640 1 -- 172.21.15.78:0/3915777907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984049040 con 0x5637b63d74d0 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f196effd640 1 -- 172.21.15.78:0/3915777907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f19840b00e0 con 0x5637b63d74d0 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1988009a10 con 0x5637b63d74d0 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f196effd640 1 -- 172.21.15.78:0/3915777907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f19840c7e80 con 0x5637b63d74d0 2024-04-25T14:07:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f199cd63740 10 client.17862 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5637b64dc620 con 0x5637b63d74d0 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.149+0000 7f196effd640 1 -- 172.21.15.78:0/3915777907 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1988009a10 con 0x5637b63d74d0 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f196effd640 1 -- 172.21.15.78:0/3915777907 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f198403bc20 con 0x5637b63d74d0 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f199cd63740 10 client.17862 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f199cd63740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f199cd63740 20 client.17862 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f199cd63740 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b63b6d80 0x5637b67b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f199cd63740 4 client.17862 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637b670c070 con 0x5637b63b6d80 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f198f7fe640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b63b6d80 0x5637b67b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f196effd640 10 client.17862 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.150+0000 7f198f7fe640 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b63b6d80 0x5637b67b0410 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b6a9161354e566 server_cookie=a407fa507f684237 in_seq=0 out_seq=0 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f196effd640 1 -- 172.21.15.78:0/3915777907 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5637b670c070 con 0x5637b63b6d80 2024-04-25T14:07:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f196effd640 10 client.17862 handle_command_reply: tid=0 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 client.17862 shutdown 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63d74d0 msgr2=0x5637b67a2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63d74d0 0x5637b67a2c40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f19840fcda0 tx=0x7f19840c7190 comp rx=0 tx=0).stop 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 shutdown_connections 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63d74d0 0x5637b67a2c40 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b63b6d80 0x5637b67b0410 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 --2- 172.21.15.78:0/3915777907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63cf2f0 0x5637b67a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 >> 172.21.15.78:0/3915777907 conn(0x5637b63ec300 msgr2=0x5637b6793250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f196effd640 0 client.17862 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 shutdown_connections 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.151+0000 7f199cd63740 1 -- 172.21.15.78:0/3915777907 wait complete. 2024-04-25T14:07:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.152+0000 7f199cd63740 20 client.17862 trim_cache size 0 max 16384 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 Processor -- start 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 -- start start 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19901adb50 con 0x5637b63cf2f0 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19901ab420 con 0x5637b63f81b0 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.166+0000 7f199cd63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19901a9270 con 0x5637b63d74d0 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38928/0 (socket says 172.21.15.78:38928) 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 -- 172.21.15.78:0/2649338497 learned_addr learned my addr 172.21.15.78:0/2649338497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198effd640 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198ffff640 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 -- 172.21.15.78:0/2649338497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 msgr2=0x5637b67a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 -- 172.21.15.78:0/2649338497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 msgr2=0x5637b67b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 -- 172.21.15.78:0/2649338497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b67a1780 con 0x5637b63cf2f0 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198ffff640 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f198f7fe640 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a0d00 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1990103620 tx=0x7f198000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ac759c500c2c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f19757fa640 1 -- 172.21.15.78:0/2649338497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980003c80 con 0x5637b63cf2f0 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f19757fa640 1 -- 172.21.15.78:0/2649338497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1980023020 con 0x5637b63cf2f0 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.167+0000 7f19757fa640 1 -- 172.21.15.78:0/2649338497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19800116a0 con 0x5637b63cf2f0 2024-04-25T14:07:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 -- 172.21.15.78:0/2649338497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 msgr2=0x5637b67a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a0d00 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1990103620 tx=0x7f198000b390 comp rx=0 tx=0).stop 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 -- 172.21.15.78:0/2649338497 shutdown_connections 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 --2- 172.21.15.78:0/2649338497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a0d00 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 -- 172.21.15.78:0/2649338497 >> 172.21.15.78:0/2649338497 conn(0x5637b63ec300 msgr2=0x5637b679cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 -- 172.21.15.78:0/2649338497 shutdown_connections 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 -- 172.21.15.78:0/2649338497 wait complete. 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.168+0000 7f199cd63740 1 -- start start 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 1 Processor -- start 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19901a98e0 con 0x5637b63cf2f0 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19901aad50 con 0x5637b63f81b0 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19901a9fb0 con 0x5637b63d74d0 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f199cd63740 10 client.0 _ll_register_callbacks cb 0x7f1984009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38938/0 (socket says 172.21.15.78:38938) 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 -- 172.21.15.78:0/2401272044 learned_addr learned my addr 172.21.15.78:0/2401272044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 -- 172.21.15.78:0/2401272044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 msgr2=0x5637b67a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198effd640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198ffff640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 -- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 msgr2=0x5637b67a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 -- 172.21.15.78:0/2401272044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b67a8040 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198effd640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198ffff640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f198f7fe640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a6610 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f1980025040 tx=0x7f1980003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4385340a2e2374bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.169+0000 7f196effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f196effd640 1 -- 172.21.15.78:0/2401272044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198002c020 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f196effd640 1 -- 172.21.15.78:0/2401272044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1980005450 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1980023020 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f196effd640 1 -- 172.21.15.78:0/2401272044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1980037050 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 10 client.17868 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5637b67a9270 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f196effd640 1 -- 172.21.15.78:0/2401272044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1980023020 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f196effd640 1 -- 172.21.15.78:0/2401272044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f198001f500 con 0x5637b63cf2f0 2024-04-25T14:07:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 10 client.17868 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 20 client.17868 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b638c1c0 0x5637b681ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 4 client.17868 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637b6705ee0 con 0x5637b638c1c0 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.170+0000 7f198effd640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b638c1c0 0x5637b681ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.171+0000 7f196effd640 10 client.17868 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.171+0000 7f198effd640 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b638c1c0 0x5637b681ace0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c363bc6e04441ab5 server_cookie=854cd44558d6ff38 in_seq=0 out_seq=0 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.184+0000 7f196effd640 1 -- 172.21.15.78:0/2401272044 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637b6705ee0 con 0x5637b638c1c0 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.184+0000 7f196effd640 10 client.17868 handle_command_reply: tid=0 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.184+0000 7f199cd63740 1 client.17868 shutdown 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 msgr2=0x5637b67a6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a6610 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f1980025040 tx=0x7f1980003e00 comp rx=0 tx=0).stop 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 shutdown_connections 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5637b63f81b0 0x5637b67a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5637b638c1c0 0x5637b681ace0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637b63d74d0 0x5637b67a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 --2- 172.21.15.78:0/2401272044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637b63cf2f0 0x5637b67a6610 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f196effd640 0 client.17868 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 >> 172.21.15.78:0/2401272044 conn(0x5637b63ec300 msgr2=0x5637b679cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 shutdown_connections 2024-04-25T14:07:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 1 -- 172.21.15.78:0/2401272044 wait complete. 2024-04-25T14:07:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f199cd63740 20 client.17868 trim_cache size 0 max 16384 2024-04-25T14:07:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:07:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:07:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:07:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19780afae0 msgr2=0x7f19780b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19780afae0 0x7f19780b1ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1988011c50 tx=0x7f198801a000 comp rx=0 tx=0).stop 2024-04-25T14:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 msgr2=0x7f199019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f199019a770 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1984006010 tx=0x7f1984004340 comp rx=0 tx=0).stop 2024-04-25T14:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 shutdown_connections 2024-04-25T14:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1990108960 0x7f19901a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f19780afae0 0x7f19780b1ff0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1990103c60 0x7f199019a770 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.185+0000 7f1995eaf640 1 --2- 172.21.15.78:0/2754492511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1990102a20 0x7f199019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.186+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 >> 172.21.15.78:0/2754492511 conn(0x7f19900fdc10 msgr2=0x7f1990109f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.186+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 shutdown_connections 2024-04-25T14:07:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:24.186+0000 7f1995eaf640 1 -- 172.21.15.78:0/2754492511 wait complete. 2024-04-25T14:07:24.708 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:07:24.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:24.708 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:07:24.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.820+0000 7f1ebf09f640 1 -- 172.21.15.78:0/3546845487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8103930 msgr2=0x7f1eb8103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.820+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/3546845487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8103930 0x7f1eb8103de0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009900 tx=0x7f1ea402a410 comp rx=0 tx=0).stop 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.820+0000 7f1ebf09f640 1 -- 172.21.15.78:0/3546845487 shutdown_connections 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.820+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/3546845487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb8104470 0x7f1eb810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.820+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/3546845487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8103930 0x7f1eb8103de0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.820+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/3546845487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb8102f50 0x7f1eb8103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 -- 172.21.15.78:0/3546845487 >> 172.21.15.78:0/3546845487 conn(0x7f1eb80fd350 msgr2=0x7f1eb80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 -- 172.21.15.78:0/3546845487 shutdown_connections 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 -- 172.21.15.78:0/3546845487 wait complete. 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 Processor -- start 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 -- start start 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb8102f50 0x7f1eb819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb8103930 0x7f1eb819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8104470 0x7f1eb81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb810e540 con 0x7f1eb8104470 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb810e3c0 con 0x7f1eb8102f50 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebf09f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb810e6c0 con 0x7f1eb8103930 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.821+0000 7f1ebd63c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8104470 0x7f1eb81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8104470 0x7f1eb81a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38954/0 (socket says 172.21.15.78:38954) 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 -- 172.21.15.78:0/1436615016 learned_addr learned my addr 172.21.15.78:0/1436615016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb8102f50 0x7f1eb819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb8103930 0x7f1eb819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 -- 172.21.15.78:0/1436615016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb8103930 msgr2=0x7f1eb819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb8103930 0x7f1eb819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 -- 172.21.15.78:0/1436615016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb8102f50 msgr2=0x7f1eb819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb8102f50 0x7f1eb819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 -- 172.21.15.78:0/1436615016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb81a1d00 con 0x7f1eb8104470 2024-04-25T14:07:24.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebd63c640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8104470 0x7f1eb81a1690 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000eaa0 tx=0x7f1eb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1eb5ffb640 1 -- 172.21.15.78:0/1436615016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000cdb0 con 0x7f1eb8104470 2024-04-25T14:07:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb81a1ff0 con 0x7f1eb8104470 2024-04-25T14:07:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1eb5ffb640 1 -- 172.21.15.78:0/1436615016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb0005020 con 0x7f1eb8104470 2024-04-25T14:07:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1eb5ffb640 1 -- 172.21.15.78:0/1436615016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00116d0 con 0x7f1eb8104470 2024-04-25T14:07:24.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.822+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb81a2450 con 0x7f1eb8104470 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.824+0000 7f1eb5ffb640 1 -- 172.21.15.78:0/1436615016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1eb00054a0 con 0x7f1eb8104470 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 Processor -- start 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 -- start start 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56510622a550 0x5651065f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565106232730 0x5651065fb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565106559b00 con 0x565106253410 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5651064f5920 con 0x56510622a550 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ec602f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565106555ac0 con 0x565106232730 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38968/0 (socket says 172.21.15.78:38968) 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 -- 172.21.15.78:0/724929605 learned_addr learned my addr 172.21.15.78:0/724929605 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb5ffb640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ea80afaf0 0x7f1ea80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565106232730 0x5651065fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ebd63c640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ea80afaf0 0x7f1ea80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56510622a550 0x5651065f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 -- 172.21.15.78:0/724929605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565106232730 msgr2=0x5651065fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565106232730 0x5651065fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 -- 172.21.15.78:0/724929605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56510622a550 msgr2=0x5651065f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56510622a550 0x5651065f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb5ffb640 1 -- 172.21.15.78:0/1436615016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1eb00ded30 con 0x7f1eb8104470 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1eb7fff640 1 -- 172.21.15.78:0/724929605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651063337e0 con 0x565106253410 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.825+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56510622a550 0x5651065f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.826+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f44a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4031500 tx=0x7f1ea4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4324159edfb0a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.826+0000 7f1ea2ffd640 1 -- 172.21.15.78:0/724929605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4035050 con 0x565106253410 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.826+0000 7f1ebd63c640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ea80afaf0 0x7f1ea80b2000 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00a7190 tx=0x7f1eb0074040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.826+0000 7f1ea2ffd640 1 -- 172.21.15.78:0/724929605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea4031820 con 0x565106253410 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.826+0000 7f1ea2ffd640 1 -- 172.21.15.78:0/724929605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004920 con 0x565106253410 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 -- 172.21.15.78:0/724929605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 msgr2=0x5651065f44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f44a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4031500 tx=0x7f1ea4005a20 comp rx=0 tx=0).stop 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 -- 172.21.15.78:0/724929605 shutdown_connections 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565106232730 0x5651065fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56510622a550 0x5651065f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 --2- 172.21.15.78:0/724929605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f44a0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 -- 172.21.15.78:0/724929605 >> 172.21.15.78:0/724929605 conn(0x565106247560 msgr2=0x5651065efe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 -- 172.21.15.78:0/724929605 shutdown_connections 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 -- 172.21.15.78:0/724929605 wait complete. 2024-04-25T14:07:24.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 -- start start 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.827+0000 7f1ec602f740 1 Processor -- start 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106605700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651065fc310 con 0x565106253410 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5651064f35d0 con 0x565106232730 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5651065fc490 con 0x56510622a550 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 10 client.0 _ll_register_callbacks cb 0x5651064e1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106605700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106605700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38974/0 (socket says 172.21.15.78:38974) 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 -- 172.21.15.78:0/474832767 learned_addr learned my addr 172.21.15.78:0/474832767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 -- 172.21.15.78:0/474832767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 msgr2=0x5651065fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065fe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 -- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 msgr2=0x5651065fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 -- 172.21.15.78:0/474832767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565106393f70 con 0x565106253410 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106605700 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4031440 tx=0x7f1ea40319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5483cba7df5133da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1e97fff640 1 -- 172.21.15.78:0/474832767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4035030 con 0x565106253410 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1e97fff640 1 -- 172.21.15.78:0/474832767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1ea4004250 con 0x565106253410 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.828+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea4031820 con 0x565106253410 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1e97fff640 1 -- 172.21.15.78:0/474832767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1ea4004450 con 0x565106253410 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 10 client.17880 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56510622b170 con 0x565106253410 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1e97fff640 1 -- 172.21.15.78:0/474832767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea4031820 con 0x565106253410 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1e97fff640 1 -- 172.21.15.78:0/474832767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1ea40038c0 con 0x565106253410 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 10 client.17880 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 20 client.17880 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651062121f0 0x56510660b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 4 client.17880 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565106567710 con 0x5651062121f0 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.829+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651062121f0 0x56510660b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.830+0000 7f1e97fff640 10 client.17880 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.830+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651062121f0 0x56510660b7c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a26915faba6658 server_cookie=754337de4b06c258 in_seq=0 out_seq=0 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.830+0000 7f1e97fff640 1 -- 172.21.15.78:0/474832767 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565106567710 con 0x5651062121f0 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.830+0000 7f1e97fff640 10 client.17880 handle_command_reply: tid=0 2024-04-25T14:07:24.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 client.17880 shutdown 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 msgr2=0x565106605700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106605700 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4031440 tx=0x7f1ea40319d0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 shutdown_connections 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106605700 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651062121f0 0x56510660b7c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 --2- 172.21.15.78:0/474832767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1e97fff640 0 client.17880 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 >> 172.21.15.78:0/474832767 conn(0x565106247560 msgr2=0x5651065eeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 shutdown_connections 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 1 -- 172.21.15.78:0/474832767 wait complete. 2024-04-25T14:07:24.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.831+0000 7f1ec602f740 20 client.17880 trim_cache size 0 max 16384 2024-04-25T14:07:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 Processor -- start 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 -- start start 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81adaa0 con 0x565106253410 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81ab370 con 0x565106232730 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ec602f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81a91c0 con 0x56510622a550 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38976/0 (socket says 172.21.15.78:38976) 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 -- 172.21.15.78:0/305348169 learned_addr learned my addr 172.21.15.78:0/305348169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 -- 172.21.15.78:0/305348169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 msgr2=0x5651065f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 -- 172.21.15.78:0/305348169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 msgr2=0x5651065f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 -- 172.21.15.78:0/305348169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651065fee60 con 0x565106253410 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f3880 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4033000 tx=0x7f1ea4032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e33bf7fd6308f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ea27fc640 1 -- 172.21.15.78:0/305348169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4035070 con 0x565106253410 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ea27fc640 1 -- 172.21.15.78:0/305348169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea4031820 con 0x565106253410 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.849+0000 7f1ea27fc640 1 -- 172.21.15.78:0/305348169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40046c0 con 0x565106253410 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 -- 172.21.15.78:0/305348169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 msgr2=0x5651065f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f3880 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4033000 tx=0x7f1ea4032dc0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 -- 172.21.15.78:0/305348169 shutdown_connections 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x5651065f3880 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x5651065f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 --2- 172.21.15.78:0/305348169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x5651065f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 -- 172.21.15.78:0/305348169 >> 172.21.15.78:0/305348169 conn(0x565106247560 msgr2=0x5651065f2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 -- 172.21.15.78:0/305348169 shutdown_connections 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.850+0000 7f1ec602f740 1 -- 172.21.15.78:0/305348169 wait complete. 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 -- start start 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 Processor -- start 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x565106605620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x565106605b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106672150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81a9830 con 0x565106253410 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81aaca0 con 0x565106232730 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81a9f00 con 0x56510622a550 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ec602f740 10 client.0 _ll_register_callbacks cb 0x7f1ea4005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:24.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106672150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106672150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38992/0 (socket says 172.21.15.78:38992) 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 -- 172.21.15.78:0/183129945 learned_addr learned my addr 172.21.15.78:0/183129945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x565106605620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x565106605b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 -- 172.21.15.78:0/183129945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 msgr2=0x565106605620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x565106605620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 -- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 msgr2=0x565106605b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x565106605b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.851+0000 7f1eb7fff640 1 -- 172.21.15.78:0/183129945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565106672710 con 0x565106253410 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1eb7fff640 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106672150 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4003ed0 tx=0x7f1ea4003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bc73f45fd20b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1e97fff640 1 -- 172.21.15.78:0/183129945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4035040 con 0x565106253410 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1e97fff640 1 -- 172.21.15.78:0/183129945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f1ea4004ad0 con 0x565106253410 2024-04-25T14:07:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea4031820 con 0x565106253410 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1e97fff640 1 -- 172.21.15.78:0/183129945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1ea4067050 con 0x565106253410 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 10 client.17886 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5651066739d0 con 0x565106253410 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1e97fff640 1 -- 172.21.15.78:0/183129945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea4031820 con 0x565106253410 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1e97fff640 1 -- 172.21.15.78:0/183129945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1ea4039b60 con 0x565106253410 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 10 client.17886 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 20 client.17886 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651061e7630 0x565106676240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 4 client.17886 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565106561580 con 0x5651061e7630 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.852+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651061e7630 0x565106676240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.853+0000 7f1e97fff640 10 client.17886 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.853+0000 7f1ebce3b640 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651061e7630 0x565106676240 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf1f5ae44aa62cb server_cookie=4d9e78c8079e9807 in_seq=0 out_seq=0 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1e97fff640 1 -- 172.21.15.78:0/183129945 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565106561580 con 0x5651061e7630 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1e97fff640 10 client.17886 handle_command_reply: tid=0 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 client.17886 shutdown 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 msgr2=0x565106672150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106672150 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4003ed0 tx=0x7f1ea4003fd0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 shutdown_connections 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565106253410 0x565106672150 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565106232730 0x565106605b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5651061e7630 0x565106676240 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 --2- 172.21.15.78:0/183129945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56510622a550 0x565106605620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1e97fff640 0 client.17886 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 >> 172.21.15.78:0/183129945 conn(0x565106247560 msgr2=0x5651065f3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.854+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 shutdown_connections 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ec602f740 1 -- 172.21.15.78:0/183129945 wait complete. 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ec602f740 20 client.17886 trim_cache size 0 max 16384 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "6a5c3c44-3be0-4ac9-833b-a84a74a987a6", 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:07:24.861 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:07:24.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ea80afaf0 msgr2=0x7f1ea80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ea80afaf0 0x7f1ea80b2000 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00a7190 tx=0x7f1eb0074040 comp rx=0 tx=0).stop 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8104470 msgr2=0x7f1eb81a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8104470 0x7f1eb81a1690 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000eaa0 tx=0x7f1eb000ef60 comp rx=0 tx=0).stop 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 shutdown_connections 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f1ea80afaf0 0x7f1ea80b2000 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb8104470 0x7f1eb81a1690 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb8103930 0x7f1eb819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 --2- 172.21.15.78:0/1436615016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1eb8102f50 0x7f1eb819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 >> 172.21.15.78:0/1436615016 conn(0x7f1eb80fd350 msgr2=0x7f1eb8109cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 shutdown_connections 2024-04-25T14:07:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.855+0000 7f1ebf09f640 1 -- 172.21.15.78:0/1436615016 wait complete. 2024-04-25T14:07:24.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:07:24.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:07:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 -- 172.21.15.78:0/3034285293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 msgr2=0x7f40b81037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 --2- 172.21.15.78:0/3034285293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b81037e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009900 tx=0x7f40ac02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 -- 172.21.15.78:0/3034285293 shutdown_connections 2024-04-25T14:07:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 --2- 172.21.15.78:0/3034285293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b81080d0 0x7f40b810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 --2- 172.21.15.78:0/3034285293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b81037e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 --2- 172.21.15.78:0/3034285293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b8102190 0x7f40b8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 -- 172.21.15.78:0/3034285293 >> 172.21.15.78:0/3034285293 conn(0x7f40b80fd380 msgr2=0x7f40b80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 -- 172.21.15.78:0/3034285293 shutdown_connections 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.976+0000 7f40bd89f640 1 -- 172.21.15.78:0/3034285293 wait complete. 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 Processor -- start 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 -- start start 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b8102190 0x7f40b8109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b81080d0 0x7f40b81a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40b810dcb0 con 0x7f40b81033d0 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b810db30 con 0x7f40b8102190 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40bd89f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b810de30 con 0x7f40b81080d0 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39002/0 (socket says 172.21.15.78:39002) 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40b67fc640 1 -- 172.21.15.78:0/2490390718 learned_addr learned my addr 172.21.15.78:0/2490390718 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.977+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b8102190 0x7f40b8109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40b77fe640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b81080d0 0x7f40b81a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40b67fc640 1 -- 172.21.15.78:0/2490390718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b81080d0 msgr2=0x7f40b81a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40b67fc640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b81080d0 0x7f40b81a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40b67fc640 1 -- 172.21.15.78:0/2490390718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b8102190 msgr2=0x7f40b8109fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40b67fc640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b8102190 0x7f40b8109fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40b67fc640 1 -- 172.21.15.78:0/2490390718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b81a1cf0 con 0x7f40b81033d0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40b67fc640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b819f010 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006010 tx=0x7f40ac004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40bc89d640 1 -- 172.21.15.78:0/2490390718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac03f030 con 0x7f40b81033d0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40bc89d640 1 -- 172.21.15.78:0/2490390718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ac003cb0 con 0x7f40b81033d0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b81a1f80 con 0x7f40b81033d0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40bc89d640 1 -- 172.21.15.78:0/2490390718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac035070 con 0x7f40b81033d0 2024-04-25T14:07:24.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.978+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a2390 con 0x7f40b81033d0 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.980+0000 7f40bc89d640 1 -- 172.21.15.78:0/2490390718 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f40ac042030 con 0x7f40b81033d0 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 Processor -- start 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 -- start start 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f69faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66304d0 0x5575f69fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f69fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575f695fd80 con 0x5575f66304d0 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575f69198b0 con 0x5575f66386b0 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40c482a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5575f695bd40 con 0x5575f6659390 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66304d0 0x5575f69fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40bc89d640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a00afaf0 0x7f40a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66304d0 0x5575f69fafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39012/0 (socket says 172.21.15.78:39012) 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 -- 172.21.15.78:0/966444248 learned_addr learned my addr 172.21.15.78:0/966444248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b77fe640 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f69faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b77fe640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a00afaf0 0x7f40a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b67fc640 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f69fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 -- 172.21.15.78:0/966444248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 msgr2=0x5575f69faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f69faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40bc89d640 1 -- 172.21.15.78:0/2490390718 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f40ac0ff2c0 con 0x7f40b81033d0 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 -- 172.21.15.78:0/966444248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 msgr2=0x5575f69fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f69fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.981+0000 7f40b6ffd640 1 -- 172.21.15.78:0/966444248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f68eaa40 con 0x5575f66304d0 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.982+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66304d0 0x5575f69fafe0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f40a800bc70 tx=0x7f40a80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3346b84c02db0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.982+0000 7f40b77fe640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a00afaf0 0x7f40a00b2000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5575f69fbab0 tx=0x7f40b0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.982+0000 7f409dffb640 1 -- 172.21.15.78:0/966444248 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8004220 con 0x5575f66304d0 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.982+0000 7f409dffb640 1 -- 172.21.15.78:0/966444248 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a800be20 con 0x5575f66304d0 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.982+0000 7f409dffb640 1 -- 172.21.15.78:0/966444248 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8004c20 con 0x5575f66304d0 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 -- 172.21.15.78:0/966444248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66304d0 msgr2=0x5575f69fafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66304d0 0x5575f69fafe0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f40a800bc70 tx=0x7f40a80107a0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 -- 172.21.15.78:0/966444248 shutdown_connections 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f69fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66304d0 0x5575f69fafe0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 --2- 172.21.15.78:0/966444248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f69faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 -- 172.21.15.78:0/966444248 >> 172.21.15.78:0/966444248 conn(0x5575f664d4e0 msgr2=0x5575f69f6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 -- 172.21.15.78:0/966444248 shutdown_connections 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 -- 172.21.15.78:0/966444248 wait complete. 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 -- start start 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 Processor -- start 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.983+0000 7f40c482a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f66304d0 0x5575f6a048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f6a04de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f6659390 0x5575f6a0bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5575f69fc780 con 0x5575f6659390 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575f68e6d10 con 0x5575f66386b0 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5575f69fc900 con 0x5575f66304d0 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 10 client.0 _ll_register_callbacks cb 0x5575f6913210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:24.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f6659390 0x5575f6a0bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f6659390 0x5575f6a0bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39022/0 (socket says 172.21.15.78:39022) 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 -- 172.21.15.78:0/2704544642 learned_addr learned my addr 172.21.15.78:0/2704544642 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b67fc640 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f6a04de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f66304d0 0x5575f6a048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 -- 172.21.15.78:0/2704544642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f66304d0 msgr2=0x5575f6a048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f66304d0 0x5575f6a048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 -- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 msgr2=0x5575f6a04de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f6a04de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 -- 172.21.15.78:0/2704544642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f673b7d0 con 0x5575f6659390 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40b77fe640 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f6659390 0x5575f6a0bd40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e6e0 tx=0x7f40b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff69782ece958e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f4096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f4096ffd640 1 -- 172.21.15.78:0/2704544642 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0025070 con 0x5575f6659390 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f4096ffd640 1 -- 172.21.15.78:0/2704544642 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f40b001f860 con 0x5575f6659390 2024-04-25T14:07:24.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a800be20 con 0x5575f6659390 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f4096ffd640 1 -- 172.21.15.78:0/2704544642 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f40b0031050 con 0x5575f6659390 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 10 client.17898 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.984+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5575f66713f0 con 0x5575f6659390 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f4096ffd640 1 -- 172.21.15.78:0/2704544642 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a800be20 con 0x5575f6659390 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f4096ffd640 1 -- 172.21.15.78:0/2704544642 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f40b001e870 con 0x5575f6659390 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f40c482a740 10 client.17898 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f40c482a740 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f40c482a740 20 client.17898 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f40c482a740 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f6617f60 0x5575f6a12280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f40c482a740 4 client.17898 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f696d990 con 0x5575f6617f60 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.985+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f6617f60 0x5575f6a12280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f4096ffd640 10 client.17898 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f6617f60 0x5575f6a12280 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11aec3038a83cda server_cookie=9788572e920865eb in_seq=0 out_seq=0 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f4096ffd640 1 -- 172.21.15.78:0/2704544642 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575f696d990 con 0x5575f6617f60 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f4096ffd640 10 client.17898 handle_command_reply: tid=0 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 client.17898 shutdown 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f6659390 msgr2=0x5575f6a0bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f6659390 0x5575f6a0bd40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e6e0 tx=0x7f40b0004ab0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 shutdown_connections 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f6659390 0x5575f6a0bd40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66386b0 0x5575f6a04de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f6617f60 0x5575f6a12280 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.986+0000 7f40c482a740 1 --2- 172.21.15.78:0/2704544642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f66304d0 0x5575f6a048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.987+0000 7f4096ffd640 0 client.17898 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.987+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 >> 172.21.15.78:0/2704544642 conn(0x5575f664d4e0 msgr2=0x5575f69f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:24.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.987+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 shutdown_connections 2024-04-25T14:07:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.987+0000 7f40c482a740 1 -- 172.21.15.78:0/2704544642 wait complete. 2024-04-25T14:07:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:24.987+0000 7f40c482a740 20 client.17898 trim_cache size 0 max 16384 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 Processor -- start 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 -- start start 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f69f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f69f6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f6a02f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81adaf0 con 0x5575f66386b0 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81ab3c0 con 0x5575f66304d0 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.004+0000 7f40c482a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81a9210 con 0x5575f6659390 2024-04-25T14:07:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f69f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f69f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39036/0 (socket says 172.21.15.78:39036) 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 -- 172.21.15.78:0/1389107429 learned_addr learned my addr 172.21.15.78:0/1389107429 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b77fe640 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f6a02f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 -- 172.21.15.78:0/1389107429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 msgr2=0x5575f6a02f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f6a02f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f69f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 -- 172.21.15.78:0/1389107429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 msgr2=0x5575f69f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f69f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 -- 172.21.15.78:0/1389107429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f66310f0 con 0x5575f66386b0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f40b67fc640 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f69f6150 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0fcdb0 tx=0x7f40ac08e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437a873f34a1a106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f409d7fa640 1 -- 172.21.15.78:0/1389107429 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac049070 con 0x5575f66386b0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f409d7fa640 1 -- 172.21.15.78:0/1389107429 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ac0c72a0 con 0x5575f66386b0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.005+0000 7f409d7fa640 1 -- 172.21.15.78:0/1389107429 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0d05f0 con 0x5575f66386b0 2024-04-25T14:07:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 -- 172.21.15.78:0/1389107429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 msgr2=0x5575f69f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f69f6150 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0fcdb0 tx=0x7f40ac08e150 comp rx=0 tx=0).stop 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 -- 172.21.15.78:0/1389107429 shutdown_connections 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f6a02f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f69f6150 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 --2- 172.21.15.78:0/1389107429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f69f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 -- 172.21.15.78:0/1389107429 >> 172.21.15.78:0/1389107429 conn(0x5575f664d4e0 msgr2=0x5575f69ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 -- 172.21.15.78:0/1389107429 shutdown_connections 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 -- 172.21.15.78:0/1389107429 wait complete. 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 -- start start 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.006+0000 7f40c482a740 1 Processor -- start 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f6a083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f6a088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f6a785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81a9880 con 0x5575f66386b0 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81aacf0 con 0x5575f66304d0 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81a9f50 con 0x5575f6659390 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 10 client.0 _ll_register_callbacks cb 0x7f40b0010dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f6a088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f6a088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39040/0 (socket says 172.21.15.78:39040) 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 -- 172.21.15.78:0/4073789412 learned_addr learned my addr 172.21.15.78:0/4073789412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f6a083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 -- 172.21.15.78:0/4073789412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 msgr2=0x5575f6a785e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f6a785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 -- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 msgr2=0x5575f6a083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f6a083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 -- 172.21.15.78:0/4073789412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f69204b0 con 0x5575f66386b0 2024-04-25T14:07:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f6a083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40b67fc640 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f6a088f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0b0010 tx=0x7f40ac009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5336f6d92e5f4cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f4096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f4096ffd640 1 -- 172.21.15.78:0/4073789412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac049040 con 0x5575f66386b0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f4096ffd640 1 -- 172.21.15.78:0/4073789412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f40ac0c3a20 con 0x5575f66386b0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40ac0c72a0 con 0x5575f66386b0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f4096ffd640 1 -- 172.21.15.78:0/4073789412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f40ac003e60 con 0x5575f66386b0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 10 client.17904 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.007+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5575f6a79be0 con 0x5575f66386b0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f4096ffd640 1 -- 172.21.15.78:0/4073789412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ac0c72a0 con 0x5575f66386b0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f4096ffd640 1 -- 172.21.15.78:0/4073789412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f40ac0c79f0 con 0x5575f66386b0 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f40c482a740 10 client.17904 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f40c482a740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f40c482a740 20 client.17904 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f40c482a740 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f65ed3a0 0x5575f6a7c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f40c482a740 4 client.17904 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5575f6967800 con 0x5575f65ed3a0 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.008+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f65ed3a0 0x5575f6a7c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f4096ffd640 10 client.17904 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40b6ffd640 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f65ed3a0 0x5575f6a7c620 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6914ef30181395 server_cookie=946534502f08b488 in_seq=0 out_seq=0 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f4096ffd640 1 -- 172.21.15.78:0/4073789412 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f40a8028e50 con 0x5575f65ed3a0 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f4096ffd640 10 client.17904 handle_command_reply: tid=0 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 client.17904 shutdown 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 msgr2=0x5575f6a088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f6a088f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0b0010 tx=0x7f40ac009560 comp rx=0 tx=0).stop 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 shutdown_connections 2024-04-25T14:07:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5575f65ed3a0 0x5575f6a7c620 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5575f6659390 0x5575f6a785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f4096ffd640 0 client.17904 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5575f66386b0 0x5575f6a088f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 --2- 172.21.15.78:0/4073789412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575f66304d0 0x5575f6a083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 >> 172.21.15.78:0/4073789412 conn(0x5575f664d4e0 msgr2=0x5575f69f5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 shutdown_connections 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 1 -- 172.21.15.78:0/4073789412 wait complete. 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.009+0000 7f40c482a740 20 client.17904 trim_cache size 0 max 16384 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:07:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "6a5c3c44-3be0-4ac9-833b-a84a74a987a6": { 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "6a5c3c44-3be0-4ac9-833b-a84a74a987a6", 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:07:25.017 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a00afaf0 msgr2=0x7f40a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a00afaf0 0x7f40a00b2000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5575f69fbab0 tx=0x7f40b0004a40 comp rx=0 tx=0).stop 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 msgr2=0x7f40b819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b819f010 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006010 tx=0x7f40ac004340 comp rx=0 tx=0).stop 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 shutdown_connections 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40b81080d0 0x7f40b81a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f40a00afaf0 0x7f40a00b2000 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40b81033d0 0x7f40b819f010 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 --2- 172.21.15.78:0/2490390718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b8102190 0x7f40b8109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 >> 172.21.15.78:0/2490390718 conn(0x7f40b80fd380 msgr2=0x7f40b80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 shutdown_connections 2024-04-25T14:07:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:25.010+0000 7f40bd89f640 1 -- 172.21.15.78:0/2490390718 wait complete. 2024-04-25T14:07:25.021 INFO:tasks.cephfs.filesystem:scrub status for tag:6a5c3c44-3be0-4ac9-833b-a84a74a987a6 - {'path': '/', 'tag': '6a5c3c44-3be0-4ac9-833b-a84a74a987a6', 'options': 'recursive,force'} 2024-04-25T14:07:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:25 smithi078 ceph-mon[29163]: pgmap v585: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:25 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:07:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:25 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:25 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:07:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:25 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:25.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:25 smithi112 ceph-mon[32617]: pgmap v585: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:25.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:25 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:07:25.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:25 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:25.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:25 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:07:25.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:25 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:25.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:25 smithi129 ceph-mon[32569]: pgmap v585: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:25.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:25 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:07:25.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:25 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:25.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:25 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:07:25.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:25 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:26 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:07:26.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:26 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:07:26.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:26 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:07:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:27 smithi078 ceph-mon[29163]: pgmap v586: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:27.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:27 smithi112 ceph-mon[32617]: pgmap v586: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:27.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:27 smithi129 ceph-mon[32569]: pgmap v586: 97 pgs: 97 active+clean; 1.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:29.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:29 smithi112 ceph-mon[32617]: pgmap v587: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:29.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:29 smithi129 ceph-mon[32569]: pgmap v587: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:29.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:29 smithi078 ceph-mon[29163]: pgmap v587: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:31.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:31 smithi112 ceph-mon[32617]: pgmap v588: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:31.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:31 smithi078 ceph-mon[29163]: pgmap v588: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:31.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:31 smithi129 ceph-mon[32569]: pgmap v588: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:33.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:33 smithi112 ceph-mon[32617]: pgmap v589: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:33.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:33 smithi129 ceph-mon[32569]: pgmap v589: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:33.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:33 smithi078 ceph-mon[29163]: pgmap v589: 97 pgs: 97 active+clean; 1.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:35 smithi112 ceph-mon[32617]: pgmap v590: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:35.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:35 smithi129 ceph-mon[32569]: pgmap v590: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:35.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:35 smithi078 ceph-mon[29163]: pgmap v590: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 -- 172.21.15.78:0/813343981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454108970 msgr2=0x7f245410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 --2- 172.21.15.78:0/813343981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454108970 0x7f245410ad90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f244c009ca0 tx=0x7f244c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 -- 172.21.15.78:0/813343981 shutdown_connections 2024-04-25T14:07:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 --2- 172.21.15.78:0/813343981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454108970 0x7f245410ad90 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 --2- 172.21.15.78:0/813343981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2454103c70 0x7f2454104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 --2- 172.21.15.78:0/813343981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454102a30 0x7f2454103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 -- 172.21.15.78:0/813343981 >> 172.21.15.78:0/813343981 conn(0x7f24540fdc20 msgr2=0x7f24541000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 -- 172.21.15.78:0/813343981 shutdown_connections 2024-04-25T14:07:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.293+0000 7f245b5c7640 1 -- 172.21.15.78:0/813343981 wait complete. 2024-04-25T14:07:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 Processor -- start 2024-04-25T14:07:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 -- start start 2024-04-25T14:07:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454102a30 0x7f245419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2454103c70 0x7f245419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454108970 0x7f24541a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f245410e540 con 0x7f2454102a30 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f245410e3c0 con 0x7f2454108970 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f245b5c7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f245410e6c0 con 0x7f2454103c70 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f2459363640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454102a30 0x7f245419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f2459363640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454102a30 0x7f245419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38616/0 (socket says 172.21.15.78:38616) 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f2459363640 1 -- 172.21.15.78:0/1064107096 learned_addr learned my addr 172.21.15.78:0/1064107096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.294+0000 7f2459b64640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454108970 0x7f24541a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2458b62640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2454103c70 0x7f245419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2459363640 1 -- 172.21.15.78:0/1064107096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2454103c70 msgr2=0x7f245419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2459363640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2454103c70 0x7f245419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2459363640 1 -- 172.21.15.78:0/1064107096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454108970 msgr2=0x7f24541a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2459363640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454108970 0x7f24541a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2459363640 1 -- 172.21.15.78:0/1064107096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24541a1c20 con 0x7f2454102a30 2024-04-25T14:07:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2459b64640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454108970 0x7f24541a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f2459363640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454102a30 0x7f245419a190 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f244800d740 tx=0x7f244800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f24467fc640 1 -- 172.21.15.78:0/1064107096 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448017070 con 0x7f2454102a30 2024-04-25T14:07:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f24467fc640 1 -- 172.21.15.78:0/1064107096 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f244800de50 con 0x7f2454102a30 2024-04-25T14:07:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24541a1f10 con 0x7f2454102a30 2024-04-25T14:07:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f24467fc640 1 -- 172.21.15.78:0/1064107096 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448003c20 con 0x7f2454102a30 2024-04-25T14:07:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.295+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24541a23d0 con 0x7f2454102a30 2024-04-25T14:07:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 Processor -- start 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 -- start start 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f24467fc640 1 -- 172.21.15.78:0/1064107096 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2448003dc0 con 0x7f2454102a30 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 0x55f13b21e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae552f0 0x55f13b21eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae5d4d0 0x55f13b225df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f13b184460 con 0x55f13ae552f0 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f13b13e590 con 0x55f13ae5d4d0 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2462474740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f13b180420 con 0x55f13ae7e1b0 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2459363640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae552f0 0x55f13b21eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2459363640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae552f0 0x55f13b21eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38622/0 (socket says 172.21.15.78:38622) 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2459363640 1 -- 172.21.15.78:0/3463843401 learned_addr learned my addr 172.21.15.78:0/3463843401 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.298+0000 7f2458b62640 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae5d4d0 0x55f13b225df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f2459363640 1 -- 172.21.15.78:0/3463843401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 msgr2=0x55f13b21e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f2459363640 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 0x55f13b21e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f2459363640 1 -- 172.21.15.78:0/3463843401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae5d4d0 msgr2=0x55f13b225df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f2459363640 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae5d4d0 0x55f13b225df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f2459363640 1 -- 172.21.15.78:0/3463843401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f13b14b0d0 con 0x55f13ae552f0 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f24467fc640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f24400afb60 0x7f24400b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f2459363640 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae552f0 0x55f13b21eef0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f24480e4c10 tx=0x7f24480e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e984bc44ecb97c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f2459b64640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f24400afb60 0x7f24400b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f243f7fe640 1 -- 172.21.15.78:0/3463843401 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480a2a00 con 0x55f13ae552f0 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f24467fc640 1 -- 172.21.15.78:0/1064107096 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f24480de670 con 0x7f2454102a30 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f243f7fe640 1 -- 172.21.15.78:0/3463843401 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24480a6650 con 0x55f13ae552f0 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.299+0000 7f243f7fe640 1 -- 172.21.15.78:0/3463843401 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480a62b0 con 0x55f13ae552f0 2024-04-25T14:07:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 -- 172.21.15.78:0/3463843401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae552f0 msgr2=0x55f13b21eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2459b64640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f24400afb60 0x7f24400b2070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f244c004af0 tx=0x7f244c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae552f0 0x55f13b21eef0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f24480e4c10 tx=0x7f24480e6690 comp rx=0 tx=0).stop 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 -- 172.21.15.78:0/3463843401 shutdown_connections 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae5d4d0 0x55f13b225df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae552f0 0x55f13b21eef0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 --2- 172.21.15.78:0/3463843401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 0x55f13b21e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 -- 172.21.15.78:0/3463843401 >> 172.21.15.78:0/3463843401 conn(0x55f13ae72300 msgr2=0x55f13b21a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 -- 172.21.15.78:0/3463843401 shutdown_connections 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.300+0000 7f2462474740 1 -- 172.21.15.78:0/3463843401 wait complete. 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 -- start start 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 Processor -- start 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b2286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae5d4d0 0x55f13b228c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae7e1b0 0x55f13b22fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f13b226840 con 0x55f13ae7e1b0 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f13b10bb60 con 0x55f13ae552f0 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f13b2269c0 con 0x55f13ae5d4d0 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2462474740 10 client.0 _ll_register_callbacks cb 0x55f13b1302b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2459b64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae7e1b0 0x55f13b22fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2459b64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae7e1b0 0x55f13b22fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38630/0 (socket says 172.21.15.78:38630) 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2459b64640 1 -- 172.21.15.78:0/3772070982 learned_addr learned my addr 172.21.15.78:0/3772070982 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2458b62640 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae5d4d0 0x55f13b228c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.301+0000 7f2459363640 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b2286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2459b64640 1 -- 172.21.15.78:0/3772070982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae5d4d0 msgr2=0x55f13b228c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2459b64640 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae5d4d0 0x55f13b228c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2459b64640 1 -- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 msgr2=0x55f13b2286d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2459b64640 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b2286d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2459b64640 1 -- 172.21.15.78:0/3772070982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13ae96210 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2459b64640 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae7e1b0 0x55f13b22fb70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f244c035000 tx=0x7f244c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb963d4dfad06656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f243cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f243cff9640 1 -- 172.21.15.78:0/3772070982 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c004510 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f243cff9640 1 -- 172.21.15.78:0/3772070982 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f244c0046b0 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24480a6650 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f243cff9640 1 -- 172.21.15.78:0/3772070982 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f244c02f660 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2462474740 10 client.17916 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f13af62620 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.302+0000 7f243cff9640 1 -- 172.21.15.78:0/3772070982 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24480a6650 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f243cff9640 1 -- 172.21.15.78:0/3772070982 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f244c030960 con 0x55f13ae7e1b0 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2462474740 10 client.17916 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2462474740 10 client.17916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2462474740 20 client.17916 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2462474740 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae3cd80 0x55f13b236440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2462474740 4 client.17916 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f13b192070 con 0x55f13ae3cd80 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2459363640 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae3cd80 0x55f13b236440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f243cff9640 10 client.17916 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.303+0000 7f2459363640 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae3cd80 0x55f13b236440 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aeb15bed415c5f2 server_cookie=e11ea3814089ff57 in_seq=0 out_seq=0 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f243cff9640 1 -- 172.21.15.78:0/3772070982 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f13b192070 con 0x55f13ae3cd80 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f243cff9640 10 client.17916 handle_command_reply: tid=0 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 client.17916 shutdown 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae7e1b0 msgr2=0x55f13b22fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae7e1b0 0x55f13b22fb70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f244c035000 tx=0x7f244c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 shutdown_connections 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae7e1b0 0x55f13b22fb70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae3cd80 0x55f13b236440 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae5d4d0 0x55f13b228c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 --2- 172.21.15.78:0/3772070982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b2286d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f243cff9640 0 client.17916 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.304+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 >> 172.21.15.78:0/3772070982 conn(0x55f13ae72300 msgr2=0x55f13b219250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.305+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 shutdown_connections 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.305+0000 7f2462474740 1 -- 172.21.15.78:0/3772070982 wait complete. 2024-04-25T14:07:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.305+0000 7f2462474740 20 client.17916 trim_cache size 0 max 16384 2024-04-25T14:07:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.319+0000 7f2462474740 1 Processor -- start 2024-04-25T14:07:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2462474740 1 -- start start 2024-04-25T14:07:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2462474740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae552f0 0x55f13b21a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2462474740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b226dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2462474740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae7e1b0 0x55f13b227300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2462474740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24541ada70 con 0x55f13ae5d4d0 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2462474740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24541ab340 con 0x55f13ae7e1b0 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2462474740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24541a9190 con 0x55f13ae552f0 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b226dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b226dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38640/0 (socket says 172.21.15.78:38640) 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 -- 172.21.15.78:0/2401253660 learned_addr learned my addr 172.21.15.78:0/2401253660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 -- 172.21.15.78:0/2401253660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae552f0 msgr2=0x55f13b21a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2459363640 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae552f0 0x55f13b21a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae552f0 0x55f13b21a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 -- 172.21.15.78:0/2401253660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae7e1b0 msgr2=0x55f13b227300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae7e1b0 0x55f13b227300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 -- 172.21.15.78:0/2401253660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f13b21ecd0 con 0x55f13ae5d4d0 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f2458b62640 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b226dc0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f245000c770 tx=0x7f245000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc77c200453b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f243effd640 1 -- 172.21.15.78:0/2401253660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245000ce80 con 0x55f13ae5d4d0 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f243effd640 1 -- 172.21.15.78:0/2401253660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2450003ce0 con 0x55f13ae5d4d0 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.320+0000 7f243effd640 1 -- 172.21.15.78:0/2401253660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245001a740 con 0x55f13ae5d4d0 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 -- 172.21.15.78:0/2401253660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 msgr2=0x55f13b226dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b226dc0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f245000c770 tx=0x7f245000cc30 comp rx=0 tx=0).stop 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 -- 172.21.15.78:0/2401253660 shutdown_connections 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae7e1b0 0x55f13b227300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b226dc0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 --2- 172.21.15.78:0/2401253660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae552f0 0x55f13b21a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 -- 172.21.15.78:0/2401253660 >> 172.21.15.78:0/2401253660 conn(0x55f13ae72300 msgr2=0x55f13b219be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 -- 172.21.15.78:0/2401253660 shutdown_connections 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 -- 172.21.15.78:0/2401253660 wait complete. 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.321+0000 7f2462474740 1 -- start start 2024-04-25T14:07:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 1 Processor -- start 2024-04-25T14:07:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b29a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b29a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 0x55f13b2a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24541a9800 con 0x55f13ae5d4d0 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24541aac70 con 0x55f13ae552f0 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24541a9ed0 con 0x55f13ae7e1b0 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2462474740 10 client.0 _ll_register_callbacks cb 0x7f244c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b29a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b29a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38644/0 (socket says 172.21.15.78:38644) 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 -- 172.21.15.78:0/3647556378 learned_addr learned my addr 172.21.15.78:0/3647556378 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2459363640 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b29a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 -- 172.21.15.78:0/3647556378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 msgr2=0x55f13b2a1500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 0x55f13b2a1500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 -- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 msgr2=0x55f13b29a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b29a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2458b62640 1 -- 172.21.15.78:0/3647556378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13b2a1ac0 con 0x55f13ae5d4d0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.322+0000 7f2459363640 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b29a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2458b62640 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b29a5a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f245000c740 tx=0x7f2450003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a343bd04dea2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f243cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f243cff9640 1 -- 172.21.15.78:0/3647556378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450020040 con 0x55f13ae5d4d0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f243cff9640 1 -- 172.21.15.78:0/3647556378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2450010430 con 0x55f13ae5d4d0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2450003ce0 con 0x55f13ae5d4d0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f243cff9640 1 -- 172.21.15.78:0/3647556378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f24500043b0 con 0x55f13ae5d4d0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 10 client.17922 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f13b2a2d80 con 0x55f13ae5d4d0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f243cff9640 1 -- 172.21.15.78:0/3647556378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2450003ce0 con 0x55f13ae5d4d0 2024-04-25T14:07:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f243cff9640 1 -- 172.21.15.78:0/3647556378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2450023070 con 0x55f13ae5d4d0 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 10 client.17922 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 10 client.17922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 20 client.17922 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae121c0 0x55f13b2a5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 4 client.17922 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.323+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f13b18bee0 con 0x55f13ae121c0 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.324+0000 7f2459363640 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae121c0 0x55f13b2a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.324+0000 7f243cff9640 10 client.17922 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.324+0000 7f2459363640 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae121c0 0x55f13b2a5740 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca74bb2641babec server_cookie=68167f7219b9dff7 in_seq=0 out_seq=0 2024-04-25T14:07:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f243cff9640 1 -- 172.21.15.78:0/3647556378 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f13b18bee0 con 0x55f13ae121c0 2024-04-25T14:07:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f243cff9640 10 client.17922 handle_command_reply: tid=0 2024-04-25T14:07:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 client.17922 shutdown 2024-04-25T14:07:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 msgr2=0x55f13b29a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b29a5a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f245000c740 tx=0x7f2450003e90 comp rx=0 tx=0).stop 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 shutdown_connections 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55f13ae121c0 0x55f13b2a5740 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f13ae7e1b0 0x55f13b2a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f13ae5d4d0 0x55f13b29a5a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 --2- 172.21.15.78:0/3647556378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f13ae552f0 0x55f13b29a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f243cff9640 0 client.17922 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 >> 172.21.15.78:0/3647556378 conn(0x55f13ae72300 msgr2=0x55f13b226dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 shutdown_connections 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.338+0000 7f2462474740 1 -- 172.21.15.78:0/3647556378 wait complete. 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f2462474740 20 client.17922 trim_cache size 0 max 16384 2024-04-25T14:07:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:07:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:07:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:07:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:07:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f24400afb60 msgr2=0x7f24400b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f24400afb60 0x7f24400b2070 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f244c004af0 tx=0x7f244c009360 comp rx=0 tx=0).stop 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454102a30 msgr2=0x7f245419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454102a30 0x7f245419a190 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f244800d740 tx=0x7f244800dc00 comp rx=0 tx=0).stop 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 shutdown_connections 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2454108970 0x7f24541a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2454103c70 0x7f245419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f24400afb60 0x7f24400b2070 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 --2- 172.21.15.78:0/1064107096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2454102a30 0x7f245419a190 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 >> 172.21.15.78:0/1064107096 conn(0x7f24540fdc20 msgr2=0x7f24541010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 shutdown_connections 2024-04-25T14:07:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:37.339+0000 7f245b5c7640 1 -- 172.21.15.78:0/1064107096 wait complete. 2024-04-25T14:07:37.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:37 smithi112 ceph-mon[32617]: pgmap v591: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:37.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:37 smithi129 ceph-mon[32569]: pgmap v591: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:37.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:37 smithi078 ceph-mon[29163]: pgmap v591: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:07:39.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:39 smithi112 ceph-mon[32617]: pgmap v592: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:07:39.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:39 smithi129 ceph-mon[32569]: pgmap v592: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:07:39.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:39 smithi078 ceph-mon[29163]: pgmap v592: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:07:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:41 smithi112 ceph-mon[32617]: pgmap v593: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:07:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:07:41.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:41 smithi129 ceph-mon[32569]: pgmap v593: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:07:41.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:41.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:41.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:41.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:07:41.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:41 smithi078 ceph-mon[29163]: pgmap v593: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:07:41.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:07:41.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:07:41.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:07:41.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:07:43.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:43 smithi112 ceph-mon[32617]: pgmap v594: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:43.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:43 smithi129 ceph-mon[32569]: pgmap v594: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:43.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:43 smithi078 ceph-mon[29163]: pgmap v594: 97 pgs: 97 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:45.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:45 smithi112 ceph-mon[32617]: pgmap v595: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:07:45.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:45 smithi129 ceph-mon[32569]: pgmap v595: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:07:45.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:45 smithi078 ceph-mon[29163]: pgmap v595: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:07:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:47 smithi112 ceph-mon[32617]: pgmap v596: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:47.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:47 smithi129 ceph-mon[32569]: pgmap v596: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:47.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:47 smithi078 ceph-mon[29163]: pgmap v596: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:07:49.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:49 smithi112 ceph-mon[32617]: pgmap v597: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:49.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:49 smithi129 ceph-mon[32569]: pgmap v597: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:49.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:49 smithi078 ceph-mon[29163]: pgmap v597: 97 pgs: 97 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:07:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 -- 172.21.15.78:0/2345843806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220109a80 msgr2=0x7f722010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 --2- 172.21.15.78:0/2345843806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220109a80 0x7f722010bea0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7218009ca0 tx=0x7f721802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 -- 172.21.15.78:0/2345843806 shutdown_connections 2024-04-25T14:07:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 --2- 172.21.15.78:0/2345843806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220109a80 0x7f722010bea0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 --2- 172.21.15.78:0/2345843806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7220104d80 0x7f7220105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 --2- 172.21.15.78:0/2345843806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7220103b40 0x7f7220104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 -- 172.21.15.78:0/2345843806 >> 172.21.15.78:0/2345843806 conn(0x7f72200fed10 msgr2=0x7f72201011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 -- 172.21.15.78:0/2345843806 shutdown_connections 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.445+0000 7f72268aa640 1 -- 172.21.15.78:0/2345843806 wait complete. 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 Processor -- start 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 -- start start 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7220103b40 0x7f722019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220104d80 0x7f722019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7220109a80 0x7f72201a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f722010f670 con 0x7f7220104d80 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f722010f4f0 con 0x7f7220109a80 2024-04-25T14:07:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f72268aa640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f722010f7f0 con 0x7f7220103b40 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220104d80 0x7f722019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220104d80 0x7f722019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45128/0 (socket says 172.21.15.78:45128) 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f721f7fe640 1 -- 172.21.15.78:0/2734559527 learned_addr learned my addr 172.21.15.78:0/2734559527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f7224e47640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7220109a80 0x7f72201a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.446+0000 7f721ffff640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7220103b40 0x7f722019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721f7fe640 1 -- 172.21.15.78:0/2734559527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7220103b40 msgr2=0x7f722019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721f7fe640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7220103b40 0x7f722019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721f7fe640 1 -- 172.21.15.78:0/2734559527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7220109a80 msgr2=0x7f72201a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721f7fe640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7220109a80 0x7f72201a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721f7fe640 1 -- 172.21.15.78:0/2734559527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72201a1cc0 con 0x7f7220104d80 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721f7fe640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220104d80 0x7f722019a6c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f721000d770 tx=0x7f721000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721d7fa640 1 -- 172.21.15.78:0/2734559527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210017070 con 0x7f7220104d80 2024-04-25T14:07:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721d7fa640 1 -- 172.21.15.78:0/2734559527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72100054d0 con 0x7f7220104d80 2024-04-25T14:07:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72201a1fb0 con 0x7f7220104d80 2024-04-25T14:07:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f721d7fa640 1 -- 172.21.15.78:0/2734559527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210004c30 con 0x7f7220104d80 2024-04-25T14:07:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.447+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72201a2410 con 0x7f7220104d80 2024-04-25T14:07:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.449+0000 7f721d7fa640 1 -- 172.21.15.78:0/2734559527 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7210004e10 con 0x7f7220104d80 2024-04-25T14:07:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 Processor -- start 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 -- start start 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6d111b0 0x55e0a70b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e0a7017460 con 0x55e0a6cf04d0 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0a6fd1590 con 0x55e0a6d111b0 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f722d835740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0a7013420 con 0x55e0a6ce82f0 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721d7fa640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f720c0afae0 0x7f720c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45132/0 (socket says 172.21.15.78:45132) 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721f7fe640 1 -- 172.21.15.78:0/3024659356 learned_addr learned my addr 172.21.15.78:0/3024659356 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721ffff640 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f7224e47640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f720c0afae0 0x7f720c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f7224e47640 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6d111b0 0x55e0a70b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721d7fa640 1 -- 172.21.15.78:0/2734559527 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f72100de640 con 0x7f7220104d80 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721f7fe640 1 -- 172.21.15.78:0/3024659356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 msgr2=0x55e0a70b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.450+0000 7f721f7fe640 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f721f7fe640 1 -- 172.21.15.78:0/3024659356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6d111b0 msgr2=0x55e0a70b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f721f7fe640 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6d111b0 0x55e0a70b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f721f7fe640 1 -- 172.21.15.78:0/3024659356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a6fde0d0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f7224e47640 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6d111b0 0x55e0a70b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f721f7fe640 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b8df0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f721001d000 tx=0x7f72100a6a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c19004e4a99711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f7224e47640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f720c0afae0 0x7f720c0b1ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55e0a70b2b40 tx=0x7f7218004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f72167fc640 1 -- 172.21.15.78:0/3024659356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210020030 con 0x55e0a6cf04d0 2024-04-25T14:07:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f72167fc640 1 -- 172.21.15.78:0/3024659356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72100af2c0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.451+0000 7f72167fc640 1 -- 172.21.15.78:0/3024659356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72100a6e20 con 0x55e0a6cf04d0 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 -- 172.21.15.78:0/3024659356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 msgr2=0x55e0a70b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b8df0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f721001d000 tx=0x7f72100a6a40 comp rx=0 tx=0).stop 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 -- 172.21.15.78:0/3024659356 shutdown_connections 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b8df0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 --2- 172.21.15.78:0/3024659356 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6d111b0 0x55e0a70b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 -- 172.21.15.78:0/3024659356 >> 172.21.15.78:0/3024659356 conn(0x55e0a6d05300 msgr2=0x55e0a70ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 -- 172.21.15.78:0/3024659356 shutdown_connections 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 -- 172.21.15.78:0/3024659356 wait complete. 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 -- start start 2024-04-25T14:07:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.452+0000 7f722d835740 1 Processor -- start 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ce82f0 0x55e0a70bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6d111b0 0x55e0a70c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e0a70b97e0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0a6f9eb60 con 0x55e0a6ce82f0 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0a70b9960 con 0x55e0a6d111b0 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 10 client.0 _ll_register_callbacks cb 0x55e0a6fc32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70bbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45146/0 (socket says 172.21.15.78:45146) 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 -- 172.21.15.78:0/2048239296 learned_addr learned my addr 172.21.15.78:0/2048239296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f7224e47640 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6d111b0 0x55e0a70c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 -- 172.21.15.78:0/2048239296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6d111b0 msgr2=0x55e0a70c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6d111b0 0x55e0a70c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 -- 172.21.15.78:0/2048239296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ce82f0 msgr2=0x55e0a70bb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ce82f0 0x55e0a70bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 -- 172.21.15.78:0/2048239296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a6d29210 con 0x55e0a6cf04d0 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f7224e47640 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6d111b0 0x55e0a70c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f721f7fe640 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70bbb50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f72100a68c0 tx=0x7f72100a30f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d29134672c2a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f71ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f71ff7fe640 1 -- 172.21.15.78:0/2048239296 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210020040 con 0x55e0a6cf04d0 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f71ff7fe640 1 -- 172.21.15.78:0/2048239296 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f72100a7030 con 0x55e0a6cf04d0 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.453+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72100af2c0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f71ff7fe640 1 -- 172.21.15.78:0/2048239296 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f72100aaec0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 10 client.17934 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e0a6df5620 con 0x55e0a6cf04d0 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f71ff7fe640 1 -- 172.21.15.78:0/2048239296 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72100af2c0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f71ff7fe640 1 -- 172.21.15.78:0/2048239296 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f72100a26f0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 10 client.17934 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 10 client.17934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 20 client.17934 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6ccfd80 0x55e0a70c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 4 client.17934 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0a7025070 con 0x55e0a6ccfd80 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.454+0000 7f721ffff640 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6ccfd80 0x55e0a70c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.455+0000 7f71ff7fe640 10 client.17934 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.455+0000 7f721ffff640 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6ccfd80 0x55e0a70c8f00 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16bb2c1959ac24d7 server_cookie=7f49e2321a1d04e9 in_seq=0 out_seq=0 2024-04-25T14:07:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.455+0000 7f71ff7fe640 1 -- 172.21.15.78:0/2048239296 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0a7025070 con 0x55e0a6ccfd80 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.455+0000 7f71ff7fe640 10 client.17934 handle_command_reply: tid=0 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 client.17934 shutdown 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 msgr2=0x55e0a70bbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70bbb50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f72100a68c0 tx=0x7f72100a30f0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 shutdown_connections 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6ccfd80 0x55e0a70c8f00 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6d111b0 0x55e0a70c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70bbb50 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f71ff7fe640 0 client.17934 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 --2- 172.21.15.78:0/2048239296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ce82f0 0x55e0a70bb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 >> 172.21.15.78:0/2048239296 conn(0x55e0a6d05300 msgr2=0x55e0a70b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 shutdown_connections 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 1 -- 172.21.15.78:0/2048239296 wait complete. 2024-04-25T14:07:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.456+0000 7f722d835740 20 client.17934 trim_cache size 0 max 16384 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 Processor -- start 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 -- start start 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ccfd80 0x55e0a70acc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6ce82f0 0x55e0a70ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72201adad0 con 0x55e0a6ce82f0 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72201ab3a0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f722d835740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72201a91f0 con 0x55e0a6ccfd80 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6ce82f0 0x55e0a70ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f721f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6ce82f0 0x55e0a70ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45150/0 (socket says 172.21.15.78:45150) 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f721f7fe640 1 -- 172.21.15.78:0/3978274025 learned_addr learned my addr 172.21.15.78:0/3978274025 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.471+0000 7f721ffff640 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ccfd80 0x55e0a70acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f7224e47640 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f721f7fe640 1 -- 172.21.15.78:0/3978274025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ccfd80 msgr2=0x55e0a70acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f721f7fe640 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ccfd80 0x55e0a70acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f721f7fe640 1 -- 172.21.15.78:0/3978274025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6cf04d0 msgr2=0x55e0a70b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f721f7fe640 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f721f7fe640 1 -- 172.21.15.78:0/3978274025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a70a8a60 con 0x55e0a6ce82f0 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f7224e47640 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f721f7fe640 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6ce82f0 0x55e0a70ad190 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f72100ab080 tx=0x7f72100dc180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4894d3ab238ee8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f7215ffb640 1 -- 172.21.15.78:0/3978274025 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210020030 con 0x55e0a6ce82f0 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f7215ffb640 1 -- 172.21.15.78:0/3978274025 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72100af2c0 con 0x55e0a6ce82f0 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.472+0000 7f7215ffb640 1 -- 172.21.15.78:0/3978274025 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72100de110 con 0x55e0a6ce82f0 2024-04-25T14:07:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 -- 172.21.15.78:0/3978274025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6ce82f0 msgr2=0x55e0a70ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6ce82f0 0x55e0a70ad190 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f72100ab080 tx=0x7f72100dc180 comp rx=0 tx=0).stop 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 -- 172.21.15.78:0/3978274025 shutdown_connections 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6cf04d0 0x55e0a70b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6ce82f0 0x55e0a70ad190 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 --2- 172.21.15.78:0/3978274025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ccfd80 0x55e0a70acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 -- 172.21.15.78:0/3978274025 >> 172.21.15.78:0/3978274025 conn(0x55e0a6d05300 msgr2=0x55e0a70b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 -- 172.21.15.78:0/3978274025 shutdown_connections 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 -- 172.21.15.78:0/3978274025 wait complete. 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 -- start start 2024-04-25T14:07:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.473+0000 7f722d835740 1 Processor -- start 2024-04-25T14:07:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ccfd80 0x55e0a70bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72201a9860 con 0x55e0a6cf04d0 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72201aacd0 con 0x55e0a6ccfd80 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72201a9f30 con 0x55e0a6ce82f0 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 10 client.0 _ll_register_callbacks cb 0x7f720800a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45160/0 (socket says 172.21.15.78:45160) 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 -- 172.21.15.78:0/3381501166 learned_addr learned my addr 172.21.15.78:0/3381501166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f721ffff640 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ccfd80 0x55e0a70bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f721f7fe640 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 -- 172.21.15.78:0/3381501166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 msgr2=0x55e0a70bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 -- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ccfd80 msgr2=0x55e0a70bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ccfd80 0x55e0a70bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 -- 172.21.15.78:0/3381501166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a70c1190 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f7224e47640 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70c0bd0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f7218009ca0 tx=0x7f721802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5347f39de4a8de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f71ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f71ff7fe640 1 -- 172.21.15.78:0/3381501166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218004590 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f71ff7fe640 1 -- 172.21.15.78:0/3381501166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7218004730 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72100af2c0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f71ff7fe640 1 -- 172.21.15.78:0/3381501166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f721802f660 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 10 client.17940 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.474+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e0a70c2400 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f71ff7fe640 1 -- 172.21.15.78:0/3381501166 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72100af2c0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f71ff7fe640 1 -- 172.21.15.78:0/3381501166 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f72180308f0 con 0x55e0a6cf04d0 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f722d835740 10 client.17940 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f722d835740 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f722d835740 20 client.17940 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f722d835740 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6d111b0 0x55e0a7133a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f722d835740 4 client.17940 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:07:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0a701eee0 con 0x55e0a6d111b0 2024-04-25T14:07:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f721ffff640 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6d111b0 0x55e0a7133a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f71ff7fe640 10 client.17940 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.475+0000 7f721ffff640 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6d111b0 0x55e0a7133a10 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0cd965c2f7708e server_cookie=3dad6c1812728b26 in_seq=0 out_seq=0 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.489+0000 7f71ff7fe640 1 -- 172.21.15.78:0/3381501166 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0a701eee0 con 0x55e0a6d111b0 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.489+0000 7f71ff7fe640 10 client.17940 handle_command_reply: tid=0 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 client.17940 shutdown 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 msgr2=0x55e0a70c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70c0bd0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f7218009ca0 tx=0x7f721802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 shutdown_connections 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0a6cf04d0 0x55e0a70c0bd0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0a6d111b0 0x55e0a7133a10 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0a6ce82f0 0x55e0a70bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 --2- 172.21.15.78:0/3381501166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0a6ccfd80 0x55e0a70bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f71ff7fe640 0 client.17940 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 >> 172.21.15.78:0/3381501166 conn(0x55e0a6d05300 msgr2=0x55e0a70b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 shutdown_connections 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 1 -- 172.21.15.78:0/3381501166 wait complete. 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f722d835740 20 client.17940 trim_cache size 0 max 16384 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:07:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:07:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f720c0afae0 msgr2=0x7f720c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f72268aa640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f720c0afae0 0x7f720c0b1ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55e0a70b2b40 tx=0x7f7218004a40 comp rx=0 tx=0).stop 2024-04-25T14:07:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220104d80 msgr2=0x7f722019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.490+0000 7f72268aa640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220104d80 0x7f722019a6c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f721000d770 tx=0x7f721000dc30 comp rx=0 tx=0).stop 2024-04-25T14:07:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 shutdown_connections 2024-04-25T14:07:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7220109a80 0x7f72201a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f720c0afae0 0x7f720c0b1ff0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7220104d80 0x7f722019a6c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 --2- 172.21.15.78:0/2734559527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7220103b40 0x7f722019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 >> 172.21.15.78:0/2734559527 conn(0x7f72200fed10 msgr2=0x7f7220105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 shutdown_connections 2024-04-25T14:07:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:07:50.491+0000 7f72268aa640 1 -- 172.21.15.78:0/2734559527 wait complete. 2024-04-25T14:07:51.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:51 smithi112 ceph-mon[32617]: pgmap v598: 97 pgs: 97 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T14:07:51.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:51 smithi129 ceph-mon[32569]: pgmap v598: 97 pgs: 97 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T14:07:52.054 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:51 smithi078 ceph-mon[29163]: pgmap v598: 97 pgs: 97 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T14:07:53.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:53 smithi112 ceph-mon[32617]: pgmap v599: 97 pgs: 97 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:53.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:53 smithi129 ceph-mon[32569]: pgmap v599: 97 pgs: 97 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:53.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:53 smithi078 ceph-mon[29163]: pgmap v599: 97 pgs: 97 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:07:55.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:07:55.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:07:55.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 -- 172.21.15.78:0/4121947499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd788106f40 msgr2=0x7fd788109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 --2- 172.21.15.78:0/4121947499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd788106f40 0x7fd788109360 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009900 tx=0x7fd77c02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:55.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 -- 172.21.15.78:0/4121947499 shutdown_connections 2024-04-25T14:07:55.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 --2- 172.21.15.78:0/4121947499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7881098a0 0x7fd78810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 --2- 172.21.15.78:0/4121947499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd788106f40 0x7fd788109360 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 --2- 172.21.15.78:0/4121947499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7881045e0 0x7fd788106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 -- 172.21.15.78:0/4121947499 >> 172.21.15.78:0/4121947499 conn(0x7fd7880fdc00 msgr2=0x7fd788100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 -- 172.21.15.78:0/4121947499 shutdown_connections 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.153+0000 7fd78f00c640 1 -- 172.21.15.78:0/4121947499 wait complete. 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 Processor -- start 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 -- start start 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7881045e0 0x7fd78819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd788106f40 0x7fd78819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7881098a0 0x7fd7881a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810f6b0 con 0x7fd7881098a0 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810f530 con 0x7fd7881045e0 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78f00c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810f830 con 0x7fd788106f40 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7881098a0 0x7fd7881a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7881098a0 0x7fd7881a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45174/0 (socket says 172.21.15.78:45174) 2024-04-25T14:07:55.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1543668314 learned_addr learned my addr 172.21.15.78:0/1543668314 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7881045e0 0x7fd78819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd787fff640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd788106f40 0x7fd78819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1543668314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd788106f40 msgr2=0x7fd78819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd788106f40 0x7fd78819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1543668314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7881045e0 msgr2=0x7fd78819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7881045e0 0x7fd78819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1543668314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7881a1df0 con 0x7fd7881098a0 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.154+0000 7fd787fff640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd788106f40 0x7fd78819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.155+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7881098a0 0x7fd7881a1780 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd78000b9e0 tx=0x7fd78000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.155+0000 7fd785ffb640 1 -- 172.21.15.78:0/1543668314 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780019070 con 0x7fd7881098a0 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.155+0000 7fd785ffb640 1 -- 172.21.15.78:0/1543668314 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd780014440 con 0x7fd7881098a0 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.155+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7881a20e0 con 0x7fd7881098a0 2024-04-25T14:07:55.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.155+0000 7fd785ffb640 1 -- 172.21.15.78:0/1543668314 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800125a0 con 0x7fd7881098a0 2024-04-25T14:07:55.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.155+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7881a26b0 con 0x7fd7881098a0 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.157+0000 7fd785ffb640 1 -- 172.21.15.78:0/1543668314 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd78001c070 con 0x7fd7881098a0 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 Processor -- start 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 -- start start 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660a3390 0x564c66448e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd785ffb640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd7700afb70 0x7fd7700b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c664493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660826b0 0x564c6644a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c663a9d80 con 0x564c6607a4d0 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c663638b0 con 0x564c660826b0 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd795f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564c663a5d40 con 0x564c660a3390 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd7700afb70 0x7fd7700b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd787fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c664493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd787fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c664493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45186/0 (socket says 172.21.15.78:45186) 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd787fff640 1 -- 172.21.15.78:0/1107309203 learned_addr learned my addr 172.21.15.78:0/1107309203 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd785ffb640 1 -- 172.21.15.78:0/1543668314 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd7800de0d0 con 0x7fd7881098a0 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660826b0 0x564c6644a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.158+0000 7fd787fff640 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660a3390 0x564c66448e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd787fff640 1 -- 172.21.15.78:0/1107309203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660a3390 msgr2=0x564c66448e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd787fff640 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660a3390 0x564c66448e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd787fff640 1 -- 172.21.15.78:0/1107309203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660826b0 msgr2=0x564c6644a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd787fff640 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660826b0 0x564c6644a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd787fff640 1 -- 172.21.15.78:0/1107309203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c66334a40 con 0x564c6607a4d0 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660826b0 0x564c6644a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd7700afb70 0x7fd7700b2080 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd778005fd0 tx=0x7fd77800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd787fff640 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c664493f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005ab0 tx=0x7fd77c009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794ba7d6c6c34dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd76effd640 1 -- 172.21.15.78:0/1107309203 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c035040 con 0x564c6607a4d0 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd76effd640 1 -- 172.21.15.78:0/1107309203 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd77c032d10 con 0x564c6607a4d0 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.159+0000 7fd76effd640 1 -- 172.21.15.78:0/1107309203 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c004ae0 con 0x564c6607a4d0 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 -- 172.21.15.78:0/1107309203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 msgr2=0x564c664493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c664493f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd77c005ab0 tx=0x7fd77c009660 comp rx=0 tx=0).stop 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 -- 172.21.15.78:0/1107309203 shutdown_connections 2024-04-25T14:07:55.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660826b0 0x564c6644a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c664493f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 --2- 172.21.15.78:0/1107309203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660a3390 0x564c66448e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 -- 172.21.15.78:0/1107309203 >> 172.21.15.78:0/1107309203 conn(0x564c660974e0 msgr2=0x564c66440270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 -- 172.21.15.78:0/1107309203 shutdown_connections 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 -- 172.21.15.78:0/1107309203 wait complete. 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 -- start start 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.160+0000 7fd795f99740 1 Processor -- start 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd795f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c6607a4d0 0x564c66452e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd795f99740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660826b0 0x564c664533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd795f99740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c6645a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd795f99740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c6644ab30 con 0x564c660826b0 2024-04-25T14:07:55.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd795f99740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564c66330d10 con 0x564c660a3390 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd795f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564c6644acb0 con 0x564c6607a4d0 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd795f99740 10 client.0 _ll_register_callbacks cb 0x564c6635d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660826b0 0x564c664533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660826b0 0x564c664533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45200/0 (socket says 172.21.15.78:45200) 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 -- 172.21.15.78:0/2399612836 learned_addr learned my addr 172.21.15.78:0/2399612836 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c6645a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 -- 172.21.15.78:0/2399612836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c6607a4d0 msgr2=0x564c66452e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c6607a4d0 0x564c66452e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 -- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 msgr2=0x564c6645a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c6645a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.161+0000 7fd787fff640 1 -- 172.21.15.78:0/2399612836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c661857d0 con 0x564c660826b0 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd787fff640 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660826b0 0x564c664533b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd77c02a450 tx=0x7fd77c031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b2647fdaff9764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd767fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd767fff640 1 -- 172.21.15.78:0/2399612836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c035050 con 0x564c660826b0 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd767fff640 1 -- 172.21.15.78:0/2399612836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd77c005430 con 0x564c660826b0 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77c032d10 con 0x564c660826b0 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd767fff640 1 -- 172.21.15.78:0/2399612836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd77c004860 con 0x564c660826b0 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 10 client.17952 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564c660bb3f0 con 0x564c660826b0 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd767fff640 1 -- 172.21.15.78:0/2399612836 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd77c032d10 con 0x564c660826b0 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd767fff640 1 -- 172.21.15.78:0/2399612836 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd77c03a040 con 0x564c660826b0 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 10 client.17952 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 20 client.17952 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c66061f60 0x564c664603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 4 client.17952 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.162+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c663b7990 con 0x564c66061f60 2024-04-25T14:07:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.163+0000 7fd78cda8640 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c66061f60 0x564c664603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.163+0000 7fd767fff640 10 client.17952 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.163+0000 7fd78cda8640 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c66061f60 0x564c664603d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6170143284f541b server_cookie=5176d5e85bd128de in_seq=0 out_seq=0 2024-04-25T14:07:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd767fff640 1 -- 172.21.15.78:0/2399612836 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c663b7990 con 0x564c66061f60 2024-04-25T14:07:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd767fff640 10 client.17952 handle_command_reply: tid=0 2024-04-25T14:07:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 client.17952 shutdown 2024-04-25T14:07:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660826b0 msgr2=0x564c664533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660826b0 0x564c664533b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd77c02a450 tx=0x7fd77c031660 comp rx=0 tx=0).stop 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 shutdown_connections 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c6645a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660826b0 0x564c664533b0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c66061f60 0x564c664603d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 --2- 172.21.15.78:0/2399612836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c6607a4d0 0x564c66452e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd767fff640 0 client.17952 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 >> 172.21.15.78:0/2399612836 conn(0x564c660974e0 msgr2=0x564c66443180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 shutdown_connections 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 1 -- 172.21.15.78:0/2399612836 wait complete. 2024-04-25T14:07:55.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.164+0000 7fd795f99740 20 client.17952 trim_cache size 0 max 16384 2024-04-25T14:07:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.182+0000 7fd795f99740 1 Processor -- start 2024-04-25T14:07:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd795f99740 1 -- start start 2024-04-25T14:07:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd795f99740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c6643fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd795f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66440150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd795f99740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c66451490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd795f99740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881ab9e0 con 0x564c6607a4d0 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd795f99740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881a92b0 con 0x564c660a3390 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd795f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881a7100 con 0x564c660826b0 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c6643fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c6643fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45202/0 (socket says 172.21.15.78:45202) 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 -- 172.21.15.78:0/1621430889 learned_addr learned my addr 172.21.15.78:0/1621430889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c66451490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd787fff640 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66440150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 -- 172.21.15.78:0/1621430889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 msgr2=0x564c66440150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66440150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 -- 172.21.15.78:0/1621430889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 msgr2=0x564c66451490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c66451490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 -- 172.21.15.78:0/1621430889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c6607b0f0 con 0x564c6607a4d0 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd787fff640 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66440150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c6643fc10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd778004650 tx=0x7fd778013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0fdd251ac33d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.183+0000 7fd76e7fc640 1 -- 172.21.15.78:0/1621430889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778011e20 con 0x564c6607a4d0 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd76e7fc640 1 -- 172.21.15.78:0/1621430889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7780054d0 con 0x564c6607a4d0 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd76e7fc640 1 -- 172.21.15.78:0/1621430889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778025250 con 0x564c6607a4d0 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 -- 172.21.15.78:0/1621430889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 msgr2=0x564c6643fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c6643fc10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd778004650 tx=0x7fd778013070 comp rx=0 tx=0).stop 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 -- 172.21.15.78:0/1621430889 shutdown_connections 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c660a3390 0x564c66451490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66440150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 --2- 172.21.15.78:0/1621430889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c6607a4d0 0x564c6643fc10 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 -- 172.21.15.78:0/1621430889 >> 172.21.15.78:0/1621430889 conn(0x564c660974e0 msgr2=0x564c6644dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.184+0000 7fd795f99740 1 -- 172.21.15.78:0/1621430889 shutdown_connections 2024-04-25T14:07:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 -- 172.21.15.78:0/1621430889 wait complete. 2024-04-25T14:07:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 -- start start 2024-04-25T14:07:55.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 Processor -- start 2024-04-25T14:07:55.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6607a4d0 0x564c664569a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66456ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660a3390 0x564c664c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881a7770 con 0x564c660a3390 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881a8be0 con 0x564c6607a4d0 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881a7e40 con 0x564c660826b0 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd795f99740 10 client.0 _ll_register_callbacks cb 0x7fd7800a2510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd78d5a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660a3390 0x564c664c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.185+0000 7fd78d5a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660a3390 0x564c664c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45216/0 (socket says 172.21.15.78:45216) 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1693534625 learned_addr learned my addr 172.21.15.78:0/1693534625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6607a4d0 0x564c664569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd787fff640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66456ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1693534625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 msgr2=0x564c66456ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66456ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6607a4d0 msgr2=0x564c664569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6607a4d0 0x564c664569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78d5a9640 1 -- 172.21.15.78:0/1693534625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c6636a4b0 con 0x564c660a3390 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd787fff640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66456ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd78d5a9640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660a3390 0x564c664c6660 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd7800dbbc0 tx=0x7fd780014cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54da4362b7948141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd767fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd767fff640 1 -- 172.21.15.78:0/1693534625 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800a6650 con 0x564c660a3390 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd767fff640 1 -- 172.21.15.78:0/1693534625 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd78008f0b0 con 0x564c660a3390 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7780054d0 con 0x564c660a3390 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd767fff640 1 -- 172.21.15.78:0/1693534625 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd7800a6f30 con 0x564c660a3390 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd795f99740 10 client.17958 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564c664c7c30 con 0x564c660a3390 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.186+0000 7fd767fff640 1 -- 172.21.15.78:0/1693534625 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7780054d0 con 0x564c660a3390 2024-04-25T14:07:55.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd767fff640 1 -- 172.21.15.78:0/1693534625 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd7800cb0f0 con 0x564c660a3390 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd795f99740 10 client.17958 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd795f99740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd795f99740 20 client.17958 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd795f99740 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c660373a0 0x564c664ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd795f99740 4 client.17958 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c663b1800 con 0x564c660373a0 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c660373a0 0x564c664ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd78cda8640 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c660373a0 0x564c664ca650 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a264f2b499633971 server_cookie=bda46818f2cdb1f1 in_seq=0 out_seq=0 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd767fff640 10 client.17958 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd767fff640 1 -- 172.21.15.78:0/1693534625 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564c663b1800 con 0x564c660373a0 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd767fff640 10 client.17958 handle_command_reply: tid=0 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.187+0000 7fd795f99740 1 client.17958 shutdown 2024-04-25T14:07:55.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660a3390 msgr2=0x564c664c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660a3390 0x564c664c6660 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd7800dbbc0 tx=0x7fd780014cf0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 shutdown_connections 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c660a3390 0x564c664c6660 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564c660373a0 0x564c664ca650 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564c660826b0 0x564c66456ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 --2- 172.21.15.78:0/1693534625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564c6607a4d0 0x564c664569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd767fff640 0 client.17958 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 >> 172.21.15.78:0/1693534625 conn(0x564c660974e0 msgr2=0x564c6643fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 shutdown_connections 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 1 -- 172.21.15.78:0/1693534625 wait complete. 2024-04-25T14:07:55.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd795f99740 20 client.17958 trim_cache size 0 max 16384 2024-04-25T14:07:55.195 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:07:55.195 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:07:55.195 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:07:55.195 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd7700afb70 msgr2=0x7fd7700b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd78f00c640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd7700afb70 0x7fd7700b2080 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd778005fd0 tx=0x7fd77800b390 comp rx=0 tx=0).stop 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7881098a0 msgr2=0x7fd7881a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.188+0000 7fd78f00c640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7881098a0 0x7fd7881a1780 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd78000b9e0 tx=0x7fd78000bea0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 shutdown_connections 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd7700afb70 0x7fd7700b2080 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7881098a0 0x7fd7881a1780 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd788106f40 0x7fd78819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 --2- 172.21.15.78:0/1543668314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7881045e0 0x7fd78819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 >> 172.21.15.78:0/1543668314 conn(0x7fd7880fdc00 msgr2=0x7fd788105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 shutdown_connections 2024-04-25T14:07:55.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.189+0000 7fd78f00c640 1 -- 172.21.15.78:0/1543668314 wait complete. 2024-04-25T14:07:55.200 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:07:55.200 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:07:55.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 -- 172.21.15.78:0/956857707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c103ad0 msgr2=0x7ff66c1047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 --2- 172.21.15.78:0/956857707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c103ad0 0x7ff66c1047d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ff664009ca0 tx=0x7ff66402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 -- 172.21.15.78:0/956857707 shutdown_connections 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 --2- 172.21.15.78:0/956857707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff66c109a10 0x7ff66c10be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 --2- 172.21.15.78:0/956857707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff66c104d10 0x7ff66c105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 --2- 172.21.15.78:0/956857707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c103ad0 0x7ff66c1047d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 -- 172.21.15.78:0/956857707 >> 172.21.15.78:0/956857707 conn(0x7ff66c0fece0 msgr2=0x7ff66c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 -- 172.21.15.78:0/956857707 shutdown_connections 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 -- 172.21.15.78:0/956857707 wait complete. 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 Processor -- start 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.309+0000 7ff672bb7640 1 -- start start 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff672bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff66c103ad0 0x7ff66c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff672bb7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff66c104d10 0x7ff66c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff672bb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c109a10 0x7ff66c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff672bb7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10f5d0 con 0x7ff66c109a10 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff672bb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10f450 con 0x7ff66c103ad0 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff672bb7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10f750 con 0x7ff66c104d10 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c109a10 0x7ff66c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c109a10 0x7ff66c1a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45234/0 (socket says 172.21.15.78:45234) 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 -- 172.21.15.78:0/3181121344 learned_addr learned my addr 172.21.15.78:0/3181121344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff66bfff640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff66c104d10 0x7ff66c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff670953640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff66c103ad0 0x7ff66c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 -- 172.21.15.78:0/3181121344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff66c104d10 msgr2=0x7ff66c19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff66c104d10 0x7ff66c19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 -- 172.21.15.78:0/3181121344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff66c103ad0 msgr2=0x7ff66c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff66c103ad0 0x7ff66c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 -- 172.21.15.78:0/3181121344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1a1d50 con 0x7ff66c109a10 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff671154640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c109a10 0x7ff66c1a16e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c970 tx=0x7ff65400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff669ffb640 1 -- 172.21.15.78:0/3181121344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654007bb0 con 0x7ff66c109a10 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff669ffb640 1 -- 172.21.15.78:0/3181121344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff654007d50 con 0x7ff66c109a10 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c1a1fe0 con 0x7ff66c109a10 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.310+0000 7ff669ffb640 1 -- 172.21.15.78:0/3181121344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654004e60 con 0x7ff66c109a10 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.311+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1a24f0 con 0x7ff66c109a10 2024-04-25T14:07:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.311+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c108820 con 0x7ff66c109a10 2024-04-25T14:07:55.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.314+0000 7ff669ffb640 1 -- 172.21.15.78:0/3181121344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff654005000 con 0x7ff66c109a10 2024-04-25T14:07:55.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.314+0000 7ff669ffb640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6580afb70 0x7ff6580b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.314+0000 7ff670953640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6580afb70 0x7ff6580b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.314+0000 7ff669ffb640 1 -- 172.21.15.78:0/3181121344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7ff6540debd0 con 0x7ff66c109a10 2024-04-25T14:07:55.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.315+0000 7ff670953640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6580afb70 0x7ff6580b2080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff6640094e0 tx=0x7ff664005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.318+0000 7ff669ffb640 1 -- 172.21.15.78:0/3181121344 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff654014030 con 0x7ff66c109a10 2024-04-25T14:07:55.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.533+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff66c02db50 con 0x7ff66c109a10 2024-04-25T14:07:55.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.534+0000 7ff669ffb640 1 -- 172.21.15.78:0/3181121344 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7ff6540a7820 con 0x7ff66c109a10 2024-04-25T14:07:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:07:55.534 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:07:55.534 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:07:55.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6580afb70 msgr2=0x7ff6580b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6580afb70 0x7ff6580b2080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff6640094e0 tx=0x7ff664005fd0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c109a10 msgr2=0x7ff66c1a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c109a10 0x7ff66c1a16e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c970 tx=0x7ff65400ce30 comp rx=0 tx=0).stop 2024-04-25T14:07:55.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 shutdown_connections 2024-04-25T14:07:55.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7ff6580afb70 0x7ff6580b2080 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff66c109a10 0x7ff66c1a16e0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff66c104d10 0x7ff66c19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 --2- 172.21.15.78:0/3181121344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff66c103ad0 0x7ff66c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 >> 172.21.15.78:0/3181121344 conn(0x7ff66c0fece0 msgr2=0x7ff66c1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 shutdown_connections 2024-04-25T14:07:55.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.536+0000 7ff672bb7640 1 -- 172.21.15.78:0/3181121344 wait complete. 2024-04-25T14:07:55.547 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 -- 172.21.15.78:0/2648377112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4103c20 msgr2=0x7f23f4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/2648377112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4103c20 0x7f23f4104030 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009900 tx=0x7f23e802a410 comp rx=0 tx=0).stop 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 -- 172.21.15.78:0/2648377112 shutdown_connections 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/2648377112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f4108920 0x7f23f410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/2648377112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4103c20 0x7f23f4104030 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/2648377112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23f41029e0 0x7f23f41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 -- 172.21.15.78:0/2648377112 >> 172.21.15.78:0/2648377112 conn(0x7f23f40fdbd0 msgr2=0x7f23f4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 -- 172.21.15.78:0/2648377112 shutdown_connections 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.657+0000 7f23f8ae7640 1 -- 172.21.15.78:0/2648377112 wait complete. 2024-04-25T14:07:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 Processor -- start 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 -- start start 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f41029e0 0x7f23f406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f4103c20 0x7f23f419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23f4108920 0x7f23f41a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410e500 con 0x7f23f41029e0 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410e380 con 0x7f23f4103c20 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f8ae7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23f410e680 con 0x7f23f4108920 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f41029e0 0x7f23f406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f41029e0 0x7f23f406a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45244/0 (socket says 172.21.15.78:45244) 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 -- 172.21.15.78:0/3736397426 learned_addr learned my addr 172.21.15.78:0/3736397426 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23f4108920 0x7f23f41a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f4103c20 0x7f23f419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 -- 172.21.15.78:0/3736397426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23f4108920 msgr2=0x7f23f41a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23f4108920 0x7f23f41a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 -- 172.21.15.78:0/3736397426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f4103c20 msgr2=0x7f23f419f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f4103c20 0x7f23f419f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.658+0000 7f23f259c640 1 -- 172.21.15.78:0/3736397426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f41a2070 con 0x7f23f41029e0 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.659+0000 7f23f259c640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f41029e0 0x7f23f406a920 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f23e4005d00 tx=0x7f23e400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.659+0000 7f23e37fe640 1 -- 172.21.15.78:0/3736397426 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4017070 con 0x7f23f41029e0 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.659+0000 7f23e37fe640 1 -- 172.21.15.78:0/3736397426 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e4009c30 con 0x7f23f41029e0 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.659+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f41a2360 con 0x7f23f41029e0 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.659+0000 7f23e37fe640 1 -- 172.21.15.78:0/3736397426 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e400f620 con 0x7f23f41029e0 2024-04-25T14:07:55.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.659+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f41a2860 con 0x7f23f41029e0 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.661+0000 7f23e37fe640 1 -- 172.21.15.78:0/3736397426 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f23e401a070 con 0x7f23f41029e0 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.661+0000 7f23ff997740 1 Processor -- start 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23ff997740 1 -- start start 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23ff997740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa84c390 0x5635fabf2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23ff997740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabf31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23ff997740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa82b6b0 0x5635fabf3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23ff997740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5635fab52d80 con 0x5635fa84c390 2024-04-25T14:07:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23ff997740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5635fab0ca20 con 0x5635fa8234d0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23ff997740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635fab4ed40 con 0x5635fa82b6b0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23e37fe640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23dc0afaf0 0x7f23dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa84c390 0x5635fabf2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa84c390 0x5635fabf2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45256/0 (socket says 172.21.15.78:45256) 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f2d9d640 1 -- 172.21.15.78:0/1230488433 learned_addr learned my addr 172.21.15.78:0/1230488433 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23dc0afaf0 0x7f23dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa82b6b0 0x5635fabf3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23e37fe640 1 -- 172.21.15.78:0/3736397426 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f23e40de5e0 con 0x7f23f41029e0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f259c640 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabf31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f1d9b640 1 -- 172.21.15.78:0/1230488433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 msgr2=0x5635fabf31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabf31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f1d9b640 1 -- 172.21.15.78:0/1230488433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa84c390 msgr2=0x5635fabf2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa84c390 0x5635fabf2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.662+0000 7f23f1d9b640 1 -- 172.21.15.78:0/1230488433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fab17c30 con 0x5635fa82b6b0 2024-04-25T14:07:55.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.663+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23dc0afaf0 0x7f23dc0b2000 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005c20 tx=0x7f23ec009bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.663+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa82b6b0 0x5635fabf3e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23e8005ab0 tx=0x7f23e8031be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a13cb0aeaa117a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.663+0000 7f23e0ff9640 1 -- 172.21.15.78:0/1230488433 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8035030 con 0x5635fa82b6b0 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.663+0000 7f23e0ff9640 1 -- 172.21.15.78:0/1230488433 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e8040070 con 0x5635fa82b6b0 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.663+0000 7f23e0ff9640 1 -- 172.21.15.78:0/1230488433 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e80038e0 con 0x5635fa82b6b0 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 -- 172.21.15.78:0/1230488433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa82b6b0 msgr2=0x5635fabf3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa82b6b0 0x5635fabf3e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23e8005ab0 tx=0x7f23e8031be0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 -- 172.21.15.78:0/1230488433 shutdown_connections 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa82b6b0 0x5635fabf3e30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabf31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 --2- 172.21.15.78:0/1230488433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa84c390 0x5635fabf2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 -- 172.21.15.78:0/1230488433 >> 172.21.15.78:0/1230488433 conn(0x5635fa8404e0 msgr2=0x5635fabea020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 -- 172.21.15.78:0/1230488433 shutdown_connections 2024-04-25T14:07:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 -- 172.21.15.78:0/1230488433 wait complete. 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 -- start start 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.664+0000 7f23ff997740 1 Processor -- start 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa8234d0 0x5635fabfc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa82b6b0 0x5635fabfcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa84c390 0x5635fac03e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5635fabf48e0 con 0x5635fa8234d0 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5635faad9dc0 con 0x5635fa82b6b0 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635fabf4a60 con 0x5635fa84c390 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 10 client.0 _ll_register_callbacks cb 0x5635faaf5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa8234d0 0x5635fabfc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa8234d0 0x5635fabfc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45272/0 (socket says 172.21.15.78:45272) 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 -- 172.21.15.78:0/206587822 learned_addr learned my addr 172.21.15.78:0/206587822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa82b6b0 0x5635fabfcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa84c390 0x5635fac03e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 -- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa84c390 msgr2=0x5635fac03e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa84c390 0x5635fac03e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 -- 172.21.15.78:0/206587822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa82b6b0 msgr2=0x5635fabfcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa82b6b0 0x5635fabfcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 -- 172.21.15.78:0/206587822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fa8643f0 con 0x5635fa8234d0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa82b6b0 0x5635fabfcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa84c390 0x5635fac03e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23f259c640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa8234d0 0x5635fabfc9a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f23e40a3dd0 tx=0x7f23e40a6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82dbf5f2629fa38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23d9ffb640 1 -- 172.21.15.78:0/206587822 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4021040 con 0x5635fa8234d0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23d9ffb640 1 -- 172.21.15.78:0/206587822 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f23e40a5dc0 con 0x5635fa8234d0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.665+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e8040070 con 0x5635fa8234d0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23d9ffb640 1 -- 172.21.15.78:0/206587822 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f23e40a6670 con 0x5635fa8234d0 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 10 client.17976 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5635fa92e7d0 con 0x5635fa8234d0 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23d9ffb640 1 -- 172.21.15.78:0/206587822 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e8040070 con 0x5635fa8234d0 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23d9ffb640 1 -- 172.21.15.78:0/206587822 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f23e40a2040 con 0x5635fa8234d0 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 10 client.17976 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 10 client.17976 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 20 client.17976 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa80af60 0x5635fac0a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 4 client.17976 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635fab60990 con 0x5635fa80af60 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.666+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa80af60 0x5635fac0a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.667+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa80af60 0x5635fac0a530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a18b5df3037541f server_cookie=1de44e84f82d21c2 in_seq=0 out_seq=0 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.667+0000 7f23d9ffb640 10 client.17976 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23d9ffb640 1 -- 172.21.15.78:0/206587822 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5635fab60990 con 0x5635fa80af60 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23d9ffb640 10 client.17976 handle_command_reply: tid=0 2024-04-25T14:07:55.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 client.17976 shutdown 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa8234d0 msgr2=0x5635fabfc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa8234d0 0x5635fabfc9a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f23e40a3dd0 tx=0x7f23e40a6a20 comp rx=0 tx=0).stop 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 shutdown_connections 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa80af60 0x5635fac0a530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa84c390 0x5635fac03e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23d9ffb640 0 client.17976 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa82b6b0 0x5635fabfcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 --2- 172.21.15.78:0/206587822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa8234d0 0x5635fabfc9a0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 >> 172.21.15.78:0/206587822 conn(0x5635fa8404e0 msgr2=0x5635fabe9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 shutdown_connections 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 1 -- 172.21.15.78:0/206587822 wait complete. 2024-04-25T14:07:55.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.668+0000 7f23ff997740 20 client.17976 trim_cache size 0 max 16384 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.681+0000 7f23ff997740 1 Processor -- start 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.681+0000 7f23ff997740 1 -- start start 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.681+0000 7f23ff997740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa80af60 0x5635fabfcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.681+0000 7f23ff997740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabe8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.681+0000 7f23ff997740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fabe8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.681+0000 7f23ff997740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41ab990 con 0x5635fa82b6b0 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.681+0000 7f23ff997740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a9260 con 0x5635fa8234d0 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23ff997740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a70b0 con 0x5635fa80af60 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fabe8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fabe8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45284/0 (socket says 172.21.15.78:45284) 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 -- 172.21.15.78:0/2106705062 learned_addr learned my addr 172.21.15.78:0/2106705062 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f259c640 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa80af60 0x5635fabfcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabe8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 -- 172.21.15.78:0/2106705062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa80af60 msgr2=0x5635fabfcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa80af60 0x5635fabfcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 -- 172.21.15.78:0/2106705062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 msgr2=0x5635fabe8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabe8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 -- 172.21.15.78:0/2106705062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fabf4bb0 con 0x5635fa82b6b0 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f259c640 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa80af60 0x5635fabfcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fabe8b50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005c80 tx=0x7f23ec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7ecc05490ede88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23dbfff640 1 -- 172.21.15.78:0/2106705062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec012ce0 con 0x5635fa82b6b0 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23dbfff640 1 -- 172.21.15.78:0/2106705062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ec012e80 con 0x5635fa82b6b0 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.682+0000 7f23dbfff640 1 -- 172.21.15.78:0/2106705062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0053a0 con 0x5635fa82b6b0 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 -- 172.21.15.78:0/2106705062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 msgr2=0x5635fabe8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fabe8b50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005c80 tx=0x7f23ec01f000 comp rx=0 tx=0).stop 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 -- 172.21.15.78:0/2106705062 shutdown_connections 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fabe8b50 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa8234d0 0x5635fabe8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 --2- 172.21.15.78:0/2106705062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa80af60 0x5635fabfcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 -- 172.21.15.78:0/2106705062 >> 172.21.15.78:0/2106705062 conn(0x5635fa8404e0 msgr2=0x5635fabfc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 -- 172.21.15.78:0/2106705062 shutdown_connections 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 -- 172.21.15.78:0/2106705062 wait complete. 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 -- start start 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.683+0000 7f23ff997740 1 Processor -- start 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa80af60 0x5635fac00f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa8234d0 0x5635fac01450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fac022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a7720 con 0x5635fa82b6b0 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a8b90 con 0x5635fa80af60 2024-04-25T14:07:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41a7df0 con 0x5635fa8234d0 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 10 client.0 _ll_register_callbacks cb 0x7f23e4005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fac022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fac022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45300/0 (socket says 172.21.15.78:45300) 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 -- 172.21.15.78:0/3146560373 learned_addr learned my addr 172.21.15.78:0/3146560373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f1d9b640 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa8234d0 0x5635fac01450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 -- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa8234d0 msgr2=0x5635fac01450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa8234d0 0x5635fac01450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 -- 172.21.15.78:0/3146560373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa80af60 msgr2=0x5635fac00f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa80af60 0x5635fac00f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 -- 172.21.15.78:0/3146560373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fabf8120 con 0x5635fa82b6b0 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23f2d9d640 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fac022c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f23ec006010 tx=0x7f23ec004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d989e76a760ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23d9ffb640 1 -- 172.21.15.78:0/3146560373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec01eb50 con 0x5635fa82b6b0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23d9ffb640 1 -- 172.21.15.78:0/3146560373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f23ec031070 con 0x5635fa82b6b0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5635fab5a800 con 0x5635fa82b6b0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23d9ffb640 1 -- 172.21.15.78:0/3146560373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f23ec01ecf0 con 0x5635fa82b6b0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 10 client.17982 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.684+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5635fac728b0 con 0x5635fa82b6b0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23d9ffb640 1 -- 172.21.15.78:0/3146560373 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5635fab5a800 con 0x5635fa82b6b0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23d9ffb640 1 -- 172.21.15.78:0/3146560373 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f23ec01f070 con 0x5635fa82b6b0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23ff997740 10 client.17982 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23ff997740 10 client.17982 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23ff997740 20 client.17982 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23ff997740 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa84c390 0x5635fac75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23ff997740 4 client.17982 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635faad5610 con 0x5635fa84c390 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.685+0000 7f23f259c640 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa84c390 0x5635fac75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23f259c640 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa84c390 0x5635fac75130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46a92efdfff4de80 server_cookie=834ce0cf4b453b0f in_seq=0 out_seq=0 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23d9ffb640 10 client.17982 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23d9ffb640 1 -- 172.21.15.78:0/3146560373 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635faad5610 con 0x5635fa84c390 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23d9ffb640 10 client.17982 handle_command_reply: tid=0 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 client.17982 shutdown 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 msgr2=0x5635fac022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fac022c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f23ec006010 tx=0x7f23ec004e20 comp rx=0 tx=0).stop 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 shutdown_connections 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5635fa84c390 0x5635fac75130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5635fa82b6b0 0x5635fac022c0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23d9ffb640 0 client.17982 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635fa8234d0 0x5635fac01450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 --2- 172.21.15.78:0/3146560373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635fa80af60 0x5635fac00f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 >> 172.21.15.78:0/3146560373 conn(0x5635fa8404e0 msgr2=0x5635fabece20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 shutdown_connections 2024-04-25T14:07:55.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 1 -- 172.21.15.78:0/3146560373 wait complete. 2024-04-25T14:07:55.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.686+0000 7f23ff997740 20 client.17982 trim_cache size 0 max 16384 2024-04-25T14:07:55.693 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23dc0afaf0 msgr2=0x7f23dc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23dc0afaf0 0x7f23dc0b2000 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005c20 tx=0x7f23ec009bd0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f41029e0 msgr2=0x7f23f406a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f41029e0 0x7f23f406a920 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f23e4005d00 tx=0x7f23e400dde0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 shutdown_connections 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23f4108920 0x7f23f41a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f4103c20 0x7f23f419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f23dc0afaf0 0x7f23dc0b2000 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 --2- 172.21.15.78:0/3736397426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f41029e0 0x7f23f406a920 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 >> 172.21.15.78:0/3736397426 conn(0x7f23f40fdbd0 msgr2=0x7f23f4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 shutdown_connections 2024-04-25T14:07:55.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.687+0000 7f23f8ae7640 1 -- 172.21.15.78:0/3736397426 wait complete. 2024-04-25T14:07:55.697 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:07:55.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:55 smithi112 ceph-mon[32617]: pgmap v600: 97 pgs: 97 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-25T14:07:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 -- 172.21.15.78:0/1482439660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc102830 msgr2=0x7fb5bc103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/1482439660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc102830 0x7fb5bc103530 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009900 tx=0x7fb5ac02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 -- 172.21.15.78:0/1482439660 shutdown_connections 2024-04-25T14:07:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/1482439660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5bc108770 0x7fb5bc10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/1482439660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5bc103a70 0x7fb5bc103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/1482439660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc102830 0x7fb5bc103530 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 -- 172.21.15.78:0/1482439660 >> 172.21.15.78:0/1482439660 conn(0x7fb5bc0fda20 msgr2=0x7fb5bc0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 -- 172.21.15.78:0/1482439660 shutdown_connections 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.807+0000 7fb5bbfff640 1 -- 172.21.15.78:0/1482439660 wait complete. 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 Processor -- start 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 -- start start 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5bc102830 0x7fb5bc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc103a70 0x7fb5bc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5bc108770 0x7fb5bc1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10e330 con 0x7fb5bc103a70 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10e1b0 con 0x7fb5bc102830 2024-04-25T14:07:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bbfff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10e4b0 con 0x7fb5bc108770 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc103a70 0x7fb5bc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc103a70 0x7fb5bc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45326/0 (socket says 172.21.15.78:45326) 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5b3fff640 1 -- 172.21.15.78:0/897186182 learned_addr learned my addr 172.21.15.78:0/897186182 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5bc108770 0x7fb5bc1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5b3fff640 1 -- 172.21.15.78:0/897186182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5bc108770 msgr2=0x7fb5bc1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5bc108770 0x7fb5bc1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5b3fff640 1 -- 172.21.15.78:0/897186182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5bc102830 msgr2=0x7fb5bc19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5baffd640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5bc102830 0x7fb5bc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.808+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5bc102830 0x7fb5bc19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5b3fff640 1 -- 172.21.15.78:0/897186182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc1a1bc0 con 0x7fb5bc103a70 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5bc108770 0x7fb5bc1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc103a70 0x7fb5bc19a5c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d950 tx=0x7fb5a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5b8ff9640 1 -- 172.21.15.78:0/897186182 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4017030 con 0x7fb5bc103a70 2024-04-25T14:07:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5b8ff9640 1 -- 172.21.15.78:0/897186182 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a4004020 con 0x7fb5bc103a70 2024-04-25T14:07:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5b8ff9640 1 -- 172.21.15.78:0/897186182 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40047a0 con 0x7fb5bc103a70 2024-04-25T14:07:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc1a1eb0 con 0x7fb5bc103a70 2024-04-25T14:07:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.809+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1a2320 con 0x7fb5bc103a70 2024-04-25T14:07:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.811+0000 7fb5b8ff9640 1 -- 172.21.15.78:0/897186182 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb5a4009a30 con 0x7fb5bc103a70 2024-04-25T14:07:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.811+0000 7fb5c878c740 1 Processor -- start 2024-04-25T14:07:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.811+0000 7fb5c878c740 1 -- start start 2024-04-25T14:07:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5c878c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f151528d80 con 0x55f151222390 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5c878c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f1514e2a20 con 0x55f1511f94d0 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5c878c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f151524d40 con 0x55f1512016b0 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45334/0 (socket says 172.21.15.78:45334) 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3984895559 learned_addr learned my addr 172.21.15.78:0/3984895559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5b8ff9640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5a80afae0 0x7fb5a80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5a80afae0 0x7fb5a80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5b8ff9640 1 -- 172.21.15.78:0/897186182 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5a40de780 con 0x7fb5bc103a70 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3984895559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 msgr2=0x55f1515c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3984895559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 msgr2=0x55f1515c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3984895559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1514edc30 con 0x55f151222390 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.812+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.813+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515c49e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40139e0 tx=0x7fb5b4013ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5004ea1825374b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.813+0000 7fb5b17fa640 1 -- 172.21.15.78:0/3984895559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401fb80 con 0x55f151222390 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.813+0000 7fb5b17fa640 1 -- 172.21.15.78:0/3984895559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b401fd20 con 0x55f151222390 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.813+0000 7fb5b17fa640 1 -- 172.21.15.78:0/3984895559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40273c0 con 0x55f151222390 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- 172.21.15.78:0/3984895559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 msgr2=0x55f1515c49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5a80afae0 0x7fb5a80b1ff0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4022000 tx=0x7fb5b4020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515c49e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40139e0 tx=0x7fb5b4013ea0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- 172.21.15.78:0/3984895559 shutdown_connections 2024-04-25T14:07:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3984895559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515c49e0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- 172.21.15.78:0/3984895559 >> 172.21.15.78:0/3984895559 conn(0x55f1512164e0 msgr2=0x55f1515c01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- 172.21.15.78:0/3984895559 shutdown_connections 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- 172.21.15.78:0/3984895559 wait complete. 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- start start 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 Processor -- start 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1511f94d0 0x55f1515ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1512016b0 0x55f1515cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f1515c6700 con 0x55f151222390 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f1514afdc0 con 0x55f1512016b0 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f1515c6880 con 0x55f1511f94d0 2024-04-25T14:07:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.814+0000 7fb5c878c740 10 client.0 _ll_register_callbacks cb 0x55f1514cb600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45348/0 (socket says 172.21.15.78:45348) 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3087377567 learned_addr learned my addr 172.21.15.78:0/3087377567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1511f94d0 0x55f1515ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1512016b0 0x55f1515cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1511f94d0 msgr2=0x55f1515ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1511f94d0 0x55f1515ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3087377567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1512016b0 msgr2=0x55f1515cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1512016b0 0x55f1515cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/3087377567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15123a3f0 con 0x55f151222390 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1512016b0 0x55f1515cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515d5bd0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4005fd0 tx=0x7fb5b4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195264857b41bab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3087377567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401f980 con 0x55f151222390 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3087377567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5b401fb20 con 0x55f151222390 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b401fd20 con 0x55f151222390 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3087377567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb5b4003970 con 0x55f151222390 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5c878c740 10 client.17994 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f1513047d0 con 0x55f151222390 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.815+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3087377567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b401fd20 con 0x55f151222390 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3087377567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb5b4027660 con 0x55f151222390 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5c878c740 10 client.17994 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5c878c740 10 client.17994 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5c878c740 20 client.17994 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511e0f60 0x55f1515dc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5c878c740 4 client.17994 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f151536990 con 0x55f1511e0f60 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.816+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511e0f60 0x55f1515dc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.817+0000 7fb5a27fc640 10 client.17994 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.817+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511e0f60 0x55f1515dc0f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f505b73ec07002ae server_cookie=f659af651ab13106 in_seq=0 out_seq=0 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.817+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3087377567 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f151536990 con 0x55f1511e0f60 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.817+0000 7fb5a27fc640 10 client.17994 handle_command_reply: tid=0 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.817+0000 7fb5c878c740 1 client.17994 shutdown 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 msgr2=0x55f1515d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515d5bd0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4005fd0 tx=0x7fb5b4009240 comp rx=0 tx=0).stop 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 shutdown_connections 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515d5bd0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1512016b0 0x55f1515cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511e0f60 0x55f1515dc0f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3087377567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1511f94d0 0x55f1515ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5a27fc640 0 client.17994 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:55.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 >> 172.21.15.78:0/3087377567 conn(0x55f1512164e0 msgr2=0x55f1515bf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 shutdown_connections 2024-04-25T14:07:55.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 1 -- 172.21.15.78:0/3087377567 wait complete. 2024-04-25T14:07:55.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.818+0000 7fb5c878c740 20 client.17994 trim_cache size 0 max 16384 2024-04-25T14:07:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.831+0000 7fb5c878c740 1 Processor -- start 2024-04-25T14:07:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.831+0000 7fb5c878c740 1 -- start start 2024-04-25T14:07:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5c878c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1adad0 con 0x55f151222390 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5c878c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1ab3a0 con 0x55f1511f94d0 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5c878c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1a91f0 con 0x55f1512016b0 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515e08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45352/0 (socket says 172.21.15.78:45352) 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/4116023831 learned_addr learned my addr 172.21.15.78:0/4116023831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5baffd640 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/4116023831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 msgr2=0x55f1515cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/4116023831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 msgr2=0x55f1515cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 -- 172.21.15.78:0/4116023831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1515b1ba0 con 0x55f151222390 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515e08a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000f80 tx=0x7fb5b4009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa805cc6f8c1f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5b0ff9640 1 -- 172.21.15.78:0/4116023831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4027d10 con 0x55f151222390 2024-04-25T14:07:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5b0ff9640 1 -- 172.21.15.78:0/4116023831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b4037070 con 0x55f151222390 2024-04-25T14:07:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.832+0000 7fb5b0ff9640 1 -- 172.21.15.78:0/4116023831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401fbe0 con 0x55f151222390 2024-04-25T14:07:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 -- 172.21.15.78:0/4116023831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 msgr2=0x55f1515e08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515e08a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000f80 tx=0x7fb5b4009cc0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 -- 172.21.15.78:0/4116023831 shutdown_connections 2024-04-25T14:07:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f151222390 0x55f1515e08a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1512016b0 0x55f1515cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 --2- 172.21.15.78:0/4116023831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 -- 172.21.15.78:0/4116023831 >> 172.21.15.78:0/4116023831 conn(0x55f1512164e0 msgr2=0x55f1515c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 -- 172.21.15.78:0/4116023831 shutdown_connections 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.833+0000 7fb5c878c740 1 -- 172.21.15.78:0/4116023831 wait complete. 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 -- start start 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 Processor -- start 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1512016b0 0x55f1515d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f151222390 0x55f1515d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1a9860 con 0x55f1512016b0 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1aacd0 con 0x55f1511f94d0 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1a9f30 con 0x55f151222390 2024-04-25T14:07:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5c878c740 10 client.0 _ll_register_callbacks cb 0x7fb5b40137b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1512016b0 0x55f1515d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1512016b0 0x55f1515d2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45362/0 (socket says 172.21.15.78:45362) 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 -- 172.21.15.78:0/3593881429 learned_addr learned my addr 172.21.15.78:0/3593881429 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f151222390 0x55f1515d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 -- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f151222390 msgr2=0x55f1515d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f151222390 0x55f1515d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 -- 172.21.15.78:0/3593881429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 msgr2=0x55f1515d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 -- 172.21.15.78:0/3593881429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1515d40f0 con 0x55f1512016b0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5bb7fe640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f151222390 0x55f1515d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5b3fff640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1512016b0 0x55f1515d2c00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40dc270 tx=0x7fb5a4007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e84ce0c67d5aeb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.834+0000 7fb5a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3593881429 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4018270 con 0x55f1512016b0 2024-04-25T14:07:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3593881429 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5a40c12e0 con 0x55f1512016b0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b4037070 con 0x55f1512016b0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3593881429 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb5a40a6c50 con 0x55f1512016b0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 10 client.18000 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f1515d5440 con 0x55f1512016b0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3593881429 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b4037070 con 0x55f1512016b0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3593881429 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb5a40a2ee0 con 0x55f1512016b0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 10 client.18000 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 10 client.18000 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 20 client.18000 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511b63a0 0x55f151646bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 4 client.18000 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f151530800 con 0x55f1511b63a0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.835+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511b63a0 0x55f151646bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5baffd640 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511b63a0 0x55f151646bb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c8e1ac1f8c8b68 server_cookie=e32783b3e1d4a786 in_seq=0 out_seq=0 2024-04-25T14:07:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5a27fc640 10 client.18000 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5a27fc640 1 -- 172.21.15.78:0/3593881429 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f151530800 con 0x55f1511b63a0 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5a27fc640 10 client.18000 handle_command_reply: tid=0 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 client.18000 shutdown 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1512016b0 msgr2=0x55f1515d2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1512016b0 0x55f1515d2c00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40dc270 tx=0x7fb5a4007590 comp rx=0 tx=0).stop 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 shutdown_connections 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f1511b63a0 0x55f151646bb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f151222390 0x55f1515d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1512016b0 0x55f1515d2c00 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5a27fc640 0 client.18000 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 --2- 172.21.15.78:0/3593881429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f1511f94d0 0x55f1515d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 >> 172.21.15.78:0/3593881429 conn(0x55f1512164e0 msgr2=0x55f1515c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 shutdown_connections 2024-04-25T14:07:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 1 -- 172.21.15.78:0/3593881429 wait complete. 2024-04-25T14:07:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.836+0000 7fb5c878c740 20 client.18000 trim_cache size 0 max 16384 2024-04-25T14:07:55.843 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5a80afae0 msgr2=0x7fb5a80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5a80afae0 0x7fb5a80b1ff0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4022000 tx=0x7fb5b4020000 comp rx=0 tx=0).stop 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc103a70 msgr2=0x7fb5bc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc103a70 0x7fb5bc19a5c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d950 tx=0x7fb5a400de10 comp rx=0 tx=0).stop 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 shutdown_connections 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5bc108770 0x7fb5bc1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5a80afae0 0x7fb5a80b1ff0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5bc103a70 0x7fb5bc19a5c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 --2- 172.21.15.78:0/897186182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5bc102830 0x7fb5bc19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 >> 172.21.15.78:0/897186182 conn(0x7fb5bc0fda20 msgr2=0x7fb5bc109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 shutdown_connections 2024-04-25T14:07:55.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.837+0000 7fb5bbfff640 1 -- 172.21.15.78:0/897186182 wait complete. 2024-04-25T14:07:55.846 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:55 smithi078 ceph-mon[29163]: pgmap v600: 97 pgs: 97 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-25T14:07:55.847 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:07:55.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:55 smithi129 ceph-mon[32569]: pgmap v600: 97 pgs: 97 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-25T14:07:55.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.956+0000 7fb5c9757640 1 -- 172.21.15.78:0/3240970050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 msgr2=0x7fb5c4109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.956+0000 7fb5c9757640 1 --2- 172.21.15.78:0/3240970050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c4109100 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009900 tx=0x7fb5b802a410 comp rx=0 tx=0).stop 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 -- 172.21.15.78:0/3240970050 shutdown_connections 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 --2- 172.21.15.78:0/3240970050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5c4109790 0x7fb5c410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 --2- 172.21.15.78:0/3240970050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c4109100 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 --2- 172.21.15.78:0/3240970050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c4101e40 0x7fb5c4102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 -- 172.21.15.78:0/3240970050 >> 172.21.15.78:0/3240970050 conn(0x7fb5c40fd350 msgr2=0x7fb5c40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 -- 172.21.15.78:0/3240970050 shutdown_connections 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 -- 172.21.15.78:0/3240970050 wait complete. 2024-04-25T14:07:55.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 Processor -- start 2024-04-25T14:07:55.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 -- start start 2024-04-25T14:07:55.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c4101e40 0x7fb5c419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.957+0000 7fb5c9757640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c9757640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5c4109790 0x7fb5c41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c9757640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410f620 con 0x7fb5c4106c20 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c9757640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410f4a0 con 0x7fb5c4109790 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c9757640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410f7a0 con 0x7fb5c4101e40 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45380/0 (socket says 172.21.15.78:45380) 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 -- 172.21.15.78:0/2885127768 learned_addr learned my addr 172.21.15.78:0/2885127768 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c4101e40 0x7fb5c419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5c4109790 0x7fb5c41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 -- 172.21.15.78:0/2885127768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c4101e40 msgr2=0x7fb5c419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c4101e40 0x7fb5c419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 -- 172.21.15.78:0/2885127768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5c4109790 msgr2=0x7fb5c41a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5c4109790 0x7fb5c41a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 -- 172.21.15.78:0/2885127768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c41a1ce0 con 0x7fb5c4106c20 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c419a790 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5abfff640 1 -- 172.21.15.78:0/2885127768 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b803f030 con 0x7fb5c4106c20 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5abfff640 1 -- 172.21.15.78:0/2885127768 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b8003cb0 con 0x7fb5c4106c20 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41a1f70 con 0x7fb5c4106c20 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5abfff640 1 -- 172.21.15.78:0/2885127768 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8035070 con 0x7fb5c4106c20 2024-04-25T14:07:55.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.958+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41a2380 con 0x7fb5c4106c20 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 Processor -- start 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 -- start start 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5abfff640 1 -- 172.21.15.78:0/2885127768 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb5b8042030 con 0x7fb5c4106c20 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bb8c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb8cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb8da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de9baf0d80 con 0x55de9b7c96b0 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de9baaaa20 con 0x55de9b7ea390 2024-04-25T14:07:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5d0609740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de9baecd40 con 0x55de9b7c14d0 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5c37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bb8c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb8da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5c37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bb8c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:60172/0 (socket says 172.21.15.78:60172) 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5c37fe640 1 -- 172.21.15.78:0/3134271172 learned_addr learned my addr 172.21.15.78:0/3134271172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb8da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45392/0 (socket says 172.21.15.78:45392) 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.961+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb8cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5c37fe640 1 -- 172.21.15.78:0/3134271172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 msgr2=0x55de9bb8cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb8cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5c37fe640 1 -- 172.21.15.78:0/3134271172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 msgr2=0x55de9bb8da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb8da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5c37fe640 1 -- 172.21.15.78:0/3134271172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9bab5c30 con 0x55de9b7ea390 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5abfff640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5ac0afaf0 0x7fb5ac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5ac0afaf0 0x7fb5ac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bb8c810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc007f40 tx=0x7fb5bc00f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d795e2900448253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5abfff640 1 -- 172.21.15.78:0/2885127768 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5b80ff210 con 0x7fb5c4106c20 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5a97fa640 1 -- 172.21.15.78:0/3134271172 <== mon.2 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc010900 con 0x55de9b7ea390 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5a97fa640 1 -- 172.21.15.78:0/3134271172 <== mon.2 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5bc0183f0 con 0x55de9b7ea390 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.962+0000 7fb5a97fa640 1 -- 172.21.15.78:0/3134271172 <== mon.2 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc017600 con 0x55de9b7ea390 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 -- 172.21.15.78:0/3134271172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 msgr2=0x55de9bb8c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bb8c810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc007f40 tx=0x7fb5bc00f850 comp rx=0 tx=0).stop 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5ac0afaf0 0x7fb5ac0b2000 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55de9bb8c530 tx=0x7fb5b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 -- 172.21.15.78:0/3134271172 shutdown_connections 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb8da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb8cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 --2- 172.21.15.78:0/3134271172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bb8c810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 -- 172.21.15.78:0/3134271172 >> 172.21.15.78:0/3134271172 conn(0x55de9b7de4e0 msgr2=0x55de9bb88020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 -- 172.21.15.78:0/3134271172 shutdown_connections 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.963+0000 7fb5d0609740 1 -- 172.21.15.78:0/3134271172 wait complete. 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 -- start start 2024-04-25T14:07:55.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 Processor -- start 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb96620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bb96b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7ea390 0x55de9bb9dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de9bb8e530 con 0x55de9b7ea390 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55de9ba77dc0 con 0x55de9b7c96b0 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55de9bb8e6b0 con 0x55de9b7c14d0 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5d0609740 10 client.0 _ll_register_callbacks cb 0x55de9ba93600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7ea390 0x55de9bb9dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7ea390 0x55de9bb9dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45394/0 (socket says 172.21.15.78:45394) 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 -- 172.21.15.78:0/401987248 learned_addr learned my addr 172.21.15.78:0/401987248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb96620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bb96b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 -- 172.21.15.78:0/401987248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 msgr2=0x55de9bb96620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb96620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 -- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 msgr2=0x55de9bb96b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bb96b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 -- 172.21.15.78:0/401987248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9b8023f0 con 0x55de9b7ea390 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.964+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7ea390 0x55de9bb9dac0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc01a040 tx=0x7fb5bc018f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18000a1ff531a785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/401987248 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc025020 con 0x55de9b7ea390 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/401987248 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5bc004c50 con 0x55de9b7ea390 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5bc0183f0 con 0x55de9b7ea390 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/401987248 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb5bc030050 con 0x55de9b7ea390 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 10 client.18012 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55de9b8cc7d0 con 0x55de9b7ea390 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/401987248 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5bc0183f0 con 0x55de9b7ea390 2024-04-25T14:07:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/401987248 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb5bc010aa0 con 0x55de9b7ea390 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 10 client.18012 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 20 client.18012 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b7a8f60 0x55de9bba4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 4 client.18012 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de9bafe990 con 0x55de9b7a8f60 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.965+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b7a8f60 0x55de9bba4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.966+0000 7fb5a6ff5640 10 client.18012 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.966+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b7a8f60 0x55de9bba4080 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fcbbf2633d1e82 server_cookie=5a7f99d8f49de727 in_seq=0 out_seq=0 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/401987248 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de9bafe990 con 0x55de9b7a8f60 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5a6ff5640 10 client.18012 handle_command_reply: tid=0 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 client.18012 shutdown 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7ea390 msgr2=0x55de9bb9dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7ea390 0x55de9bb9dac0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc01a040 tx=0x7fb5bc018f60 comp rx=0 tx=0).stop 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 shutdown_connections 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7ea390 0x55de9bb9dac0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bb96b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b7a8f60 0x55de9bba4080 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 --2- 172.21.15.78:0/401987248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb96620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5a6ff5640 0 client.18012 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 >> 172.21.15.78:0/401987248 conn(0x55de9b7de4e0 msgr2=0x55de9bb8af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 shutdown_connections 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 1 -- 172.21.15.78:0/401987248 wait complete. 2024-04-25T14:07:55.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.967+0000 7fb5d0609740 20 client.18012 trim_cache size 0 max 16384 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.980+0000 7fb5d0609740 1 Processor -- start 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.980+0000 7fb5d0609740 1 -- start start 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bba87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5d0609740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41adaa0 con 0x55de9b7c96b0 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5d0609740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41ab370 con 0x55de9b7ea390 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5d0609740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41a91c0 con 0x55de9b7c14d0 2024-04-25T14:07:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45396/0 (socket says 172.21.15.78:45396) 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 -- 172.21.15.78:0/1046429071 learned_addr learned my addr 172.21.15.78:0/1046429071 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 -- 172.21.15.78:0/1046429071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 msgr2=0x55de9bb919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb919e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 -- 172.21.15.78:0/1046429071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 msgr2=0x55de9bba87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bba87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bba87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 -- 172.21.15.78:0/1046429071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de9bba43f0 con 0x55de9b7c96b0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb953c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8032af0 tx=0x7fb5b8102cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a922a5df9eeb614 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5a8ff9640 1 -- 172.21.15.78:0/1046429071 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8049040 con 0x55de9b7c96b0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5a8ff9640 1 -- 172.21.15.78:0/1046429071 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b80c72a0 con 0x55de9b7c96b0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.981+0000 7fb5a8ff9640 1 -- 172.21.15.78:0/1046429071 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80c7790 con 0x55de9b7c96b0 2024-04-25T14:07:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 -- 172.21.15.78:0/1046429071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 msgr2=0x55de9bb953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb953c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8032af0 tx=0x7fb5b8102cf0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 -- 172.21.15.78:0/1046429071 shutdown_connections 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7ea390 0x55de9bba87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c96b0 0x55de9bb953c0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1046429071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7c14d0 0x55de9bb919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 -- 172.21.15.78:0/1046429071 >> 172.21.15.78:0/1046429071 conn(0x55de9b7de4e0 msgr2=0x55de9bb94cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 -- 172.21.15.78:0/1046429071 shutdown_connections 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 -- 172.21.15.78:0/1046429071 wait complete. 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.982+0000 7fb5d0609740 1 -- start start 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 1 Processor -- start 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c14d0 0x55de9bc06eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bc073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7ea390 0x55de9bc0e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41a9830 con 0x55de9b7c14d0 2024-04-25T14:07:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41aaca0 con 0x55de9b7c96b0 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c41a9f00 con 0x55de9b7ea390 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5d0609740 10 client.0 _ll_register_callbacks cb 0x7fb5b8102bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c14d0 0x55de9bc06eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c14d0 0x55de9bc06eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45398/0 (socket says 172.21.15.78:45398) 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 -- 172.21.15.78:0/1174150432 learned_addr learned my addr 172.21.15.78:0/1174150432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bc073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 -- 172.21.15.78:0/1174150432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7ea390 msgr2=0x55de9bc0e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c37fe640 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7ea390 0x55de9bc0e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7ea390 0x55de9bc0e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 -- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 msgr2=0x55de9bc073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bc073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 -- 172.21.15.78:0/1174150432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de9bc0e910 con 0x55de9b7c14d0 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5c2ffd640 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c14d0 0x55de9bc06eb0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4003c80 tx=0x7fb5b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861efaa11f372aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.983+0000 7fb5a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/1174150432 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401e9b0 con 0x55de9b7c14d0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/1174150432 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb5b401eb50 con 0x55de9b7c14d0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b80c72a0 con 0x55de9b7c14d0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/1174150432 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb5b401ed50 con 0x55de9b7c14d0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 10 client.18018 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55de9bc0fc60 con 0x55de9b7c14d0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/1174150432 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b80c72a0 con 0x55de9b7c14d0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/1174150432 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb5b403d6b0 con 0x55de9b7c14d0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 10 client.18018 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 20 client.18018 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b77e3a0 0x55de9bc12630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 4 client.18018 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de9baf8800 con 0x55de9b77e3a0 2024-04-25T14:07:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.984+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b77e3a0 0x55de9bc12630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5a6ff5640 10 client.18018 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5c27fc640 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b77e3a0 0x55de9bc12630 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38159c4bc53acecc server_cookie=bae99e8493a86cb7 in_seq=0 out_seq=0 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5a6ff5640 1 -- 172.21.15.78:0/1174150432 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de9baf8800 con 0x55de9b77e3a0 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5a6ff5640 10 client.18018 handle_command_reply: tid=0 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 client.18018 shutdown 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c14d0 msgr2=0x55de9bc06eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c14d0 0x55de9bc06eb0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4003c80 tx=0x7fb5b40046c0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 shutdown_connections 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55de9b77e3a0 0x55de9bc12630 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55de9b7ea390 0x55de9bc0e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55de9b7c96b0 0x55de9bc073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5a6ff5640 0 client.18018 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 --2- 172.21.15.78:0/1174150432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de9b7c14d0 0x55de9bc06eb0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 >> 172.21.15.78:0/1174150432 conn(0x55de9b7de4e0 msgr2=0x55de9bb8ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 shutdown_connections 2024-04-25T14:07:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 1 -- 172.21.15.78:0/1174150432 wait complete. 2024-04-25T14:07:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.985+0000 7fb5d0609740 20 client.18018 trim_cache size 0 max 16384 2024-04-25T14:07:55.992 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5ac0afaf0 msgr2=0x7fb5ac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5ac0afaf0 0x7fb5ac0b2000 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55de9bb8c530 tx=0x7fb5b400b390 comp rx=0 tx=0).stop 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 msgr2=0x7fb5c419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c419a790 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8004340 comp rx=0 tx=0).stop 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 shutdown_connections 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5c4109790 0x7fb5c41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5ac0afaf0 0x7fb5ac0b2000 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5c4106c20 0x7fb5c419a790 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 --2- 172.21.15.78:0/2885127768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb5c4101e40 0x7fb5c419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 >> 172.21.15.78:0/2885127768 conn(0x7fb5c40fd350 msgr2=0x7fb5c410a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 shutdown_connections 2024-04-25T14:07:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:55.986+0000 7fb5c9757640 1 -- 172.21.15.78:0/2885127768 wait complete. 2024-04-25T14:07:56.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:56 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3181121344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:56.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:56 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3181121344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:56.997 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:07:56.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:07:56.998 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:07:57.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:56 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3181121344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:07:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 -- 172.21.15.78:0/2408398847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4108990 msgr2=0x7efcb410adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 --2- 172.21.15.78:0/2408398847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4108990 0x7efcb410adb0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7efcac009900 tx=0x7efcac02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 -- 172.21.15.78:0/2408398847 shutdown_connections 2024-04-25T14:07:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 --2- 172.21.15.78:0/2408398847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4108990 0x7efcb410adb0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 --2- 172.21.15.78:0/2408398847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb4103c90 0x7efcb41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 --2- 172.21.15.78:0/2408398847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb4102a50 0x7efcb4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 -- 172.21.15.78:0/2408398847 >> 172.21.15.78:0/2408398847 conn(0x7efcb40fdc40 msgr2=0x7efcb41000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 -- 172.21.15.78:0/2408398847 shutdown_connections 2024-04-25T14:07:57.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 -- 172.21.15.78:0/2408398847 wait complete. 2024-04-25T14:07:57.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.119+0000 7efcb993c640 1 Processor -- start 2024-04-25T14:07:57.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb993c640 1 -- start start 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb993c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4102a50 0x7efcb419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb993c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb4103c90 0x7efcb419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb993c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb4108990 0x7efcb41a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb993c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410f600 con 0x7efcb4102a50 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb993c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410f480 con 0x7efcb4103c90 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb993c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410f780 con 0x7efcb4108990 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4102a50 0x7efcb419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4102a50 0x7efcb419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40324/0 (socket says 172.21.15.78:40324) 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 -- 172.21.15.78:0/966202268 learned_addr learned my addr 172.21.15.78:0/966202268 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb37fe640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb4108990 0x7efcb41a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb27fc640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb4103c90 0x7efcb419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 -- 172.21.15.78:0/966202268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb4108990 msgr2=0x7efcb41a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb4108990 0x7efcb41a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 -- 172.21.15.78:0/966202268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb4103c90 msgr2=0x7efcb419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb4103c90 0x7efcb419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 -- 172.21.15.78:0/966202268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb41a1cc0 con 0x7efcb4102a50 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.120+0000 7efcb2ffd640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4102a50 0x7efcb419a1f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7efca400ea50 tx=0x7efca400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.121+0000 7efcb893a640 1 -- 172.21.15.78:0/966202268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca400cdc0 con 0x7efcb4102a50 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.121+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb41a1fb0 con 0x7efcb4102a50 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.121+0000 7efcb893a640 1 -- 172.21.15.78:0/966202268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca4011920 con 0x7efcb4102a50 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.121+0000 7efcb893a640 1 -- 172.21.15.78:0/966202268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca401aae0 con 0x7efcb4102a50 2024-04-25T14:07:57.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.121+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a2410 con 0x7efcb4102a50 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 Processor -- start 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 -- start start 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee63114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f47550 0x55cee6311a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f4f730 0x55cee6318920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cee6276b00 con 0x55cee5f70410 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cee6212920 con 0x55cee5f4f730 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcc07ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cee6272ac0 con 0x55cee5f47550 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee63114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee63114a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40336/0 (socket says 172.21.15.78:40336) 2024-04-25T14:07:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 -- 172.21.15.78:0/1001717414 learned_addr learned my addr 172.21.15.78:0/1001717414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb893a640 1 -- 172.21.15.78:0/966202268 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efca401ac80 con 0x7efcb4102a50 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb27fc640 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f4f730 0x55cee6318920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 -- 172.21.15.78:0/1001717414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f47550 msgr2=0x55cee6311a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f47550 0x55cee6311a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 -- 172.21.15.78:0/1001717414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f4f730 msgr2=0x55cee6318920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f4f730 0x55cee6318920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 -- 172.21.15.78:0/1001717414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee60507e0 con 0x55cee5f70410 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.124+0000 7efcb37fe640 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee63114a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7efcac009900 tx=0x7efcac032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7698616141e2cc94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.125+0000 7efc99ffb640 1 -- 172.21.15.78:0/1001717414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac03f070 con 0x55cee5f70410 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.125+0000 7efc99ffb640 1 -- 172.21.15.78:0/1001717414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac004040 con 0x55cee5f70410 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.125+0000 7efc99ffb640 1 -- 172.21.15.78:0/1001717414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac035070 con 0x55cee5f70410 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.125+0000 7efcb893a640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efc9c0afaf0 0x7efc9c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.125+0000 7efcb2ffd640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efc9c0afaf0 0x7efc9c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.125+0000 7efcb893a640 1 -- 172.21.15.78:0/966202268 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efca40df1a0 con 0x7efcb4102a50 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 -- 172.21.15.78:0/1001717414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 msgr2=0x55cee63114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee63114a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7efcac009900 tx=0x7efcac032ba0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcb2ffd640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efc9c0afaf0 0x7efc9c0b2000 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca40a7370 tx=0x7efca4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 -- 172.21.15.78:0/1001717414 shutdown_connections 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f4f730 0x55cee6318920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f47550 0x55cee6311a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1001717414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee63114a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 -- 172.21.15.78:0/1001717414 >> 172.21.15.78:0/1001717414 conn(0x55cee5f64560 msgr2=0x55cee630ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 -- 172.21.15.78:0/1001717414 shutdown_connections 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 -- 172.21.15.78:0/1001717414 wait complete. 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 -- start start 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.126+0000 7efcc07ea740 1 Processor -- start 2024-04-25T14:07:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee631b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 0x55cee6322730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cee6319310 con 0x55cee5f47550 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cee62105d0 con 0x55cee5f70410 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cee6319490 con 0x55cee5f4f730 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 10 client.0 _ll_register_callbacks cb 0x55cee61fe080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40342/0 (socket says 172.21.15.78:40342) 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 -- 172.21.15.78:0/2189930070 learned_addr learned my addr 172.21.15.78:0/2189930070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb27fc640 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee631b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 0x55cee6322730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 -- 172.21.15.78:0/2189930070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 msgr2=0x55cee631b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee631b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 -- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 msgr2=0x55cee6322730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 0x55cee6322730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 -- 172.21.15.78:0/2189930070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee60b0f70 con 0x55cee5f47550 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcb37fe640 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b290 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efcac005eb0 tx=0x7efcac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4aed41c5142969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efc92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efc92ffd640 1 -- 172.21.15.78:0/2189930070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac052060 con 0x55cee5f47550 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efc92ffd640 1 -- 172.21.15.78:0/2189930070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efcac0038b0 con 0x55cee5f47550 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcac004040 con 0x55cee5f47550 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.127+0000 7efc92ffd640 1 -- 172.21.15.78:0/2189930070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efcac048070 con 0x55cee5f47550 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 10 client.18030 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cee5f48170 con 0x55cee5f47550 2024-04-25T14:07:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efc92ffd640 1 -- 172.21.15.78:0/2189930070 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac004040 con 0x55cee5f47550 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efc92ffd640 1 -- 172.21.15.78:0/2189930070 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7efcac004d90 con 0x55cee5f47550 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 10 client.18030 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 20 client.18030 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f2f1f0 0x55cee63287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 4 client.18030 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cee6284710 con 0x55cee5f2f1f0 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.128+0000 7efcb27fc640 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f2f1f0 0x55cee63287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.129+0000 7efcb27fc640 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f2f1f0 0x55cee63287f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b03963104e6d74d server_cookie=375a0502109634a7 in_seq=0 out_seq=0 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.129+0000 7efc92ffd640 10 client.18030 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.129+0000 7efc92ffd640 1 -- 172.21.15.78:0/2189930070 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cee6284710 con 0x55cee5f2f1f0 2024-04-25T14:07:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.129+0000 7efc92ffd640 10 client.18030 handle_command_reply: tid=0 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.129+0000 7efcc07ea740 1 client.18030 shutdown 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 msgr2=0x55cee631b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b290 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efcac005eb0 tx=0x7efcac02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 shutdown_connections 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 0x55cee6322730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f2f1f0 0x55cee63287f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee631b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 --2- 172.21.15.78:0/2189930070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b290 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efc92ffd640 0 client.18030 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 >> 172.21.15.78:0/2189930070 conn(0x55cee5f64560 msgr2=0x55cee630bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 shutdown_connections 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 1 -- 172.21.15.78:0/2189930070 wait complete. 2024-04-25T14:07:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.130+0000 7efcc07ea740 20 client.18030 trim_cache size 0 max 16384 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 Processor -- start 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 -- start start 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 0x55cee630b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41adad0 con 0x55cee5f47550 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ab3a0 con 0x55cee5f70410 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcc07ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41a91f0 con 0x55cee5f4f730 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40358/0 (socket says 172.21.15.78:40358) 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 -- 172.21.15.78:0/1322272316 learned_addr learned my addr 172.21.15.78:0/1322272316 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb27fc640 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 -- 172.21.15.78:0/1322272316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 msgr2=0x55cee63226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 -- 172.21.15.78:0/1322272316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 msgr2=0x55cee630b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 0x55cee630b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb37fe640 1 -- 172.21.15.78:0/1322272316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee6307160 con 0x55cee5f47550 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.148+0000 7efcb27fc640 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efcb37fe640 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b570 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efcac005f70 tx=0x7efcac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497573ee56479ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efc997fa640 1 -- 172.21.15.78:0/1322272316 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac03f070 con 0x55cee5f47550 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efc997fa640 1 -- 172.21.15.78:0/1322272316 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac004040 con 0x55cee5f47550 2024-04-25T14:07:57.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efc997fa640 1 -- 172.21.15.78:0/1322272316 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac035040 con 0x55cee5f47550 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efcc07ea740 1 -- 172.21.15.78:0/1322272316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 msgr2=0x55cee631b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b570 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efcac005f70 tx=0x7efcac005fd0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efcc07ea740 1 -- 172.21.15.78:0/1322272316 shutdown_connections 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f70410 0x55cee630b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.149+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 --2- 172.21.15.78:0/1322272316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f47550 0x55cee631b570 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 -- 172.21.15.78:0/1322272316 >> 172.21.15.78:0/1322272316 conn(0x55cee5f64560 msgr2=0x55cee631ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 -- 172.21.15.78:0/1322272316 shutdown_connections 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 -- 172.21.15.78:0/1322272316 wait complete. 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 -- start start 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 Processor -- start 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f47550 0x55cee6320490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee6321900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41a9860 con 0x55cee5f70410 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41aacd0 con 0x55cee5f47550 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41a9f30 con 0x55cee5f4f730 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcc07ea740 10 client.0 _ll_register_callbacks cb 0x7efcac005500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee6321900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee6321900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40364/0 (socket says 172.21.15.78:40364) 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.150+0000 7efcb37fe640 1 -- 172.21.15.78:0/3380651382 learned_addr learned my addr 172.21.15.78:0/3380651382 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb37fe640 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f47550 0x55cee6320490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb27fc640 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb37fe640 1 -- 172.21.15.78:0/3380651382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 msgr2=0x55cee63209d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb37fe640 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63209d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb37fe640 1 -- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f47550 msgr2=0x55cee6320490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb37fe640 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f47550 0x55cee6320490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb37fe640 1 -- 172.21.15.78:0/3380651382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee6321ec0 con 0x55cee5f70410 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcb37fe640 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee6321900 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7efcac044550 tx=0x7efcac004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5f44a3bdaa2330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efc92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efc92ffd640 1 -- 172.21.15.78:0/3380651382 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac035070 con 0x55cee5f70410 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efc92ffd640 1 -- 172.21.15.78:0/3380651382 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7efcac0036b0 con 0x55cee5f70410 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcac004040 con 0x55cee5f70410 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efc92ffd640 1 -- 172.21.15.78:0/3380651382 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efcac03f040 con 0x55cee5f70410 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcc07ea740 10 client.18036 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cee6390ea0 con 0x55cee5f70410 2024-04-25T14:07:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.151+0000 7efc92ffd640 1 -- 172.21.15.78:0/3380651382 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac004040 con 0x55cee5f70410 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efc92ffd640 1 -- 172.21.15.78:0/3380651382 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7efcac05f020 con 0x55cee5f70410 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcc07ea740 10 client.18036 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcc07ea740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcc07ea740 20 client.18036 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcc07ea740 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f04630 0x55cee63937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcc07ea740 4 client.18036 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cee627e580 con 0x55cee5f04630 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcb27fc640 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f04630 0x55cee63937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efcb27fc640 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f04630 0x55cee63937d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778743fe44e6a5eb server_cookie=c5a434b9287543b2 in_seq=0 out_seq=0 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.152+0000 7efc92ffd640 10 client.18036 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.153+0000 7efc92ffd640 1 -- 172.21.15.78:0/3380651382 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cee627e580 con 0x55cee5f04630 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.153+0000 7efc92ffd640 10 client.18036 handle_command_reply: tid=0 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.153+0000 7efcc07ea740 1 client.18036 shutdown 2024-04-25T14:07:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.153+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 msgr2=0x55cee6321900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.153+0000 7efcc07ea740 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee6321900 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7efcac044550 tx=0x7efcac004610 comp rx=0 tx=0).stop 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 shutdown_connections 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cee5f70410 0x55cee6321900 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cee5f04630 0x55cee63937d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cee5f4f730 0x55cee63209d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 --2- 172.21.15.78:0/3380651382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cee5f47550 0x55cee6320490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 >> 172.21.15.78:0/3380651382 conn(0x55cee5f64560 msgr2=0x55cee631b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efc92ffd640 0 client.18036 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 shutdown_connections 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 1 -- 172.21.15.78:0/3380651382 wait complete. 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcc07ea740 20 client.18036 trim_cache size 0 max 16384 2024-04-25T14:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "91df76b5-020f-46cd-ac09-5c50081c2aaa", 2024-04-25T14:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:07:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efc9c0afaf0 msgr2=0x7efc9c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efc9c0afaf0 0x7efc9c0b2000 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efca40a7370 tx=0x7efca4000f40 comp rx=0 tx=0).stop 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4102a50 msgr2=0x7efcb419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4102a50 0x7efcb419a1f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7efca400ea50 tx=0x7efca400ef10 comp rx=0 tx=0).stop 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 shutdown_connections 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcb4108990 0x7efcb41a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcb4103c90 0x7efcb419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7efc9c0afaf0 0x7efc9c0b2000 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 --2- 172.21.15.78:0/966202268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcb4102a50 0x7efcb419a1f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 >> 172.21.15.78:0/966202268 conn(0x7efcb40fdc40 msgr2=0x7efcb4104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 shutdown_connections 2024-04-25T14:07:57.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.154+0000 7efcb993c640 1 -- 172.21.15.78:0/966202268 wait complete. 2024-04-25T14:07:57.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:07:57.165 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.275+0000 7f7efe36c640 1 -- 172.21.15.78:0/40990075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8101760 msgr2=0x7f7ef8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.275+0000 7f7efe36c640 1 --2- 172.21.15.78:0/40990075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8101760 0x7f7ef8101bd0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009900 tx=0x7f7eec02a410 comp rx=0 tx=0).stop 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 -- 172.21.15.78:0/40990075 shutdown_connections 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 --2- 172.21.15.78:0/40990075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef8102260 0x7f7ef810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 --2- 172.21.15.78:0/40990075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8101760 0x7f7ef8101bd0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 --2- 172.21.15.78:0/40990075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef81050b0 0x7f7ef81054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 -- 172.21.15.78:0/40990075 >> 172.21.15.78:0/40990075 conn(0x7f7ef80fd380 msgr2=0x7f7ef80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 -- 172.21.15.78:0/40990075 shutdown_connections 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 -- 172.21.15.78:0/40990075 wait complete. 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 Processor -- start 2024-04-25T14:07:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.276+0000 7f7efe36c640 1 -- start start 2024-04-25T14:07:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef8101760 0x7f7ef819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8102260 0x7f7ef819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef81050b0 0x7f7ef81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810f640 con 0x7f7ef8102260 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810f4c0 con 0x7f7ef81050b0 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810f7c0 con 0x7f7ef8101760 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8102260 0x7f7ef819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8102260 0x7f7ef819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40376/0 (socket says 172.21.15.78:40376) 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 -- 172.21.15.78:0/4029821443 learned_addr learned my addr 172.21.15.78:0/4029821443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 -- 172.21.15.78:0/4029821443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef8101760 msgr2=0x7f7ef819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef8101760 0x7f7ef819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 -- 172.21.15.78:0/4029821443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef81050b0 msgr2=0x7f7ef81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efc909640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef81050b0 0x7f7ef81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef81050b0 0x7f7ef81a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 -- 172.21.15.78:0/4029821443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef81a1db0 con 0x7f7ef8102260 2024-04-25T14:07:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8102260 0x7f7ef819a7b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7eec031440 tx=0x7f7eec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef57fa640 1 -- 172.21.15.78:0/4029821443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec03f030 con 0x7f7ef8102260 2024-04-25T14:07:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef57fa640 1 -- 172.21.15.78:0/4029821443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec003aa0 con 0x7f7ef8102260 2024-04-25T14:07:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7ef57fa640 1 -- 172.21.15.78:0/4029821443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec035070 con 0x7f7ef8102260 2024-04-25T14:07:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef81a2040 con 0x7f7ef8102260 2024-04-25T14:07:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.277+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a2530 con 0x7f7ef8102260 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 Processor -- start 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 -- start start 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9cc0390 0x564cea061a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea061fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef57fa640 1 -- 172.21.15.78:0/4029821443 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7eec042030 con 0x7f7ef8102260 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 0x564cea062c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ce9fc6d80 con 0x564ce9c974d0 2024-04-25T14:07:57.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564ce9f808b0 con 0x564ce9cc0390 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7f0521a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564ce9fc2d40 con 0x564ce9c9f6b0 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea061fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea061fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40392/0 (socket says 172.21.15.78:40392) 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2987046191 learned_addr learned my addr 172.21.15.78:0/2987046191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7efc909640 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9cc0390 0x564cea061a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2987046191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 msgr2=0x564cea062c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 0x564cea062c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2987046191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9cc0390 msgr2=0x564cea061a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9cc0390 0x564cea061a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.280+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2987046191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce9f51a40 con 0x564ce9c974d0 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.281+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea061fe0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009bc0 tx=0x7f7ee800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf2ea1c58081549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.281+0000 7f7ef57fa640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee00afaf0 0x7f7ee00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.281+0000 7f7ede7fc640 1 -- 172.21.15.78:0/2987046191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8005020 con 0x564ce9c974d0 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.281+0000 7f7efc909640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee00afaf0 0x7f7ee00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.281+0000 7f7ef57fa640 1 -- 172.21.15.78:0/4029821443 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7eec0ff450 con 0x7f7ef8102260 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.281+0000 7f7ede7fc640 1 -- 172.21.15.78:0/2987046191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee80047a0 con 0x564ce9c974d0 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.281+0000 7f7ede7fc640 1 -- 172.21.15.78:0/2987046191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8010690 con 0x564ce9c974d0 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 -- 172.21.15.78:0/2987046191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 msgr2=0x564cea061fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7efc909640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee00afaf0 0x7f7ee00b2000 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x564cea062ab0 tx=0x7f7ef00094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea061fe0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009bc0 tx=0x7f7ee800e720 comp rx=0 tx=0).stop 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 -- 172.21.15.78:0/2987046191 shutdown_connections 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 0x564cea062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea061fe0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2987046191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9cc0390 0x564cea061a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 -- 172.21.15.78:0/2987046191 >> 172.21.15.78:0/2987046191 conn(0x564ce9cb44e0 msgr2=0x564cea05d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 -- 172.21.15.78:0/2987046191 shutdown_connections 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 -- 172.21.15.78:0/2987046191 wait complete. 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 -- start start 2024-04-25T14:07:57.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.282+0000 7f7f0521a740 1 Processor -- start 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 0x564cea072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564cea063780 con 0x564ce9c974d0 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564ce9f4dd10 con 0x564ce9c9f6b0 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564cea063900 con 0x564ce9cc0390 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 10 client.0 _ll_register_callbacks cb 0x564ce9f7a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40402/0 (socket says 172.21.15.78:40402) 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2636942269 learned_addr learned my addr 172.21.15.78:0/2636942269 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2636942269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 msgr2=0x564cea072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 0x564cea072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 msgr2=0x564cea06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 -- 172.21.15.78:0/2636942269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce9da27d0 con 0x564ce9c974d0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea06b970 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800f010 tx=0x7f7ee8005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e67211f6ff9a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ed77fe640 1 -- 172.21.15.78:0/2636942269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8026020 con 0x564ce9c974d0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7ed77fe640 1 -- 172.21.15.78:0/2636942269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7ee8005220 con 0x564ce9c974d0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.283+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee80047a0 con 0x564ce9c974d0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7ed77fe640 1 -- 172.21.15.78:0/2636942269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7ee8031050 con 0x564ce9c974d0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 10 client.18048 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564ce9cd83f0 con 0x564ce9c974d0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7ed77fe640 1 -- 172.21.15.78:0/2636942269 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee80047a0 con 0x564ce9c974d0 2024-04-25T14:07:57.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7ed77fe640 1 -- 172.21.15.78:0/2636942269 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7ee8010430 con 0x564ce9c974d0 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 10 client.18048 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 10 client.18048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 20 client.18048 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c7ef60 0x564cea079000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 4 client.18048 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ce9fd4990 con 0x564ce9c7ef60 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.284+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c7ef60 0x564cea079000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.285+0000 7f7ed77fe640 10 client.18048 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.285+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c7ef60 0x564cea079000 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428e5dc597247675 server_cookie=64c920418da6870f in_seq=0 out_seq=0 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.285+0000 7f7ed77fe640 1 -- 172.21.15.78:0/2636942269 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ce9fd4990 con 0x564ce9c7ef60 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.285+0000 7f7ed77fe640 10 client.18048 handle_command_reply: tid=0 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.285+0000 7f7f0521a740 1 client.18048 shutdown 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 msgr2=0x564cea06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea06b970 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800f010 tx=0x7f7ee8005db0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 shutdown_connections 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c7ef60 0x564cea079000 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 0x564cea072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7ed77fe640 0 client.18048 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2636942269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea06b970 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 >> 172.21.15.78:0/2636942269 conn(0x564ce9cb44e0 msgr2=0x564cea060180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 shutdown_connections 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 1 -- 172.21.15.78:0/2636942269 wait complete. 2024-04-25T14:07:57.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.286+0000 7f7f0521a740 20 client.18048 trim_cache size 0 max 16384 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.303+0000 7f7f0521a740 1 Processor -- start 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.303+0000 7f7f0521a740 1 -- start start 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c974d0 0x564cea06a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 0x564cea072d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9cc0390 0x564cea073270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7f0521a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81adb50 con 0x564ce9cc0390 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7f0521a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ab420 con 0x564ce9c974d0 2024-04-25T14:07:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7f0521a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81a9270 con 0x564ce9c9f6b0 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9cc0390 0x564cea073270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9cc0390 0x564cea073270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40418/0 (socket says 172.21.15.78:40418) 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 -- 172.21.15.78:0/2622923122 learned_addr learned my addr 172.21.15.78:0/2622923122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c974d0 0x564cea06a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 -- 172.21.15.78:0/2622923122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 msgr2=0x564cea072d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 0x564cea072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 -- 172.21.15.78:0/2622923122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c974d0 msgr2=0x564cea06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c974d0 0x564cea06a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 -- 172.21.15.78:0/2622923122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce9c980f0 con 0x564ce9cc0390 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7efc909640 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9cc0390 0x564cea073270 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0012010 tx=0x7f7ef0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57afb2c441459ad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7eddffb640 1 -- 172.21.15.78:0/2622923122 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0022070 con 0x564ce9cc0390 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7eddffb640 1 -- 172.21.15.78:0/2622923122 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef001eb80 con 0x564ce9cc0390 2024-04-25T14:07:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.304+0000 7f7eddffb640 1 -- 172.21.15.78:0/2622923122 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000fd80 con 0x564ce9cc0390 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 -- 172.21.15.78:0/2622923122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9cc0390 msgr2=0x564cea073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9cc0390 0x564cea073270 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0012010 tx=0x7f7ef0014070 comp rx=0 tx=0).stop 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 -- 172.21.15.78:0/2622923122 shutdown_connections 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9cc0390 0x564cea073270 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9c9f6b0 0x564cea072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 --2- 172.21.15.78:0/2622923122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c974d0 0x564cea06a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 -- 172.21.15.78:0/2622923122 >> 172.21.15.78:0/2622923122 conn(0x564ce9cb44e0 msgr2=0x564cea06bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 -- 172.21.15.78:0/2622923122 shutdown_connections 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 -- 172.21.15.78:0/2622923122 wait complete. 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.305+0000 7f7f0521a740 1 -- start start 2024-04-25T14:07:57.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 1 Processor -- start 2024-04-25T14:07:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 0x564cea071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81a98e0 con 0x564ce9c974d0 2024-04-25T14:07:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81aad50 con 0x564ce9c9f6b0 2024-04-25T14:07:57.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81a9fb0 con 0x564ce9cc0390 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7f0521a740 10 client.0 _ll_register_callbacks cb 0x7f7ee8009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40428/0 (socket says 172.21.15.78:40428) 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 -- 172.21.15.78:0/912243493 learned_addr learned my addr 172.21.15.78:0/912243493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 -- 172.21.15.78:0/912243493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 msgr2=0x564cea071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 0x564cea071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 -- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 msgr2=0x564cea070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 -- 172.21.15.78:0/912243493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce9f874b0 con 0x564ce9c974d0 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ef7fff640 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea070780 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80108d0 tx=0x7f7ee8009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc2cb5ffc619e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.306+0000 7f7ed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:07:57.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7ed77fe640 1 -- 172.21.15.78:0/912243493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8026040 con 0x564ce9c974d0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef001eb80 con 0x564ce9c974d0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7ed77fe640 1 -- 172.21.15.78:0/912243493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7ee8010ac0 con 0x564ce9c974d0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7ed77fe640 1 -- 172.21.15.78:0/912243493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7ee801d040 con 0x564ce9c974d0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 10 client.18054 fetch_fsmap learned FSMap version 22 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564cea073270 con 0x564ce9c974d0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7ed77fe640 1 -- 172.21.15.78:0/912243493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef001eb80 con 0x564ce9c974d0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7ed77fe640 1 -- 172.21.15.78:0/912243493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7ee8023480 con 0x564ce9c974d0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 10 client.18054 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 10 client.18054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 20 client.18054 populate_metadata read hostname 'smithi078' 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c543a0 0x564cea0e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 4 client.18054 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ce9fce800 con 0x564ce9c543a0 2024-04-25T14:07:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.307+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c543a0 0x564cea0e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7ed77fe640 10 client.18054 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7ef77fe640 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c543a0 0x564cea0e3570 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d964fd8f8508652 server_cookie=a6e8fd2378e50f29 in_seq=0 out_seq=0 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7ed77fe640 1 -- 172.21.15.78:0/912243493 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564ce9fce800 con 0x564ce9c543a0 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7ed77fe640 10 client.18054 handle_command_reply: tid=0 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 client.18054 shutdown 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 msgr2=0x564cea070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea070780 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80108d0 tx=0x7f7ee8009670 comp rx=0 tx=0).stop 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 shutdown_connections 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564ce9c543a0 0x564cea0e3570 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564ce9cc0390 0x564cea071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7ed77fe640 0 client.18054 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ce9c9f6b0 0x564cea070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 --2- 172.21.15.78:0/912243493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce9c974d0 0x564cea070780 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 >> 172.21.15.78:0/912243493 conn(0x564ce9cb44e0 msgr2=0x564cea05b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 shutdown_connections 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 1 -- 172.21.15.78:0/912243493 wait complete. 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.308+0000 7f7f0521a740 20 client.18054 trim_cache size 0 max 16384 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stdout: "91df76b5-020f-46cd-ac09-5c50081c2aaa": { 2024-04-25T14:07:57.315 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:07:57.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "91df76b5-020f-46cd-ac09-5c50081c2aaa", 2024-04-25T14:07:57.316 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:07:57.316 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:07:57.316 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:07:57.316 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee00afaf0 msgr2=0x7f7ee00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee00afaf0 0x7f7ee00b2000 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x564cea062ab0 tx=0x7f7ef00094a0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8102260 msgr2=0x7f7ef819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8102260 0x7f7ef819a7b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7eec031440 tx=0x7f7eec004270 comp rx=0 tx=0).stop 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 shutdown_connections 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ef81050b0 0x7f7ef81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7ee00afaf0 0x7f7ee00b2000 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef8102260 0x7f7ef819a7b0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 --2- 172.21.15.78:0/4029821443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef8101760 0x7f7ef819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 >> 172.21.15.78:0/4029821443 conn(0x7f7ef80fd380 msgr2=0x7f7ef8105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 shutdown_connections 2024-04-25T14:07:57.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:07:57.309+0000 7f7efe36c640 1 -- 172.21.15.78:0/4029821443 wait complete. 2024-04-25T14:07:57.320 INFO:tasks.cephfs.filesystem:scrub status for tag:91df76b5-020f-46cd-ac09-5c50081c2aaa - {'path': '/', 'tag': '91df76b5-020f-46cd-ac09-5c50081c2aaa', 'options': 'recursive,force'} 2024-04-25T14:07:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:57 smithi078 ceph-mon[29163]: pgmap v601: 97 pgs: 97 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:57.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:57 smithi129 ceph-mon[32569]: pgmap v601: 97 pgs: 97 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:58.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:57 smithi112 ceph-mon[32617]: pgmap v601: 97 pgs: 97 active+clean; 1.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:07:58.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:58 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:07:58.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:58 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:58.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:58 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:07:58.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:58 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:58.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:58 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:07:58.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:58 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:58.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:58 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:07:58.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:58 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:59.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:58 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:07:59.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:58 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:59.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:58 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:07:59.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:58 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:07:59.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:07:59 smithi129 ceph-mon[32569]: pgmap v602: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:07:59.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:07:59 smithi078 ceph-mon[29163]: pgmap v602: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:00.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:07:59 smithi112 ceph-mon[32617]: pgmap v602: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:01.002 INFO:tasks.workunit.client.0.smithi078.stdout:375/600: open fd = 3 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:376/600: open fd = 3 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:377/600: open fd = 3 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.003 INFO:tasks.workunit.client.0.smithi078.stdout:378/600: open fd = 3 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:379/600: open fd = 3 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:380/600: open fd = 3 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:381/600: open fd = 3 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.004 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:382/600: open fd = 3 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:383/600: open fd = 3 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:384/600: open fd = 3 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.005 INFO:tasks.workunit.client.0.smithi078.stdout:385/600: open fd = 3 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:386/600: open fd = 3 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:387/600: open fd = 3 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.006 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:388/600: open fd = 3 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:389/600: open fd = 3 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:390/600: open fd = 3 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:391/600: open fd = 3 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.007 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:392/600: open fd = 3 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:393/600: open fd = 3 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:394/600: open fd = 3 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:395/600: open fd = 3 2024-04-25T14:08:01.008 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:396/600: open fd = 3 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:397/600: open fd = 3 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:398/600: open fd = 3 2024-04-25T14:08:01.009 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:399/600: open fd = 3 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:400/600: open fd = 3 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.010 INFO:tasks.workunit.client.0.smithi078.stdout:401/600: open fd = 3 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:402/600: open fd = 3 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:403/600: open fd = 3 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:404/600: open fd = 3 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.011 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:405/600: open fd = 3 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:406/600: open fd = 3 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:407/600: open fd = 3 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.012 INFO:tasks.workunit.client.0.smithi078.stdout:408/600: open fd = 3 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:409/600: open fd = 3 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:410/600: open fd = 3 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:411/600: open fd = 3 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.013 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:412/600: open fd = 3 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:413/600: open fd = 3 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:414/600: open fd = 3 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.014 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:415/600: open fd = 3 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:416/600: open fd = 3 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.015 INFO:tasks.workunit.client.0.smithi078.stdout:417/600: open fd = 3 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:418/600: open fd = 3 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:419/600: open fd = 3 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:420/600: open fd = 3 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.016 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:421/600: open fd = 3 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:422/600: open fd = 3 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:423/600: open fd = 3 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.017 INFO:tasks.workunit.client.0.smithi078.stdout:424/600: open fd = 3 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:425/600: open fd = 3 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:426/600: open fd = 3 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:427/600: open fd = 3 2024-04-25T14:08:01.018 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:428/600: open fd = 3 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:429/600: open fd = 3 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:430/600: open fd = 3 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.019 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:431/600: open fd = 3 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:432/600: open fd = 3 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:433/600: open fd = 3 2024-04-25T14:08:01.020 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.021 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.021 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.021 INFO:tasks.workunit.client.0.smithi078.stdout:434/600: open fd = 3 2024-04-25T14:08:01.021 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.021 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.021 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.021 INFO:tasks.workunit.client.0.smithi078.stdout:435/600: open fd = 3 2024-04-25T14:08:01.022 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.022 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.022 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.022 INFO:tasks.workunit.client.0.smithi078.stdout:436/600: open fd = 3 2024-04-25T14:08:01.022 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.023 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:08:01.023 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:08:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:01 smithi129 ceph-mon[32569]: pgmap v603: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:01.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:01 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:08:01.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:01 smithi078 ceph-mon[29163]: pgmap v603: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:01.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:01 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:08:02.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:01 smithi112 ceph-mon[32617]: pgmap v603: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:02.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:01 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 -- 172.21.15.78:0/398990952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0181098d0 msgr2=0x7fd01810bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/398990952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0181098d0 0x7fd01810bcf0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd010009ca0 tx=0x7fd01002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 -- 172.21.15.78:0/398990952 shutdown_connections 2024-04-25T14:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/398990952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0181098d0 0x7fd01810bcf0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/398990952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018106f70 0x7fd018109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/398990952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd018104610 0x7fd018106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 -- 172.21.15.78:0/398990952 >> 172.21.15.78:0/398990952 conn(0x7fd0180fdc10 msgr2=0x7fd0181000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 -- 172.21.15.78:0/398990952 shutdown_connections 2024-04-25T14:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 -- 172.21.15.78:0/398990952 wait complete. 2024-04-25T14:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 Processor -- start 2024-04-25T14:08:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.596+0000 7fd01e3c0640 1 -- start start 2024-04-25T14:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd01e3c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018104610 0x7fd01819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd01e3c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd018106f70 0x7fd01819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd01e3c0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181098d0 0x7fd0181a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd01e3c0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810f6a0 con 0x7fd018106f70 2024-04-25T14:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd01e3c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810f520 con 0x7fd018104610 2024-04-25T14:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd01e3c0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810f820 con 0x7fd0181098d0 2024-04-25T14:08:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd018106f70 0x7fd01819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd018106f70 0x7fd01819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40456/0 (socket says 172.21.15.78:40456) 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 -- 172.21.15.78:0/2851111809 learned_addr learned my addr 172.21.15.78:0/2851111809 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd01c95d640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181098d0 0x7fd0181a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd017fff640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018104610 0x7fd01819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 -- 172.21.15.78:0/2851111809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181098d0 msgr2=0x7fd0181a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181098d0 0x7fd0181a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 -- 172.21.15.78:0/2851111809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018104610 msgr2=0x7fd01819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018104610 0x7fd01819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.597+0000 7fd0177fe640 1 -- 172.21.15.78:0/2851111809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0181a1db0 con 0x7fd018106f70 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.598+0000 7fd0177fe640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd018106f70 0x7fd01819a7b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00d950 tx=0x7fd00c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.598+0000 7fd0157fa640 1 -- 172.21.15.78:0/2851111809 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c017070 con 0x7fd018106f70 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.598+0000 7fd0157fa640 1 -- 172.21.15.78:0/2851111809 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00c00de80 con 0x7fd018106f70 2024-04-25T14:08:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.598+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0181a20a0 con 0x7fd018106f70 2024-04-25T14:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.598+0000 7fd0157fa640 1 -- 172.21.15.78:0/2851111809 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c00f620 con 0x7fd018106f70 2024-04-25T14:08:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.598+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181a2670 con 0x7fd018106f70 2024-04-25T14:08:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 Processor -- start 2024-04-25T14:08:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 -- start start 2024-04-25T14:08:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd0157fa640 1 -- 172.21.15.78:0/2851111809 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd00c009c30 con 0x7fd018106f70 2024-04-25T14:08:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605c854e460 con 0x5605c821f2f0 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605c8508590 con 0x5605c82274d0 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd025271740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5605c854a420 con 0x5605c82481b0 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85e8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40472/0 (socket says 172.21.15.78:40472) 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd017fff640 1 -- 172.21.15.78:0/3333756803 learned_addr learned my addr 172.21.15.78:0/3333756803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd01c95d640 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd0177fe640 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd017fff640 1 -- 172.21.15.78:0/3333756803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 msgr2=0x5605c85e8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd017fff640 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd017fff640 1 -- 172.21.15.78:0/3333756803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 msgr2=0x5605c85efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.601+0000 7fd017fff640 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85efdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fd017fff640 1 -- 172.21.15.78:0/3333756803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c85150d0 con 0x5605c821f2f0 2024-04-25T14:08:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fd0157fa640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd0000afae0 0x7fd0000b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fd017fff640 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85e8ef0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd008009a40 tx=0x7fd008009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7342cea88358da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fd01c95d640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd0000afae0 0x7fd0000b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fcffe7fc640 1 -- 172.21.15.78:0/3333756803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008017070 con 0x5605c821f2f0 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fd0157fa640 1 -- 172.21.15.78:0/2851111809 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd00c0de800 con 0x7fd018106f70 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fcffe7fc640 1 -- 172.21.15.78:0/3333756803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd008013470 con 0x5605c821f2f0 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.602+0000 7fcffe7fc640 1 -- 172.21.15.78:0/3333756803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0080125d0 con 0x5605c821f2f0 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 -- 172.21.15.78:0/3333756803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 msgr2=0x5605c85e8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd01c95d640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd0000afae0 0x7fd0000b1ff0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5605c85e9b40 tx=0x7fd01002f3a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85e8ef0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd008009a40 tx=0x7fd008009f00 comp rx=0 tx=0).stop 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 -- 172.21.15.78:0/3333756803 shutdown_connections 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85e8ef0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 --2- 172.21.15.78:0/3333756803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 -- 172.21.15.78:0/3333756803 >> 172.21.15.78:0/3333756803 conn(0x5605c823c300 msgr2=0x5605c85e4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 -- 172.21.15.78:0/3333756803 shutdown_connections 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 -- 172.21.15.78:0/3333756803 wait complete. 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.603+0000 7fd025271740 1 -- start start 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 1 Processor -- start 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c821f2f0 0x5605c85f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82481b0 0x5605c85f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605c85f0840 con 0x5605c82481b0 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605c84d5b60 con 0x5605c82274d0 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5605c85f09c0 con 0x5605c821f2f0 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd025271740 10 client.0 _ll_register_callbacks cb 0x5605c84fa2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82481b0 0x5605c85f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82481b0 0x5605c85f9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40482/0 (socket says 172.21.15.78:40482) 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 -- 172.21.15.78:0/156338485 learned_addr learned my addr 172.21.15.78:0/156338485 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd0177fe640 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd017fff640 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c821f2f0 0x5605c85f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 -- 172.21.15.78:0/156338485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c821f2f0 msgr2=0x5605c85f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c821f2f0 0x5605c85f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 -- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 msgr2=0x5605c85f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 -- 172.21.15.78:0/156338485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c8260210 con 0x5605c82481b0 2024-04-25T14:08:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fd01c95d640 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82481b0 0x5605c85f9ab0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd010035000 tx=0x7fd01002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9a95044af3d186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.604+0000 7fcff77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fcff77fe640 1 -- 172.21.15.78:0/156338485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010004310 con 0x5605c82481b0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fcff77fe640 1 -- 172.21.15.78:0/156338485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd0100044b0 con 0x5605c82481b0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd008013470 con 0x5605c82481b0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fcff77fe640 1 -- 172.21.15.78:0/156338485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd0100046f0 con 0x5605c82481b0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 10 client.18066 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5605c832c620 con 0x5605c82481b0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fcff77fe640 1 -- 172.21.15.78:0/156338485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd008013470 con 0x5605c82481b0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fcff77fe640 1 -- 172.21.15.78:0/156338485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd01005c050 con 0x5605c82481b0 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 10 client.18066 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 20 client.18066 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c8206d80 0x5605c8600380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 4 client.18066 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605c855c070 con 0x5605c8206d80 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.605+0000 7fd017fff640 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c8206d80 0x5605c8600380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.606+0000 7fcff77fe640 10 client.18066 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.606+0000 7fd017fff640 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c8206d80 0x5605c8600380 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590fdf8d5dfd695b server_cookie=4aefe0d5696591ae in_seq=0 out_seq=0 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.606+0000 7fcff77fe640 1 -- 172.21.15.78:0/156338485 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5605c855c070 con 0x5605c8206d80 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.606+0000 7fcff77fe640 10 client.18066 handle_command_reply: tid=0 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 client.18066 shutdown 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82481b0 msgr2=0x5605c85f9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82481b0 0x5605c85f9ab0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd010035000 tx=0x7fd01002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 shutdown_connections 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82481b0 0x5605c85f9ab0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82274d0 0x5605c85f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c8206d80 0x5605c8600380 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 --2- 172.21.15.78:0/156338485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c821f2f0 0x5605c85f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fcff77fe640 0 client.18066 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 >> 172.21.15.78:0/156338485 conn(0x5605c823c300 msgr2=0x5605c85e7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 shutdown_connections 2024-04-25T14:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 1 -- 172.21.15.78:0/156338485 wait complete. 2024-04-25T14:08:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.607+0000 7fd025271740 20 client.18066 trim_cache size 0 max 16384 2024-04-25T14:08:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.622+0000 7fd025271740 1 Processor -- start 2024-04-25T14:08:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.622+0000 7fd025271740 1 -- start start 2024-04-25T14:08:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.622+0000 7fd025271740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c821f2f0 0x5605c85ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.622+0000 7fd025271740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82274d0 0x5605c85ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.622+0000 7fd025271740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.622+0000 7fd025271740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181adca0 con 0x5605c82274d0 2024-04-25T14:08:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd025271740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181ab570 con 0x5605c821f2f0 2024-04-25T14:08:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd025271740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181a93c0 con 0x5605c82481b0 2024-04-25T14:08:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82274d0 0x5605c85ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82274d0 0x5605c85ed010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40498/0 (socket says 172.21.15.78:40498) 2024-04-25T14:08:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 -- 172.21.15.78:0/3905623626 learned_addr learned my addr 172.21.15.78:0/3905623626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd017fff640 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c821f2f0 0x5605c85ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd01c95d640 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 -- 172.21.15.78:0/3905623626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 msgr2=0x5605c85e3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 -- 172.21.15.78:0/3905623626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c821f2f0 msgr2=0x5605c85ecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c821f2f0 0x5605c85ecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 -- 172.21.15.78:0/3905623626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c85e4e90 con 0x5605c82274d0 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fd0177fe640 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82274d0 0x5605c85ed010 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0a6c60 tx=0x7fd00c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb950d139eca2373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fcffdffb640 1 -- 172.21.15.78:0/3905623626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c08f050 con 0x5605c82274d0 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fcffdffb640 1 -- 172.21.15.78:0/3905623626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00c0ab080 con 0x5605c82274d0 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.623+0000 7fcffdffb640 1 -- 172.21.15.78:0/3905623626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c020040 con 0x5605c82274d0 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 -- 172.21.15.78:0/3905623626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82274d0 msgr2=0x5605c85ed010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82274d0 0x5605c85ed010 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0a6c60 tx=0x7fd00c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 -- 172.21.15.78:0/3905623626 shutdown_connections 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82481b0 0x5605c85e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c82274d0 0x5605c85ed010 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 --2- 172.21.15.78:0/3905623626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c821f2f0 0x5605c85ecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 -- 172.21.15.78:0/3905623626 >> 172.21.15.78:0/3905623626 conn(0x5605c823c300 msgr2=0x5605c85e88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 -- 172.21.15.78:0/3905623626 shutdown_connections 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 -- 172.21.15.78:0/3905623626 wait complete. 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.624+0000 7fd025271740 1 -- start start 2024-04-25T14:08:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 1 Processor -- start 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82274d0 0x5605c85f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82481b0 0x5605c85f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181a9a30 con 0x5605c821f2f0 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181aaea0 con 0x5605c82481b0 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181aa100 con 0x5605c82274d0 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd025271740 10 client.0 _ll_register_callbacks cb 0x7fd0100318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40512/0 (socket says 172.21.15.78:40512) 2024-04-25T14:08:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 -- 172.21.15.78:0/2210722717 learned_addr learned my addr 172.21.15.78:0/2210722717 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd0177fe640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82274d0 0x5605c85f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd01c95d640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82481b0 0x5605c85f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 -- 172.21.15.78:0/2210722717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82274d0 msgr2=0x5605c85f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82274d0 0x5605c85f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 -- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82481b0 msgr2=0x5605c85f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82481b0 0x5605c85f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 -- 172.21.15.78:0/2210722717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c85f87a0 con 0x5605c821f2f0 2024-04-25T14:08:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd01c95d640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82481b0 0x5605c85f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fd017fff640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85f6d70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd00800ed20 tx=0x7fd008006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba418f576384940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.625+0000 7fcff77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fcff77fe640 1 -- 172.21.15.78:0/2210722717 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00802c070 con 0x5605c821f2f0 2024-04-25T14:08:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fcff77fe640 1 -- 172.21.15.78:0/2210722717 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd008005dd0 con 0x5605c821f2f0 2024-04-25T14:08:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd00c0ab080 con 0x5605c821f2f0 2024-04-25T14:08:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fcff77fe640 1 -- 172.21.15.78:0/2210722717 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd008038050 con 0x5605c821f2f0 2024-04-25T14:08:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 10 client.18072 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5605c8668b20 con 0x5605c821f2f0 2024-04-25T14:08:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fcff77fe640 1 -- 172.21.15.78:0/2210722717 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00c0ab080 con 0x5605c821f2f0 2024-04-25T14:08:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fcff77fe640 1 -- 172.21.15.78:0/2210722717 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd0080062b0 con 0x5605c821f2f0 2024-04-25T14:08:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 10 client.18072 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 20 client.18072 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c81dc1c0 0x5605c866b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 4 client.18072 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:08:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605c8555ee0 con 0x5605c81dc1c0 2024-04-25T14:08:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.626+0000 7fd0177fe640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c81dc1c0 0x5605c866b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.627+0000 7fcff77fe640 10 client.18072 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.627+0000 7fd0177fe640 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c81dc1c0 0x5605c866b1e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2bf99f1172b86cb server_cookie=b312516d22f91561 in_seq=0 out_seq=0 2024-04-25T14:08:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fcff77fe640 1 -- 172.21.15.78:0/2210722717 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605c8555ee0 con 0x5605c81dc1c0 2024-04-25T14:08:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fcff77fe640 10 client.18072 handle_command_reply: tid=0 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 client.18072 shutdown 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 msgr2=0x5605c85f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85f6d70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd00800ed20 tx=0x7fd008006d60 comp rx=0 tx=0).stop 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 shutdown_connections 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605c82481b0 0x5605c85f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5605c81dc1c0 0x5605c866b1e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5605c82274d0 0x5605c85f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fcff77fe640 0 client.18072 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 --2- 172.21.15.78:0/2210722717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605c821f2f0 0x5605c85f6d70 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 >> 172.21.15.78:0/2210722717 conn(0x5605c823c300 msgr2=0x5605c85e88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 shutdown_connections 2024-04-25T14:08:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 1 -- 172.21.15.78:0/2210722717 wait complete. 2024-04-25T14:08:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd025271740 20 client.18072 trim_cache size 0 max 16384 2024-04-25T14:08:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:08:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:08:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:08:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd0000afae0 msgr2=0x7fd0000b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd0000afae0 0x7fd0000b1ff0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5605c85e9b40 tx=0x7fd01002f3a0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd018106f70 msgr2=0x7fd01819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd018106f70 0x7fd01819a7b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00d950 tx=0x7fd00c00dd50 comp rx=0 tx=0).stop 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 shutdown_connections 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0181098d0 0x7fd0181a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd0000afae0 0x7fd0000b1ff0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd018106f70 0x7fd01819a7b0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 --2- 172.21.15.78:0/2851111809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd018104610 0x7fd01819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 >> 172.21.15.78:0/2851111809 conn(0x7fd0180fdc10 msgr2=0x7fd018105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.642+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 shutdown_connections 2024-04-25T14:08:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:03.643+0000 7fd01e3c0640 1 -- 172.21.15.78:0/2851111809 wait complete. 2024-04-25T14:08:03.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:03 smithi129 ceph-mon[32569]: pgmap v604: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:03.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:03 smithi078 ceph-mon[29163]: pgmap v604: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:04.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:03 smithi112 ceph-mon[32617]: pgmap v604: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:05.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:05 smithi129 ceph-mon[32569]: pgmap v605: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:05.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:05 smithi078 ceph-mon[29163]: pgmap v605: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:06.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:05 smithi112 ceph-mon[32617]: pgmap v605: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:07.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:07 smithi129 ceph-mon[32569]: pgmap v606: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:08:07.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:07 smithi078 ceph-mon[29163]: pgmap v606: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:08:08.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:07 smithi112 ceph-mon[32617]: pgmap v606: 97 pgs: 97 active+clean; 2.0 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:08:09.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:09 smithi129 ceph-mon[32569]: pgmap v607: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:08:09.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:09 smithi078 ceph-mon[29163]: pgmap v607: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:08:10.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:09 smithi112 ceph-mon[32617]: pgmap v607: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:08:11.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:11 smithi129 ceph-mon[32569]: pgmap v608: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:11.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:11 smithi078 ceph-mon[29163]: pgmap v608: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:12.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:11 smithi112 ceph-mon[32617]: pgmap v608: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:13.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:13 smithi129 ceph-mon[32569]: pgmap v609: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:13.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:13 smithi078 ceph-mon[29163]: pgmap v609: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:14.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:13 smithi112 ceph-mon[32617]: pgmap v609: 97 pgs: 97 active+clean; 2.1 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:15.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:15 smithi129 ceph-mon[32569]: pgmap v610: 97 pgs: 97 active+clean; 2.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:15.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:15 smithi078 ceph-mon[29163]: pgmap v610: 97 pgs: 97 active+clean; 2.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:16.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:15 smithi112 ceph-mon[32617]: pgmap v610: 97 pgs: 97 active+clean; 2.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 -- 172.21.15.78:0/3714876110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac105150 msgr2=0x7fe1ac105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 --2- 172.21.15.78:0/3714876110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac105150 0x7fe1ac105560 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009900 tx=0x7fe1a802a410 comp rx=0 tx=0).stop 2024-04-25T14:08:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 -- 172.21.15.78:0/3714876110 shutdown_connections 2024-04-25T14:08:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 --2- 172.21.15.78:0/3714876110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1ac102300 0x7fe1ac10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 --2- 172.21.15.78:0/3714876110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101800 0x7fe1ac101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 --2- 172.21.15.78:0/3714876110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac105150 0x7fe1ac105560 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 -- 172.21.15.78:0/3714876110 >> 172.21.15.78:0/3714876110 conn(0x7fe1ac0fd400 msgr2=0x7fe1ac0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 -- 172.21.15.78:0/3714876110 shutdown_connections 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.747+0000 7fe1b4106640 1 -- 172.21.15.78:0/3714876110 wait complete. 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 Processor -- start 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 -- start start 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac101800 0x7fe1ac104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1ac102300 0x7fe1ac1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac105150 0x7fe1ac102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10f680 con 0x7fe1ac101800 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10f500 con 0x7fe1ac102300 2024-04-25T14:08:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b4106640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10f800 con 0x7fe1ac105150 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac101800 0x7fe1ac104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac101800 0x7fe1ac104290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50706/0 (socket says 172.21.15.78:50706) 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1502214545 learned_addr learned my addr 172.21.15.78:0/1502214545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac105150 0x7fe1ac102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1502214545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac105150 msgr2=0x7fe1ac102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac105150 0x7fe1ac102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1502214545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1ac102300 msgr2=0x7fe1ac1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1ac102300 0x7fe1ac1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.748+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1502214545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac103520 con 0x7fe1ac101800 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.749+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac101800 0x7fe1ac104290 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8031440 tx=0x7fe1a8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.749+0000 7fe1a2ffd640 1 -- 172.21.15.78:0/1502214545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a803f070 con 0x7fe1ac101800 2024-04-25T14:08:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.749+0000 7fe1a2ffd640 1 -- 172.21.15.78:0/1502214545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1a803b460 con 0x7fe1ac101800 2024-04-25T14:08:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.749+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac1037b0 con 0x7fe1ac101800 2024-04-25T14:08:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.749+0000 7fe1a2ffd640 1 -- 172.21.15.78:0/1502214545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8035030 con 0x7fe1ac101800 2024-04-25T14:08:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.749+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac1a4690 con 0x7fe1ac101800 2024-04-25T14:08:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.751+0000 7fe1bafb8740 1 Processor -- start 2024-04-25T14:08:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.751+0000 7fe1bafb8740 1 -- start start 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.751+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda55f6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda55fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bda555c460 con 0x55bda522d2f0 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bda5516590 con 0x55bda52561b0 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bda5558420 con 0x55bda52354d0 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1a2ffd640 1 -- 172.21.15.78:0/1502214545 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe1a8042070 con 0x7fe1ac101800 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b16a1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda55fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50712/0 (socket says 172.21.15.78:50712) 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b16a1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda55fddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:39160/0 (socket says 172.21.15.78:39160) 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1259911719 learned_addr learned my addr 172.21.15.78:0/1259911719 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b26a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda55f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1259911719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 msgr2=0x55bda55fddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1259911719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda55fddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1259911719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 msgr2=0x55bda55f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1259911719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda55f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1259911719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda55230d0 con 0x55bda522d2f0 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1a2ffd640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1980afae0 0x7fe1980b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1259911719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f6ef0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8105800 tx=0x7fe1a81058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3902d5d1ee3d027b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.752+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1980afae0 0x7fe1980b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe1a2ffd640 1 -- 172.21.15.78:0/1502214545 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe1a80ff5b0 con 0x7fe1ac101800 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe197fff640 1 -- 172.21.15.78:0/1259911719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80c3940 con 0x55bda522d2f0 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe197fff640 1 -- 172.21.15.78:0/1259911719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1a80c7590 con 0x55bda522d2f0 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe197fff640 1 -- 172.21.15.78:0/1259911719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80c71f0 con 0x55bda522d2f0 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1259911719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 msgr2=0x55bda55f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1259911719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f6ef0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8105800 tx=0x7fe1a81058f0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1980afae0 0x7fe1980b1ff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55bda55f7b40 tx=0x7fe1a400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.753+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1259911719 shutdown_connections 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1259911719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda55fddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1259911719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f6ef0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1259911719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda55f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1259911719 >> 172.21.15.78:0/1259911719 conn(0x55bda524a300 msgr2=0x55bda55f2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1259911719 shutdown_connections 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1259911719 wait complete. 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 -- start start 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 Processor -- start 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda522d2f0 0x55bda5600390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52354d0 0x55bda56008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.754+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda5607830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bda55fe600 con 0x55bda52354d0 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bda54e3b60 con 0x55bda52561b0 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bda55fe780 con 0x55bda522d2f0 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1bafb8740 10 client.0 _ll_register_callbacks cb 0x55bda55082b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52354d0 0x55bda56008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52354d0 0x55bda56008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50714/0 (socket says 172.21.15.78:50714) 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 -- 172.21.15.78:0/1801011291 learned_addr learned my addr 172.21.15.78:0/1801011291 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda5607830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda522d2f0 0x55bda5600390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 -- 172.21.15.78:0/1801011291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda522d2f0 msgr2=0x55bda5600390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda522d2f0 0x55bda5600390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 -- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 msgr2=0x55bda5607830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda5607830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 -- 172.21.15.78:0/1801011291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda526e210 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda5607830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1b16a1640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52354d0 0x55bda56008d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00b910 tx=0x7fe19c00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4519b5cca1778017 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1957fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1957fa640 1 -- 172.21.15.78:0/1801011291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c01a470 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1957fa640 1 -- 172.21.15.78:0/1801011291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe19c01a610 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1a80c7590 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1957fa640 1 -- 172.21.15.78:0/1801011291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe19c01a810 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1bafb8740 10 client.18084 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.755+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bda533a620 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1957fa640 1 -- 172.21.15.78:0/1801011291 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1a80c7590 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1957fa640 1 -- 172.21.15.78:0/1801011291 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe19c011ba0 con 0x55bda52354d0 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1bafb8740 10 client.18084 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1bafb8740 10 client.18084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1bafb8740 20 client.18084 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda5214d80 0x55bda560e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1bafb8740 4 client.18084 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bda556a070 con 0x55bda5214d80 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.756+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda5214d80 0x55bda560e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.757+0000 7fe1957fa640 10 client.18084 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.757+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda5214d80 0x55bda560e110 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff9f342a5bbfcd2 server_cookie=c64307b09c52e660 in_seq=0 out_seq=0 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.757+0000 7fe1957fa640 1 -- 172.21.15.78:0/1801011291 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bda556a070 con 0x55bda5214d80 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.757+0000 7fe1957fa640 10 client.18084 handle_command_reply: tid=0 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 client.18084 shutdown 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52354d0 msgr2=0x55bda56008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52354d0 0x55bda56008d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00b910 tx=0x7fe19c00bd10 comp rx=0 tx=0).stop 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 shutdown_connections 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52561b0 0x55bda5607830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52354d0 0x55bda56008d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda5214d80 0x55bda560e110 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1801011291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda522d2f0 0x55bda5600390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 >> 172.21.15.78:0/1801011291 conn(0x55bda524a300 msgr2=0x55bda55f5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1957fa640 0 client.18084 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 shutdown_connections 2024-04-25T14:08:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1801011291 wait complete. 2024-04-25T14:08:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.758+0000 7fe1bafb8740 20 client.18084 trim_cache size 0 max 16384 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 Processor -- start 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 -- start start 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52354d0 0x55bda5607750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52561b0 0x55bda55f2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1afe00 con 0x55bda522d2f0 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1ad6d0 con 0x55bda52354d0 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1ab520 con 0x55bda52561b0 2024-04-25T14:08:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50722/0 (socket says 172.21.15.78:50722) 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1124659716 learned_addr learned my addr 172.21.15.78:0/1124659716 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b16a1640 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52354d0 0x55bda5607750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1124659716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52561b0 msgr2=0x55bda55f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52561b0 0x55bda55f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1124659716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52354d0 msgr2=0x55bda5607750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52354d0 0x55bda5607750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 -- 172.21.15.78:0/1124659716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda5607c90 con 0x55bda522d2f0 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b16a1640 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52354d0 0x55bda5607750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.773+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f1c50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8105a20 tx=0x7fe1a80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9c92c21a5f657b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1977fe640 1 -- 172.21.15.78:0/1124659716 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8049040 con 0x55bda522d2f0 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1977fe640 1 -- 172.21.15.78:0/1124659716 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1a8105430 con 0x55bda522d2f0 2024-04-25T14:08:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1977fe640 1 -- 172.21.15.78:0/1124659716 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80c73d0 con 0x55bda522d2f0 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1124659716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 msgr2=0x55bda55f1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f1c50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8105a20 tx=0x7fe1a80045e0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1124659716 shutdown_connections 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52561b0 0x55bda55f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda52354d0 0x55bda5607750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/1124659716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda522d2f0 0x55bda55f1c50 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1124659716 >> 172.21.15.78:0/1124659716 conn(0x55bda524a300 msgr2=0x55bda55faac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1124659716 shutdown_connections 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.774+0000 7fe1bafb8740 1 -- 172.21.15.78:0/1124659716 wait complete. 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 -- start start 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 Processor -- start 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda522d2f0 0x55bda5604da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda56052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52561b0 0x55bda5606210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1abb90 con 0x55bda52561b0 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1ad000 con 0x55bda522d2f0 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1ac260 con 0x55bda52354d0 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1bafb8740 10 client.0 _ll_register_callbacks cb 0x7fe19c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1b26a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52561b0 0x55bda5606210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1b26a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52561b0 0x55bda5606210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50736/0 (socket says 172.21.15.78:50736) 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1b26a3640 1 -- 172.21.15.78:0/670764895 learned_addr learned my addr 172.21.15.78:0/670764895 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1b16a1640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda56052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda522d2f0 0x55bda5604da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1b26a3640 1 -- 172.21.15.78:0/670764895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 msgr2=0x55bda56052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.775+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda56052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1b26a3640 1 -- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda522d2f0 msgr2=0x55bda5604da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda522d2f0 0x55bda5604da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1b26a3640 1 -- 172.21.15.78:0/670764895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda56067d0 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda522d2f0 0x55bda5604da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1b26a3640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52561b0 0x55bda5606210 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fe1a401dcd0 tx=0x7fe1a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f265880205f6215a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1957fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1957fa640 1 -- 172.21.15.78:0/670764895 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a400ee80 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1957fa640 1 -- 172.21.15.78:0/670764895 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe1a4004b20 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bda5563ee0 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1957fa640 1 -- 172.21.15.78:0/670764895 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe1a4004d60 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 10 client.18090 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bda56764c0 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1957fa640 1 -- 172.21.15.78:0/670764895 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bda5563ee0 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1957fa640 1 -- 172.21.15.78:0/670764895 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe1a4032580 con 0x55bda52561b0 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 10 client.18090 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 10 client.18090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 20 client.18090 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda51ea1c0 0x55bda5678d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 4 client.18090 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:08:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.776+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bda54df5f0 con 0x55bda51ea1c0 2024-04-25T14:08:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.777+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda51ea1c0 0x55bda5678d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.777+0000 7fe1957fa640 10 client.18090 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.777+0000 7fe1b1ea2640 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda51ea1c0 0x55bda5678d70 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac0e161af191915 server_cookie=71eac6a711869e5e in_seq=0 out_seq=0 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.791+0000 7fe1957fa640 1 -- 172.21.15.78:0/670764895 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bda54df5f0 con 0x55bda51ea1c0 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.791+0000 7fe1957fa640 10 client.18090 handle_command_reply: tid=0 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.791+0000 7fe1bafb8740 1 client.18090 shutdown 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52561b0 msgr2=0x55bda5606210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52561b0 0x55bda5606210 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fe1a401dcd0 tx=0x7fe1a4004ab0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 shutdown_connections 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bda52561b0 0x55bda5606210 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55bda51ea1c0 0x55bda5678d70 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bda52354d0 0x55bda56052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 --2- 172.21.15.78:0/670764895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bda522d2f0 0x55bda5604da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1957fa640 0 client.18090 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 >> 172.21.15.78:0/670764895 conn(0x55bda524a300 msgr2=0x55bda55f5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 shutdown_connections 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 1 -- 172.21.15.78:0/670764895 wait complete. 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1bafb8740 20 client.18090 trim_cache size 0 max 16384 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:08:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:08:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:08:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:08:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1980afae0 msgr2=0x7fe1980b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1b4106640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1980afae0 0x7fe1980b1ff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55bda55f7b40 tx=0x7fe1a400b580 comp rx=0 tx=0).stop 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac101800 msgr2=0x7fe1ac104290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.792+0000 7fe1b4106640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac101800 0x7fe1ac104290 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8031440 tx=0x7fe1a8003d50 comp rx=0 tx=0).stop 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 shutdown_connections 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac105150 0x7fe1ac102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe1ac102300 0x7fe1ac1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1980afae0 0x7fe1980b1ff0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 --2- 172.21.15.78:0/1502214545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1ac101800 0x7fe1ac104290 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 >> 172.21.15.78:0/1502214545 conn(0x7fe1ac0fd400 msgr2=0x7fe1ac105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 shutdown_connections 2024-04-25T14:08:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:16.793+0000 7fe1b4106640 1 -- 172.21.15.78:0/1502214545 wait complete. 2024-04-25T14:08:17.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:17 smithi129 ceph-mon[32569]: pgmap v611: 97 pgs: 97 active+clean; 2.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:17.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:17 smithi078 ceph-mon[29163]: pgmap v611: 97 pgs: 97 active+clean; 2.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:18.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:17 smithi112 ceph-mon[32617]: pgmap v611: 97 pgs: 97 active+clean; 2.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:19.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:19 smithi129 ceph-mon[32569]: pgmap v612: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:08:19.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:19 smithi078 ceph-mon[29163]: pgmap v612: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:08:20.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:19 smithi112 ceph-mon[32617]: pgmap v612: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:08:21.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:21 smithi129 ceph-mon[32569]: pgmap v613: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:08:21.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:21 smithi078 ceph-mon[29163]: pgmap v613: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:08:22.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:21 smithi112 ceph-mon[32617]: pgmap v613: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:08:23.818 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:23 smithi112 ceph-mon[32617]: pgmap v614: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:23.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:23 smithi129 ceph-mon[32569]: pgmap v614: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:23.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:23 smithi078 ceph-mon[29163]: pgmap v614: 97 pgs: 97 active+clean; 2.2 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:25.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:25 smithi129 ceph-mon[32569]: pgmap v615: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:25 smithi078 ceph-mon[29163]: pgmap v615: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:26.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:25 smithi112 ceph-mon[32617]: pgmap v615: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:27.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:08:27.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:08:27.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 -- 172.21.15.78:0/2310502860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144101760 msgr2=0x7f3144101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 --2- 172.21.15.78:0/2310502860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144101760 0x7f3144101bd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f3134009900 tx=0x7f313402a410 comp rx=0 tx=0).stop 2024-04-25T14:08:27.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 -- 172.21.15.78:0/2310502860 shutdown_connections 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 --2- 172.21.15.78:0/2310502860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144102260 0x7f314410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 --2- 172.21.15.78:0/2310502860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144101760 0x7f3144101bd0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 --2- 172.21.15.78:0/2310502860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31441050b0 0x7f31441054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 -- 172.21.15.78:0/2310502860 >> 172.21.15.78:0/2310502860 conn(0x7f31440fd380 msgr2=0x7f31440ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 -- 172.21.15.78:0/2310502860 shutdown_connections 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.443+0000 7f314b3ee640 1 -- 172.21.15.78:0/2310502860 wait complete. 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 Processor -- start 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 -- start start 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144101760 0x7f314419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144102260 0x7f314419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31441050b0 0x7f31441a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f314410f640 con 0x7f31441050b0 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f314410f4c0 con 0x7f3144101760 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314b3ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f314410f7c0 con 0x7f3144102260 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314998b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31441050b0 0x7f31441a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314998b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31441050b0 0x7f31441a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34892/0 (socket says 172.21.15.78:34892) 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314998b640 1 -- 172.21.15.78:0/1494712591 learned_addr learned my addr 172.21.15.78:0/1494712591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f3148989640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144102260 0x7f314419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.444+0000 7f314918a640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144101760 0x7f314419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314998b640 1 -- 172.21.15.78:0/1494712591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144102260 msgr2=0x7f314419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314998b640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144102260 0x7f314419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314998b640 1 -- 172.21.15.78:0/1494712591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144101760 msgr2=0x7f314419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314998b640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144101760 0x7f314419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314998b640 1 -- 172.21.15.78:0/1494712591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31441a1db0 con 0x7f31441050b0 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314998b640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31441050b0 0x7f31441a1740 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f313c00eaa0 tx=0x7f313c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f313a7fc640 1 -- 172.21.15.78:0/1494712591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c00cdb0 con 0x7f31441050b0 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31441a20a0 con 0x7f31441050b0 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f313a7fc640 1 -- 172.21.15.78:0/1494712591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313c005020 con 0x7f31441050b0 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f313a7fc640 1 -- 172.21.15.78:0/1494712591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0116d0 con 0x7f31441050b0 2024-04-25T14:08:27.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.445+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31441a25f0 con 0x7f31441050b0 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 Processor -- start 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f313a7fc640 1 -- 172.21.15.78:0/1494712591 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f313c0054a0 con 0x7f31441050b0 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 -- start start 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d75ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3946b0 0x562a2d75c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a2d6bbd80 con 0x562a2d3b5390 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562a2d6758b0 con 0x562a2d3946b0 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3152376740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a2d6b7d40 con 0x562a2d38c4d0 2024-04-25T14:08:27.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d75ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d75ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34898/0 (socket says 172.21.15.78:34898) 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 -- 172.21.15.78:0/861662554 learned_addr learned my addr 172.21.15.78:0/861662554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f314918a640 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f313a7fc640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f31300afaf0 0x7f31300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 -- 172.21.15.78:0/861662554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 msgr2=0x562a2d75b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 -- 172.21.15.78:0/861662554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3946b0 msgr2=0x562a2d75c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3946b0 0x562a2d75c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.448+0000 7f3148989640 1 -- 172.21.15.78:0/861662554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2d646a40 con 0x562a2d3b5390 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.449+0000 7f314998b640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f31300afaf0 0x7f31300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.449+0000 7f3148989640 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d75ae70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f31340096c0 tx=0x7f3134005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b378e1cc3c7fafe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.449+0000 7f313a7fc640 1 -- 172.21.15.78:0/1494712591 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f313c0dece0 con 0x7f31441050b0 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.449+0000 7f312f7fe640 1 -- 172.21.15.78:0/861662554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134035070 con 0x562a2d3b5390 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.449+0000 7f312f7fe640 1 -- 172.21.15.78:0/861662554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3134041030 con 0x562a2d3b5390 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.449+0000 7f312f7fe640 1 -- 172.21.15.78:0/861662554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134003a90 con 0x562a2d3b5390 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 -- 172.21.15.78:0/861662554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 msgr2=0x562a2d75ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f314998b640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f31300afaf0 0x7f31300b2000 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f313c0a44d0 tx=0x7f313c075db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d75ae70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f31340096c0 tx=0x7f3134005e30 comp rx=0 tx=0).stop 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 -- 172.21.15.78:0/861662554 shutdown_connections 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3946b0 0x562a2d75c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 --2- 172.21.15.78:0/861662554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d75ae70 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 -- 172.21.15.78:0/861662554 >> 172.21.15.78:0/861662554 conn(0x562a2d3a94e0 msgr2=0x562a2d752270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 -- 172.21.15.78:0/861662554 shutdown_connections 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 -- 172.21.15.78:0/861662554 wait complete. 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 -- start start 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.450+0000 7f3152376740 1 Processor -- start 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3152376740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d38c4d0 0x562a2d764d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3152376740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d765240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3152376740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d76c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3152376740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a2d75cb30 con 0x562a2d38c4d0 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3152376740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562a2d642d10 con 0x562a2d3b5390 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3152376740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a2d75ccb0 con 0x562a2d3946b0 2024-04-25T14:08:27.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3152376740 10 client.0 _ll_register_callbacks cb 0x562a2d66f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d38c4d0 0x562a2d764d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d38c4d0 0x562a2d764d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34900/0 (socket says 172.21.15.78:34900) 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 -- 172.21.15.78:0/391941538 learned_addr learned my addr 172.21.15.78:0/391941538 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f314918a640 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d765240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d76c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 -- 172.21.15.78:0/391941538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 msgr2=0x562a2d765240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d765240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 -- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 msgr2=0x562a2d76c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d76c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 -- 172.21.15.78:0/391941538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a2d4977d0 con 0x562a2d38c4d0 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f3148989640 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d38c4d0 0x562a2d764d00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f313402ac00 tx=0x7f313402ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56393d5bb6f32fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f312cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:27.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.451+0000 7f312cff9640 1 -- 172.21.15.78:0/391941538 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134035070 con 0x562a2d38c4d0 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3134041030 con 0x562a2d38c4d0 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f312cff9640 1 -- 172.21.15.78:0/391941538 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3134003810 con 0x562a2d38c4d0 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f312cff9640 1 -- 172.21.15.78:0/391941538 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3134003a10 con 0x562a2d38c4d0 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 10 client.18102 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562a2d3cd3f0 con 0x562a2d38c4d0 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f312cff9640 1 -- 172.21.15.78:0/391941538 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3134041030 con 0x562a2d38c4d0 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f312cff9640 1 -- 172.21.15.78:0/391941538 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3134057ab0 con 0x562a2d38c4d0 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 10 client.18102 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 20 client.18102 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d373f60 0x562a2d772260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 4 client.18102 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:27.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a2d6c9990 con 0x562a2d373f60 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.452+0000 7f314918a640 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d373f60 0x562a2d772260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.453+0000 7f312cff9640 10 client.18102 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.453+0000 7f314918a640 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d373f60 0x562a2d772260 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8defc226b7aa2fc3 server_cookie=f6ba54222aedc8b1 in_seq=0 out_seq=0 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f312cff9640 1 -- 172.21.15.78:0/391941538 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a2d6c9990 con 0x562a2d373f60 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f312cff9640 10 client.18102 handle_command_reply: tid=0 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 client.18102 shutdown 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d38c4d0 msgr2=0x562a2d764d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d38c4d0 0x562a2d764d00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f313402ac00 tx=0x7f313402ab90 comp rx=0 tx=0).stop 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 shutdown_connections 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d76c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d373f60 0x562a2d772260 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d765240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f312cff9640 0 client.18102 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:27.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 --2- 172.21.15.78:0/391941538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d38c4d0 0x562a2d764d00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 >> 172.21.15.78:0/391941538 conn(0x562a2d3a94e0 msgr2=0x562a2d755180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 shutdown_connections 2024-04-25T14:08:27.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 1 -- 172.21.15.78:0/391941538 wait complete. 2024-04-25T14:08:27.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.454+0000 7f3152376740 20 client.18102 trim_cache size 0 max 16384 2024-04-25T14:08:27.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 Processor -- start 2024-04-25T14:08:27.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 -- start start 2024-04-25T14:08:27.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3946b0 0x562a2d752310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d763a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31441ab8e0 con 0x562a2d3946b0 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31441a91b0 con 0x562a2d3b5390 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3152376740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31441a7000 con 0x562a2d38c4d0 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3946b0 0x562a2d752310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3946b0 0x562a2d752310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34902/0 (socket says 172.21.15.78:34902) 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 -- 172.21.15.78:0/3271975559 learned_addr learned my addr 172.21.15.78:0/3271975559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3148989640 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3148989640 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d763a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 -- 172.21.15.78:0/3271975559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 msgr2=0x562a2d75fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 -- 172.21.15.78:0/3271975559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 msgr2=0x562a2d763a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d763a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 -- 172.21.15.78:0/3271975559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2d38d0f0 con 0x562a2d3946b0 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f3148989640 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.472+0000 7f314918a640 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3946b0 0x562a2d752310 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f314000bdb0 tx=0x7f314000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b7811dd01f081e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f312effd640 1 -- 172.21.15.78:0/3271975559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140021070 con 0x562a2d3946b0 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f312effd640 1 -- 172.21.15.78:0/3271975559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3140005020 con 0x562a2d3946b0 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f312effd640 1 -- 172.21.15.78:0/3271975559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140004b20 con 0x562a2d3946b0 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 -- 172.21.15.78:0/3271975559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3946b0 msgr2=0x562a2d752310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3946b0 0x562a2d752310 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f314000bdb0 tx=0x7f314000b390 comp rx=0 tx=0).stop 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 -- 172.21.15.78:0/3271975559 shutdown_connections 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d3b5390 0x562a2d763a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3946b0 0x562a2d752310 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 --2- 172.21.15.78:0/3271975559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d38c4d0 0x562a2d75fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 -- 172.21.15.78:0/3271975559 >> 172.21.15.78:0/3271975559 conn(0x562a2d3a94e0 msgr2=0x562a2d763310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 -- 172.21.15.78:0/3271975559 shutdown_connections 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.473+0000 7f3152376740 1 -- 172.21.15.78:0/3271975559 wait complete. 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 -- start start 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 Processor -- start 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d38c4d0 0x562a2d769810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d769d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d7d8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31441a7670 con 0x562a2d3b5390 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31441a8ae0 con 0x562a2d38c4d0 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31441a7d40 con 0x562a2d3946b0 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3152376740 10 client.0 _ll_register_callbacks cb 0x7f3134031d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3148989640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d7d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3148989640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d7d8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34914/0 (socket says 172.21.15.78:34914) 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3148989640 1 -- 172.21.15.78:0/2540970723 learned_addr learned my addr 172.21.15.78:0/2540970723 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f314918a640 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d769d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.474+0000 7f3148989640 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d38c4d0 0x562a2d769810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3148989640 1 -- 172.21.15.78:0/2540970723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 msgr2=0x562a2d769d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3148989640 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d769d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3148989640 1 -- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d38c4d0 msgr2=0x562a2d769810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3148989640 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d38c4d0 0x562a2d769810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3148989640 1 -- 172.21.15.78:0/2540970723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a2d67c4b0 con 0x562a2d3b5390 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3148989640 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d7d8460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f313402a930 tx=0x7f313402ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660f20e8be382da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f312cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f312cff9640 1 -- 172.21.15.78:0/2540970723 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134035040 con 0x562a2d3b5390 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f312cff9640 1 -- 172.21.15.78:0/2540970723 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3134003bb0 con 0x562a2d3b5390 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3140005020 con 0x562a2d3b5390 2024-04-25T14:08:27.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f312cff9640 1 -- 172.21.15.78:0/2540970723 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3134003db0 con 0x562a2d3b5390 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 10 client.18108 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562a2d7d98c0 con 0x562a2d3b5390 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f312cff9640 1 -- 172.21.15.78:0/2540970723 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3140005020 con 0x562a2d3b5390 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f312cff9640 1 -- 172.21.15.78:0/2540970723 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f31340576e0 con 0x562a2d3b5390 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 10 client.18108 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 10 client.18108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 20 client.18108 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d3493a0 0x562a2d7dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 4 client.18108 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.475+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562a2d6c3800 con 0x562a2d3493a0 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f314918a640 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d3493a0 0x562a2d7dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f312cff9640 10 client.18108 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:27.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f314918a640 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d3493a0 0x562a2d7dc2b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2eb3374b36fc2b server_cookie=22a89a1fcb9b5274 in_seq=0 out_seq=0 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f312cff9640 1 -- 172.21.15.78:0/2540970723 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562a2d6c3800 con 0x562a2d3493a0 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f312cff9640 10 client.18108 handle_command_reply: tid=0 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 client.18108 shutdown 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 msgr2=0x562a2d7d8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d7d8460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f313402a930 tx=0x7f313402ab90 comp rx=0 tx=0).stop 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 shutdown_connections 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a2d3b5390 0x562a2d7d8460 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x562a2d3493a0 0x562a2d7dc2b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2d3946b0 0x562a2d769d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f3152376740 1 --2- 172.21.15.78:0/2540970723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562a2d38c4d0 0x562a2d769810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.476+0000 7f312cff9640 0 client.18108 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 >> 172.21.15.78:0/2540970723 conn(0x562a2d3a94e0 msgr2=0x562a2d765060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 shutdown_connections 2024-04-25T14:08:27.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f3152376740 1 -- 172.21.15.78:0/2540970723 wait complete. 2024-04-25T14:08:27.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f3152376740 20 client.18108 trim_cache size 0 max 16384 2024-04-25T14:08:27.484 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:08:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:08:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:08:27.484 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f31300afaf0 msgr2=0x7f31300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f31300afaf0 0x7f31300b2000 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f313c0a44d0 tx=0x7f313c075db0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31441050b0 msgr2=0x7f31441a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31441050b0 0x7f31441a1740 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f313c00eaa0 tx=0x7f313c00ef60 comp rx=0 tx=0).stop 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 shutdown_connections 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f31300afaf0 0x7f31300b2000 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31441050b0 0x7f31441a1740 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3144102260 0x7f314419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 --2- 172.21.15.78:0/1494712591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3144101760 0x7f314419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 >> 172.21.15.78:0/1494712591 conn(0x7f31440fd380 msgr2=0x7f3144105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 shutdown_connections 2024-04-25T14:08:27.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.477+0000 7f314b3ee640 1 -- 172.21.15.78:0/1494712591 wait complete. 2024-04-25T14:08:27.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:08:27.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:27.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 -- 172.21.15.78:0/970646514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7018103b30 msgr2=0x7f7018104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 --2- 172.21.15.78:0/970646514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7018103b30 0x7f7018104000 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f700c009900 tx=0x7f700c02a410 comp rx=0 tx=0).stop 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 -- 172.21.15.78:0/970646514 shutdown_connections 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 --2- 172.21.15.78:0/970646514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7018104690 0x7f701810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 --2- 172.21.15.78:0/970646514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7018103b30 0x7f7018104000 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 --2- 172.21.15.78:0/970646514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7018102fd0 0x7f70181033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 -- 172.21.15.78:0/970646514 >> 172.21.15.78:0/970646514 conn(0x7f70180fd3b0 msgr2=0x7f70180ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 -- 172.21.15.78:0/970646514 shutdown_connections 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.597+0000 7f701e230640 1 -- 172.21.15.78:0/970646514 wait complete. 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 Processor -- start 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 -- start start 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7018102fd0 0x7f7018102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7018103b30 0x7f701819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701819f3b0 0x7f70181a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f701810e760 con 0x7f701819f3b0 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f701810e5e0 con 0x7f7018102fd0 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f701e230640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f701810e8e0 con 0x7f7018103b30 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f7017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701819f3b0 0x7f70181a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f7017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701819f3b0 0x7f70181a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34922/0 (socket says 172.21.15.78:34922) 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f7017fff640 1 -- 172.21.15.78:0/3038311196 learned_addr learned my addr 172.21.15.78:0/3038311196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f7016ffd640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7018103b30 0x7f701819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f70177fe640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7018102fd0 0x7f7018102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f7017fff640 1 -- 172.21.15.78:0/3038311196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7018103b30 msgr2=0x7f701819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.598+0000 7f7017fff640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7018103b30 0x7f701819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f7017fff640 1 -- 172.21.15.78:0/3038311196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7018102fd0 msgr2=0x7f7018102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f7017fff640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7018102fd0 0x7f7018102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f7017fff640 1 -- 172.21.15.78:0/3038311196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70181a1f40 con 0x7f701819f3b0 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f7017fff640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701819f3b0 0x7f70181a17d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f701000cce0 tx=0x7f701000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f7014ff9640 1 -- 172.21.15.78:0/3038311196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701000f450 con 0x7f701819f3b0 2024-04-25T14:08:27.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70181a2230 con 0x7f701819f3b0 2024-04-25T14:08:27.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f7014ff9640 1 -- 172.21.15.78:0/3038311196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701000faa0 con 0x7f701819f3b0 2024-04-25T14:08:27.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f7014ff9640 1 -- 172.21.15.78:0/3038311196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010018b70 con 0x7f701819f3b0 2024-04-25T14:08:27.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.599+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70181a2780 con 0x7f701819f3b0 2024-04-25T14:08:27.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.600+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701806b3c0 con 0x7f701819f3b0 2024-04-25T14:08:27.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.602+0000 7f7014ff9640 1 -- 172.21.15.78:0/3038311196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7010018d10 con 0x7f701819f3b0 2024-04-25T14:08:27.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.603+0000 7f7014ff9640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f70000afaf0 0x7f70000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.603+0000 7f70177fe640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f70000afaf0 0x7f70000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.603+0000 7f7014ff9640 1 -- 172.21.15.78:0/3038311196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f70100df130 con 0x7f701819f3b0 2024-04-25T14:08:27.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.603+0000 7f70177fe640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f70000afaf0 0x7f70000b2000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7008005fd0 tx=0x7f7008005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.607+0000 7f7014ff9640 1 -- 172.21.15.78:0/3038311196 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7010014030 con 0x7f701819f3b0 2024-04-25T14:08:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:27 smithi078 ceph-mon[29163]: pgmap v616: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:27.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.818+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70181073e0 con 0x7f701819f3b0 2024-04-25T14:08:27.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.819+0000 7f7014ff9640 1 -- 172.21.15.78:0/3038311196 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f70100a1e20 con 0x7f701819f3b0 2024-04-25T14:08:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:08:27.819 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:08:27.819 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:08:27.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f70000afaf0 msgr2=0x7f70000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f70000afaf0 0x7f70000b2000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7008005fd0 tx=0x7f7008005850 comp rx=0 tx=0).stop 2024-04-25T14:08:27.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701819f3b0 msgr2=0x7f70181a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701819f3b0 0x7f70181a17d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f701000cce0 tx=0x7f701000a620 comp rx=0 tx=0).stop 2024-04-25T14:08:27.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 shutdown_connections 2024-04-25T14:08:27.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f70000afaf0 0x7f70000b2000 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701819f3b0 0x7f70181a17d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7018103b30 0x7f701819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 --2- 172.21.15.78:0/3038311196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7018102fd0 0x7f7018102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 >> 172.21.15.78:0/3038311196 conn(0x7f70180fd3b0 msgr2=0x7f70180ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 shutdown_connections 2024-04-25T14:08:27.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.821+0000 7f701e230640 1 -- 172.21.15.78:0/3038311196 wait complete. 2024-04-25T14:08:27.833 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:08:27.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:27 smithi129 ceph-mon[32569]: pgmap v616: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 -- 172.21.15.78:0/3066474043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 msgr2=0x7f6b20105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/3066474043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b20105140 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c02a410 comp rx=0 tx=0).stop 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 -- 172.21.15.78:0/3066474043 shutdown_connections 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/3066474043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b20109a30 0x7f6b2010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/3066474043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b20105140 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/3066474043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b20103af0 0x7f6b201047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 -- 172.21.15.78:0/3066474043 >> 172.21.15.78:0/3066474043 conn(0x7f6b200fece0 msgr2=0x7f6b20101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 -- 172.21.15.78:0/3066474043 shutdown_connections 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 -- 172.21.15.78:0/3066474043 wait complete. 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 Processor -- start 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.942+0000 7f6b26d4d640 1 -- start start 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b26d4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b20103af0 0x7f6b2019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b26d4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b2019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b26d4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b20109a30 0x7f6b201a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b26d4d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010f5f0 con 0x7f6b20104d30 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b26d4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010f470 con 0x7f6b20103af0 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b26d4d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2010f770 con 0x7f6b20109a30 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b2019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b2019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34934/0 (socket says 172.21.15.78:34934) 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 -- 172.21.15.78:0/4288321283 learned_addr learned my addr 172.21.15.78:0/4288321283 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b20103af0 0x7f6b2019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b252ea640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b20109a30 0x7f6b201a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 -- 172.21.15.78:0/4288321283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b20109a30 msgr2=0x7f6b201a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b20109a30 0x7f6b201a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 -- 172.21.15.78:0/4288321283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b20103af0 msgr2=0x7f6b2019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b20103af0 0x7f6b2019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 -- 172.21.15.78:0/4288321283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b201a1cb0 con 0x7f6b20104d30 2024-04-25T14:08:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b2019a760 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c006010 tx=0x7f6b0c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1dffb640 1 -- 172.21.15.78:0/4288321283 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c03f030 con 0x7f6b20104d30 2024-04-25T14:08:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1dffb640 1 -- 172.21.15.78:0/4288321283 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0c003aa0 con 0x7f6b20104d30 2024-04-25T14:08:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b201a1f40 con 0x7f6b20104d30 2024-04-25T14:08:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.943+0000 7f6b1dffb640 1 -- 172.21.15.78:0/4288321283 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c035070 con 0x7f6b20104d30 2024-04-25T14:08:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.944+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b201a2350 con 0x7f6b20104d30 2024-04-25T14:08:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 Processor -- start 2024-04-25T14:08:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b1dffb640 1 -- 172.21.15.78:0/4288321283 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6b0c042030 con 0x7f6b20104d30 2024-04-25T14:08:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 -- start start 2024-04-25T14:08:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4cf0390 0x5598b5092810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5092d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b5093a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598b4ff6d80 con 0x5598b4cc74d0 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598b4fb0a20 con 0x5598b4cf0390 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.946+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5598b4ff2d40 con 0x5598b4ccf6b0 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5092d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5092d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34936/0 (socket says 172.21.15.78:34936) 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 -- 172.21.15.78:0/337768282 learned_addr learned my addr 172.21.15.78:0/337768282 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b252ea640 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4cf0390 0x5598b5092810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b5093a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 -- 172.21.15.78:0/337768282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 msgr2=0x5598b5093a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b5093a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 -- 172.21.15.78:0/337768282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4cf0390 msgr2=0x5598b5092810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4cf0390 0x5598b5092810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 -- 172.21.15.78:0/337768282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b4fbbc30 con 0x5598b4cc74d0 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b1dffb640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b100afaf0 0x7f6b100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5092d90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009c40 tx=0x7f6b14011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3097b78daf004f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b252ea640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b100afaf0 0x7f6b100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b1dffb640 1 -- 172.21.15.78:0/4288321283 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6b0c0ff210 con 0x7f6b20104d30 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b0affd640 1 -- 172.21.15.78:0/337768282 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14004230 con 0x5598b4cc74d0 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b0affd640 1 -- 172.21.15.78:0/337768282 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b140043d0 con 0x5598b4cc74d0 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.947+0000 7f6b0affd640 1 -- 172.21.15.78:0/337768282 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14019660 con 0x5598b4cc74d0 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 -- 172.21.15.78:0/337768282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 msgr2=0x5598b5092d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5092d90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009c40 tx=0x7f6b14011810 comp rx=0 tx=0).stop 2024-04-25T14:08:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b252ea640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b100afaf0 0x7f6b100b2000 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5598b5093860 tx=0x7f6b18007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 -- 172.21.15.78:0/337768282 shutdown_connections 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b5093a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5092d90 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/337768282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4cf0390 0x5598b5092810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 -- 172.21.15.78:0/337768282 >> 172.21.15.78:0/337768282 conn(0x5598b4ce44e0 msgr2=0x5598b508e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 -- 172.21.15.78:0/337768282 shutdown_connections 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.948+0000 7f6b2dc04740 1 -- 172.21.15.78:0/337768282 wait complete. 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 -- start start 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 Processor -- start 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4cc74d0 0x5598b509c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b509cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cf0390 0x5598b50a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598b5094530 con 0x5598b4cf0390 2024-04-25T14:08:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5598b4f7ddc0 con 0x5598b4ccf6b0 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5598b50946b0 con 0x5598b4cc74d0 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b2dc04740 10 client.0 _ll_register_callbacks cb 0x5598b4f99600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b252ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cf0390 0x5598b50a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b24ae9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4cc74d0 0x5598b509c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b252ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cf0390 0x5598b50a3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34948/0 (socket says 172.21.15.78:34948) 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b252ea640 1 -- 172.21.15.78:0/3800715842 learned_addr learned my addr 172.21.15.78:0/3800715842 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.949+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b509cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b252ea640 1 -- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4cc74d0 msgr2=0x5598b509c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b252ea640 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4cc74d0 0x5598b509c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b252ea640 1 -- 172.21.15.78:0/3800715842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 msgr2=0x5598b509cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b252ea640 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b509cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b252ea640 1 -- 172.21.15.78:0/3800715842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b4d083f0 con 0x5598b4cf0390 2024-04-25T14:08:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4cc74d0 0x5598b509c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b252ea640 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cf0390 0x5598b50a3a30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f6b180078f0 tx=0x7f6b18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0040c7db3933aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6afffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6afffff640 1 -- 172.21.15.78:0/3800715842 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1801cd30 con 0x5598b4cf0390 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6afffff640 1 -- 172.21.15.78:0/3800715842 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6b18028480 con 0x5598b4cf0390 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b140043d0 con 0x5598b4cf0390 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6afffff640 1 -- 172.21.15.78:0/3800715842 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6b18028680 con 0x5598b4cf0390 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b2dc04740 10 client.18126 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5598b4dd27d0 con 0x5598b4cf0390 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6afffff640 1 -- 172.21.15.78:0/3800715842 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b140043d0 con 0x5598b4cf0390 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.950+0000 7f6afffff640 1 -- 172.21.15.78:0/3800715842 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6b1801d7c0 con 0x5598b4cf0390 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b2dc04740 10 client.18126 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b2dc04740 10 client.18126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b2dc04740 20 client.18126 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4caef60 0x5598b50aa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b2dc04740 4 client.18126 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598b5004990 con 0x5598b4caef60 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4caef60 0x5598b50aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6afffff640 10 client.18126 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.951+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4caef60 0x5598b50aa340 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53c158f1a668a6d1 server_cookie=fce7923c9ac95230 in_seq=0 out_seq=0 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6afffff640 1 -- 172.21.15.78:0/3800715842 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598b5004990 con 0x5598b4caef60 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6afffff640 10 client.18126 handle_command_reply: tid=0 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 client.18126 shutdown 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cf0390 msgr2=0x5598b50a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cf0390 0x5598b50a3a30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f6b180078f0 tx=0x7f6b18004a40 comp rx=0 tx=0).stop 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 shutdown_connections 2024-04-25T14:08:27.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4caef60 0x5598b50aa340 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cf0390 0x5598b50a3a30 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6afffff640 0 client.18126 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b509cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3800715842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4cc74d0 0x5598b509c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 >> 172.21.15.78:0/3800715842 conn(0x5598b4ce44e0 msgr2=0x5598b5090f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 shutdown_connections 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.952+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3800715842 wait complete. 2024-04-25T14:08:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.953+0000 7f6b2dc04740 20 client.18126 trim_cache size 0 max 16384 2024-04-25T14:08:27.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 Processor -- start 2024-04-25T14:08:27.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 -- start start 2024-04-25T14:08:27.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4caef60 0x5598b5097310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5097850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b50a3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201adad0 con 0x5598b4cc74d0 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201ab3a0 con 0x5598b4ccf6b0 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a91f0 con 0x5598b4caef60 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5097850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5097850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34952/0 (socket says 172.21.15.78:34952) 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b1ffff640 1 -- 172.21.15.78:0/3720955505 learned_addr learned my addr 172.21.15.78:0/3720955505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b252ea640 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b50a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b1ffff640 1 -- 172.21.15.78:0/3720955505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4caef60 msgr2=0x5598b5097310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4caef60 0x5598b5097310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b1ffff640 1 -- 172.21.15.78:0/3720955505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 msgr2=0x5598b50a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.966+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b50a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.967+0000 7f6b1ffff640 1 -- 172.21.15.78:0/3720955505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b50947a0 con 0x5598b4cc74d0 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.967+0000 7f6b252ea640 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b50a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.967+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5097850 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0c7670 tx=0x7f6b0c03b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b79e8b57352d4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.967+0000 7f6b0a7fc640 1 -- 172.21.15.78:0/3720955505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c049040 con 0x5598b4cc74d0 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.967+0000 7f6b0a7fc640 1 -- 172.21.15.78:0/3720955505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0c03b460 con 0x5598b4cc74d0 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.967+0000 7f6b0a7fc640 1 -- 172.21.15.78:0/3720955505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0d0540 con 0x5598b4cc74d0 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3720955505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 msgr2=0x5598b5097850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5097850 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0c7670 tx=0x7f6b0c03b640 comp rx=0 tx=0).stop 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3720955505 shutdown_connections 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4ccf6b0 0x5598b50a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b5097850 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/3720955505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4caef60 0x5598b5097310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3720955505 >> 172.21.15.78:0/3720955505 conn(0x5598b4ce44e0 msgr2=0x5598b5092790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3720955505 shutdown_connections 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- 172.21.15.78:0/3720955505 wait complete. 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- start start 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 Processor -- start 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4caef60 0x5598b50a0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b50a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b51106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a9860 con 0x5598b4cc74d0 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201aacd0 con 0x5598b4caef60 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b201a9f30 con 0x5598b4ccf6b0 2024-04-25T14:08:27.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b2dc04740 10 client.0 _ll_register_callbacks cb 0x7f6b18007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.968+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b50a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b50a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34962/0 (socket says 172.21.15.78:34962) 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 -- 172.21.15.78:0/2017926641 learned_addr learned my addr 172.21.15.78:0/2017926641 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4caef60 0x5598b50a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 -- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 msgr2=0x5598b51106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b252ea640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b51106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b51106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 -- 172.21.15.78:0/2017926641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4caef60 msgr2=0x5598b50a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4caef60 0x5598b50a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 -- 172.21.15.78:0/2017926641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b50a3e90 con 0x5598b4cc74d0 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b252ea640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b51106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b1ffff640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b50a0ed0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c004010 tx=0x7f6b0c003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b35367ed8468456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6afffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:27.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6afffff640 1 -- 172.21.15.78:0/2017926641 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c049070 con 0x5598b4cc74d0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5598b4ffe800 con 0x5598b4cc74d0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6afffff640 1 -- 172.21.15.78:0/2017926641 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6b0c0b0050 con 0x5598b4cc74d0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6afffff640 1 -- 172.21.15.78:0/2017926641 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6b0c0c7de0 con 0x5598b4cc74d0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b2dc04740 10 client.18132 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5598b5111c30 con 0x5598b4cc74d0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.969+0000 7f6afffff640 1 -- 172.21.15.78:0/2017926641 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5598b4ffe800 con 0x5598b4cc74d0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6afffff640 1 -- 172.21.15.78:0/2017926641 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6b0c0d0140 con 0x5598b4cc74d0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b2dc04740 10 client.18132 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b2dc04740 10 client.18132 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b2dc04740 20 client.18132 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4cf0390 0x5598b5114600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b2dc04740 4 client.18132 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598b4f79610 con 0x5598b4cf0390 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4cf0390 0x5598b5114600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:27.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6b24ae9640 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4cf0390 0x5598b5114600 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ed8a0e4c4f61fad server_cookie=b331bea340f18ac3 in_seq=0 out_seq=0 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.970+0000 7f6afffff640 10 client.18132 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6afffff640 1 -- 172.21.15.78:0/2017926641 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598b4f79610 con 0x5598b4cf0390 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6afffff640 10 client.18132 handle_command_reply: tid=0 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 client.18132 shutdown 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 msgr2=0x5598b50a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b50a0ed0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c004010 tx=0x7f6b0c003e80 comp rx=0 tx=0).stop 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 shutdown_connections 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5598b4cf0390 0x5598b5114600 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b4ccf6b0 0x5598b51106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598b4cc74d0 0x5598b50a0ed0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6afffff640 0 client.18132 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 --2- 172.21.15.78:0/2017926641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5598b4caef60 0x5598b50a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 >> 172.21.15.78:0/2017926641 conn(0x5598b4ce44e0 msgr2=0x5598b5092790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 shutdown_connections 2024-04-25T14:08:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 1 -- 172.21.15.78:0/2017926641 wait complete. 2024-04-25T14:08:27.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.971+0000 7f6b2dc04740 20 client.18132 trim_cache size 0 max 16384 2024-04-25T14:08:27.978 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b100afaf0 msgr2=0x7f6b100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b100afaf0 0x7f6b100b2000 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x5598b5093860 tx=0x7f6b18007680 comp rx=0 tx=0).stop 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 msgr2=0x7f6b2019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b2019a760 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c006010 tx=0x7f6b0c031a50 comp rx=0 tx=0).stop 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 shutdown_connections 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b20109a30 0x7f6b201a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6b100afaf0 0x7f6b100b2000 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b20104d30 0x7f6b2019a760 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 --2- 172.21.15.78:0/4288321283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b20103af0 0x7f6b2019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 >> 172.21.15.78:0/4288321283 conn(0x7f6b200fece0 msgr2=0x7f6b20105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 shutdown_connections 2024-04-25T14:08:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:27.972+0000 7f6b26d4d640 1 -- 172.21.15.78:0/4288321283 wait complete. 2024-04-25T14:08:27.983 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:08:28.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:27 smithi112 ceph-mon[32617]: pgmap v616: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:28.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 -- 172.21.15.78:0/4259352380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 msgr2=0x7f10b0101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/4259352380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b0101bd0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f1098009900 tx=0x7f109802a410 comp rx=0 tx=0).stop 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 -- 172.21.15.78:0/4259352380 shutdown_connections 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/4259352380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10b0102260 0x7f10b010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/4259352380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b0101bd0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/4259352380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b01050b0 0x7f10b01054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 -- 172.21.15.78:0/4259352380 >> 172.21.15.78:0/4259352380 conn(0x7f10b00fd380 msgr2=0x7f10b00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 -- 172.21.15.78:0/4259352380 shutdown_connections 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.091+0000 7f10b5ebd640 1 -- 172.21.15.78:0/4259352380 wait complete. 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 Processor -- start 2024-04-25T14:08:28.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 -- start start 2024-04-25T14:08:28.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b01041f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10b0102260 0x7f10b0102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b01050b0 0x7f10b0102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10b010f640 con 0x7f10b0101760 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10b010f4c0 con 0x7f10b01050b0 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10b5ebd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10b010f7c0 con 0x7f10b0102260 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b01041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b01041f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34980/0 (socket says 172.21.15.78:34980) 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.092+0000 7f10af7fe640 1 -- 172.21.15.78:0/2996620398 learned_addr learned my addr 172.21.15.78:0/2996620398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10affff640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b01050b0 0x7f10b0102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10aeffd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10b0102260 0x7f10b0102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10af7fe640 1 -- 172.21.15.78:0/2996620398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10b0102260 msgr2=0x7f10b0102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10af7fe640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10b0102260 0x7f10b0102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10af7fe640 1 -- 172.21.15.78:0/2996620398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b01050b0 msgr2=0x7f10b0102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10af7fe640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b01050b0 0x7f10b0102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10af7fe640 1 -- 172.21.15.78:0/2996620398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b0103570 con 0x7f10b0101760 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10aeffd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10b0102260 0x7f10b0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10affff640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b01050b0 0x7f10b0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10af7fe640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b01041f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c970 tx=0x7f10a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10acff9640 1 -- 172.21.15.78:0/2996620398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0004220 con 0x7f10b0101760 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b01a4370 con 0x7f10b0101760 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10acff9640 1 -- 172.21.15.78:0/2996620398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a00043c0 con 0x7f10b0101760 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10acff9640 1 -- 172.21.15.78:0/2996620398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a000f450 con 0x7f10b0101760 2024-04-25T14:08:28.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.093+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b01a4830 con 0x7f10b0101760 2024-04-25T14:08:28.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 Processor -- start 2024-04-25T14:08:28.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10acff9640 1 -- 172.21.15.78:0/2996620398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f10a00079b0 con 0x7f10b0101760 2024-04-25T14:08:28.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 -- start start 2024-04-25T14:08:28.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6888810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6888d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6889a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7f67ecd80 con 0x55c7f64e6390 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7f67a6a20 con 0x55c7f64c56b0 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10bcd74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c7f67e8d40 con 0x55c7f64bd4d0 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6888810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6888810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34988/0 (socket says 172.21.15.78:34988) 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10affff640 1 -- 172.21.15.78:0/2878881346 learned_addr learned my addr 172.21.15.78:0/2878881346 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10af7fe640 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6888d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10aeffd640 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6889a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10affff640 1 -- 172.21.15.78:0/2878881346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 msgr2=0x55c7f6888d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.096+0000 7f10affff640 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6888d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10acff9640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f109c0afaf0 0x7f109c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10affff640 1 -- 172.21.15.78:0/2878881346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 msgr2=0x55c7f6889a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10affff640 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6889a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10affff640 1 -- 172.21.15.78:0/2878881346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f67b1c30 con 0x55c7f64e6390 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10af7fe640 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6888d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10aeffd640 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6889a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10acff9640 1 -- 172.21.15.78:0/2996620398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f10a00ded90 con 0x7f10b0101760 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10affff640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f109c0afaf0 0x7f109c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10affff640 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6888810 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009fb0 tx=0x7f10a8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a59ab8020c2c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10a5ffb640 1 -- 172.21.15.78:0/2878881346 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8016070 con 0x55c7f64e6390 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10a5ffb640 1 -- 172.21.15.78:0/2878881346 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a801c4c0 con 0x55c7f64e6390 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.097+0000 7f10a5ffb640 1 -- 172.21.15.78:0/2878881346 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a801b700 con 0x55c7f64e6390 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 -- 172.21.15.78:0/2878881346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 msgr2=0x55c7f6888810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10affff640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f109c0afaf0 0x7f109c0b2000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f10a8011010 tx=0x7f10a801cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6888810 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009fb0 tx=0x7f10a8010800 comp rx=0 tx=0).stop 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 -- 172.21.15.78:0/2878881346 shutdown_connections 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6889a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6888d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2878881346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6888810 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 -- 172.21.15.78:0/2878881346 >> 172.21.15.78:0/2878881346 conn(0x55c7f64da4e0 msgr2=0x55c7f6884020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 -- 172.21.15.78:0/2878881346 shutdown_connections 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 -- 172.21.15.78:0/2878881346 wait complete. 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 -- start start 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.098+0000 7f10bcd74740 1 Processor -- start 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f68924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6892a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6899960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c7f688a530 con 0x55c7f64e6390 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7f6773dc0 con 0x55c7f64c56b0 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c7f688a6b0 con 0x55c7f64bd4d0 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 10 client.0 _ll_register_callbacks cb 0x55c7f678f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6899960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6899960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34996/0 (socket says 172.21.15.78:34996) 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 -- 172.21.15.78:0/1142724302 learned_addr learned my addr 172.21.15.78:0/1142724302 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10af7fe640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f68924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10aeffd640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6892a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 -- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 msgr2=0x55c7f68924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f68924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 -- 172.21.15.78:0/1142724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 msgr2=0x55c7f6892a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6892a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 -- 172.21.15.78:0/1142724302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f64fe3f0 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10aeffd640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6892a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10affff640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6899960 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f10a8010e80 tx=0x7f10a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1710d180487ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f108effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f108effd640 1 -- 172.21.15.78:0/1142724302 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8037060 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f108effd640 1 -- 172.21.15.78:0/1142724302 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f10a8012cf0 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.099+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a801c4c0 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f108effd640 1 -- 172.21.15.78:0/1142724302 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f10a801b3e0 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 10 client.18144 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c7f65c87d0 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f108effd640 1 -- 172.21.15.78:0/1142724302 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a801c4c0 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f108effd640 1 -- 172.21.15.78:0/1142724302 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f10a800f040 con 0x55c7f64e6390 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 10 client.18144 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:28.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 10 client.18144 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 20 client.18144 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f64a4f60 0x55c7f689ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 4 client.18144 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f67fa990 con 0x55c7f64a4f60 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.100+0000 7f10af7fe640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f64a4f60 0x55c7f689ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.101+0000 7f108effd640 10 client.18144 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.101+0000 7f10af7fe640 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f64a4f60 0x55c7f689ff90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6942d0f89e41458c server_cookie=4de18bd6edc2c7a2 in_seq=0 out_seq=0 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.101+0000 7f108effd640 1 -- 172.21.15.78:0/1142724302 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c7f67fa990 con 0x55c7f64a4f60 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.101+0000 7f108effd640 10 client.18144 handle_command_reply: tid=0 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 client.18144 shutdown 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 msgr2=0x55c7f6899960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6899960 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f10a8010e80 tx=0x7f10a8016000 comp rx=0 tx=0).stop 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 shutdown_connections 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6899960 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6892a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f64a4f60 0x55c7f689ff90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 --2- 172.21.15.78:0/1142724302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f68924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f108effd640 0 client.18144 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:08:28.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 >> 172.21.15.78:0/1142724302 conn(0x55c7f64da4e0 msgr2=0x55c7f6886f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 shutdown_connections 2024-04-25T14:08:28.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 1 -- 172.21.15.78:0/1142724302 wait complete. 2024-04-25T14:08:28.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.102+0000 7f10bcd74740 20 client.18144 trim_cache size 0 max 16384 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 Processor -- start 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 -- start start 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6883a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6899880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6883f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01afe30 con 0x55c7f64e6390 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01ad700 con 0x55c7f64c56b0 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10bcd74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01ab550 con 0x55c7f64bd4d0 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6883f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6883f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35012/0 (socket says 172.21.15.78:35012) 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.115+0000 7f10affff640 1 -- 172.21.15.78:0/2321743172 learned_addr learned my addr 172.21.15.78:0/2321743172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10af7fe640 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6883a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10affff640 1 -- 172.21.15.78:0/2321743172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 msgr2=0x55c7f6883a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10affff640 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6883a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10affff640 1 -- 172.21.15.78:0/2321743172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 msgr2=0x55c7f6899880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10affff640 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6899880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10affff640 1 -- 172.21.15.78:0/2321743172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f6889840 con 0x55c7f64e6390 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10affff640 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6883f90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009ed0 tx=0x7f10a8012720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f60b5c28f26a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10a57fa640 1 -- 172.21.15.78:0/2321743172 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8037070 con 0x55c7f64e6390 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10a57fa640 1 -- 172.21.15.78:0/2321743172 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a802e040 con 0x55c7f64e6390 2024-04-25T14:08:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.116+0000 7f10a57fa640 1 -- 172.21.15.78:0/2321743172 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8041040 con 0x55c7f64e6390 2024-04-25T14:08:28.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- 172.21.15.78:0/2321743172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 msgr2=0x55c7f6883f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6883f90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009ed0 tx=0x7f10a8012720 comp rx=0 tx=0).stop 2024-04-25T14:08:28.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- 172.21.15.78:0/2321743172 shutdown_connections 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6883f90 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64c56b0 0x55c7f6899880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 --2- 172.21.15.78:0/2321743172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64bd4d0 0x55c7f6883a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- 172.21.15.78:0/2321743172 >> 172.21.15.78:0/2321743172 conn(0x55c7f64da4e0 msgr2=0x55c7f6888790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- 172.21.15.78:0/2321743172 shutdown_connections 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- 172.21.15.78:0/2321743172 wait complete. 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- start start 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 Processor -- start 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64bd4d0 0x55c7f6896980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64c56b0 0x55c7f6896ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6897df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01abbc0 con 0x55c7f64e6390 2024-04-25T14:08:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01ad030 con 0x55c7f64bd4d0 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01ac290 con 0x55c7f64c56b0 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.117+0000 7f10bcd74740 10 client.0 _ll_register_callbacks cb 0x7f10a8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6897df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6897df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35028/0 (socket says 172.21.15.78:35028) 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 -- 172.21.15.78:0/4197694751 learned_addr learned my addr 172.21.15.78:0/4197694751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10af7fe640 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64bd4d0 0x55c7f6896980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10aeffd640 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64c56b0 0x55c7f6896ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 -- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64c56b0 msgr2=0x55c7f6896ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64c56b0 0x55c7f6896ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 -- 172.21.15.78:0/4197694751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64bd4d0 msgr2=0x55c7f6896980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64bd4d0 0x55c7f6896980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 -- 172.21.15.78:0/4197694751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f68983b0 con 0x55c7f64e6390 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10affff640 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6897df0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f10a801b7e0 tx=0x7f10a80094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3d84329871723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f108effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f108effd640 1 -- 172.21.15.78:0/4197694751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a804c030 con 0x55c7f64e6390 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f108effd640 1 -- 172.21.15.78:0/4197694751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f10a8012d00 con 0x55c7f64e6390 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a802e040 con 0x55c7f64e6390 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f108effd640 1 -- 172.21.15.78:0/4197694751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f10a8041030 con 0x55c7f64e6390 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10bcd74740 10 client.18150 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c7f6899630 con 0x55c7f64e6390 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.118+0000 7f108effd640 1 -- 172.21.15.78:0/4197694751 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a802e040 con 0x55c7f64e6390 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f108effd640 1 -- 172.21.15.78:0/4197694751 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f10a801c540 con 0x55c7f64e6390 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10bcd74740 10 client.18150 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10bcd74740 10 client.18150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10bcd74740 20 client.18150 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10bcd74740 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f647a3a0 0x55c7f690a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10bcd74740 4 client.18150 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7f67f4800 con 0x55c7f647a3a0 2024-04-25T14:08:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10af7fe640 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f647a3a0 0x55c7f690a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f108effd640 10 client.18150 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.119+0000 7f10af7fe640 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f647a3a0 0x55c7f690a5c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=931742d66bef77ed server_cookie=e11d416e0af3ecd8 in_seq=0 out_seq=0 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f108effd640 1 -- 172.21.15.78:0/4197694751 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7f67f4800 con 0x55c7f647a3a0 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f108effd640 10 client.18150 handle_command_reply: tid=0 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 client.18150 shutdown 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 msgr2=0x55c7f6897df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6897df0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f10a801b7e0 tx=0x7f10a80094a0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 shutdown_connections 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7f64e6390 0x55c7f6897df0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55c7f647a3a0 0x55c7f690a5c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7f64c56b0 0x55c7f6896ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 --2- 172.21.15.78:0/4197694751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7f64bd4d0 0x55c7f6896980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f108effd640 0 client.18150 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:08:28.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 >> 172.21.15.78:0/4197694751 conn(0x55c7f64da4e0 msgr2=0x55c7f6886e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 shutdown_connections 2024-04-25T14:08:28.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 1 -- 172.21.15.78:0/4197694751 wait complete. 2024-04-25T14:08:28.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10bcd74740 20 client.18150 trim_cache size 0 max 16384 2024-04-25T14:08:28.130 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f109c0afaf0 msgr2=0x7f109c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f109c0afaf0 0x7f109c0b2000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f10a8011010 tx=0x7f10a801cf90 comp rx=0 tx=0).stop 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 msgr2=0x7f10b01041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.120+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b01041f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c970 tx=0x7f10a000ce30 comp rx=0 tx=0).stop 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 shutdown_connections 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10b01050b0 0x7f10b0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10b0102260 0x7f10b0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f109c0afaf0 0x7f109c0b2000 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 --2- 172.21.15.78:0/2996620398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10b0101760 0x7f10b01041f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 >> 172.21.15.78:0/2996620398 conn(0x7f10b00fd380 msgr2=0x7f10b0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 shutdown_connections 2024-04-25T14:08:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.121+0000 7f10b5ebd640 1 -- 172.21.15.78:0/2996620398 wait complete. 2024-04-25T14:08:28.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:08:28.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 -- 172.21.15.78:0/2477406893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548104230 msgr2=0x7f0548106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 --2- 172.21.15.78:0/2477406893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548104230 0x7f0548106650 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0544009900 tx=0x7f054402a410 comp rx=0 tx=0).stop 2024-04-25T14:08:28.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 -- 172.21.15.78:0/2477406893 shutdown_connections 2024-04-25T14:08:28.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 --2- 172.21.15.78:0/2477406893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0548109790 0x7f054810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 --2- 172.21.15.78:0/2477406893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0548106c20 0x7f0548109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 --2- 172.21.15.78:0/2477406893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548104230 0x7f0548106650 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 -- 172.21.15.78:0/2477406893 >> 172.21.15.78:0/2477406893 conn(0x7f05480fdbf0 msgr2=0x7f05481000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 -- 172.21.15.78:0/2477406893 shutdown_connections 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.243+0000 7f0550214640 1 -- 172.21.15.78:0/2477406893 wait complete. 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 Processor -- start 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 -- start start 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0548104230 0x7f0548103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0548106c20 0x7f0548102140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548109790 0x7f0548102680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f054810f5f0 con 0x7f0548109790 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f054810f470 con 0x7f0548106c20 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f0550214640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f054810f770 con 0x7f0548104230 2024-04-25T14:08:28.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548109790 0x7f0548102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548109790 0x7f0548102680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35040/0 (socket says 172.21.15.78:35040) 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 -- 172.21.15.78:0/176631749 learned_addr learned my addr 172.21.15.78:0/176631749 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054dfb0640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0548104230 0x7f0548103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054d7af640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0548106c20 0x7f0548102140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 -- 172.21.15.78:0/176631749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0548104230 msgr2=0x7f0548103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0548104230 0x7f0548103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 -- 172.21.15.78:0/176631749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0548106c20 msgr2=0x7f0548102140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0548106c20 0x7f0548102140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054e7b1640 1 -- 172.21.15.78:0/176631749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0548102db0 con 0x7f0548109790 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.244+0000 7f054d7af640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0548106c20 0x7f0548102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.245+0000 7f054e7b1640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548109790 0x7f0548102680 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f054000eb10 tx=0x7f054000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.245+0000 7f053effd640 1 -- 172.21.15.78:0/176631749 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540004b20 con 0x7f0548109790 2024-04-25T14:08:28.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.245+0000 7f053effd640 1 -- 172.21.15.78:0/176631749 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0540004cc0 con 0x7f0548109790 2024-04-25T14:08:28.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.245+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05481a42e0 con 0x7f0548109790 2024-04-25T14:08:28.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.245+0000 7f053effd640 1 -- 172.21.15.78:0/176631749 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540010670 con 0x7f0548109790 2024-04-25T14:08:28.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.245+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05481a47d0 con 0x7f0548109790 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.247+0000 7f055719b740 1 Processor -- start 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.247+0000 7f053effd640 1 -- 172.21.15.78:0/176631749 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0540005020 con 0x7f0548109790 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f055719b740 1 -- start start 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f055719b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0ddd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f055719b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f055719b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0ddea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f055719b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556aa0d41d80 con 0x556aa0a3b390 2024-04-25T14:08:28.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f055719b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556aa0cfba20 con 0x556aa0a1a6b0 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f055719b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556aa0d3dd40 con 0x556aa0a124d0 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0ddd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0ddd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35048/0 (socket says 172.21.15.78:35048) 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f054d7af640 1 -- 172.21.15.78:0/1697751842 learned_addr learned my addr 172.21.15.78:0/1697751842 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f054dfb0640 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f053effd640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05340afb70 0x7f05340b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f054d7af640 1 -- 172.21.15.78:0/1697751842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 msgr2=0x556aa0dddd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.248+0000 7f054d7af640 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dddd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f054d7af640 1 -- 172.21.15.78:0/1697751842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 msgr2=0x556aa0ddea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f054d7af640 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0ddea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f054d7af640 1 -- 172.21.15.78:0/1697751842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa0d06c30 con 0x556aa0a3b390 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f054dfb0640 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:28.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f054e7b1640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05340afb70 0x7f05340b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f053effd640 1 -- 172.21.15.78:0/176631749 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f05400df1d0 con 0x7f0548109790 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f054d7af640 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0ddd810 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0538009cd0 tx=0x7f0538012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c5dcd84936b646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f0533fff640 1 -- 172.21.15.78:0/1697751842 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538004030 con 0x556aa0a3b390 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f0533fff640 1 -- 172.21.15.78:0/1697751842 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05380041d0 con 0x556aa0a3b390 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.249+0000 7f0533fff640 1 -- 172.21.15.78:0/1697751842 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538013660 con 0x556aa0a3b390 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 -- 172.21.15.78:0/1697751842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 msgr2=0x556aa0ddd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f054e7b1640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05340afb70 0x7f05340b2080 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f05400dcd10 tx=0x7f054004b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0ddd810 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0538009cd0 tx=0x7f0538012760 comp rx=0 tx=0).stop 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 -- 172.21.15.78:0/1697751842 shutdown_connections 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0ddea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 --2- 172.21.15.78:0/1697751842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0ddd810 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 -- 172.21.15.78:0/1697751842 >> 172.21.15.78:0/1697751842 conn(0x556aa0a2f4e0 msgr2=0x556aa0dd9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 -- 172.21.15.78:0/1697751842 shutdown_connections 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 -- 172.21.15.78:0/1697751842 wait complete. 2024-04-25T14:08:28.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 -- start start 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.250+0000 7f055719b740 1 Processor -- start 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f055719b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a124d0 0x556aa0de7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f055719b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0de7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f055719b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a3b390 0x556aa0dee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f055719b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556aa0ddf530 con 0x556aa0a124d0 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f055719b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556aa0cc8dc0 con 0x556aa0a3b390 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f055719b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556aa0ddf6b0 con 0x556aa0a1a6b0 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f055719b740 10 client.0 _ll_register_callbacks cb 0x556aa0ce4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a124d0 0x556aa0de7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a124d0 0x556aa0de7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35054/0 (socket says 172.21.15.78:35054) 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 -- 172.21.15.78:0/2679074573 learned_addr learned my addr 172.21.15.78:0/2679074573 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054dfb0640 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0de7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a3b390 0x556aa0dee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 -- 172.21.15.78:0/2679074573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 msgr2=0x556aa0de7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0de7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 -- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a3b390 msgr2=0x556aa0dee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a3b390 0x556aa0dee9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 -- 172.21.15.78:0/2679074573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa0a533f0 con 0x556aa0a124d0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f054d7af640 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a124d0 0x556aa0de7530 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0538012c60 tx=0x7f0538005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93480652ee10af9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.251+0000 7f05317fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f05317fa640 1 -- 172.21.15.78:0/2679074573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538021070 con 0x556aa0a124d0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05380041d0 con 0x556aa0a124d0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f05317fa640 1 -- 172.21.15.78:0/2679074573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f053802a440 con 0x556aa0a124d0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f05317fa640 1 -- 172.21.15.78:0/2679074573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0538024040 con 0x556aa0a124d0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 10 client.18162 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556aa0b1d7d0 con 0x556aa0a124d0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f05317fa640 1 -- 172.21.15.78:0/2679074573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05380041d0 con 0x556aa0a124d0 2024-04-25T14:08:28.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f05317fa640 1 -- 172.21.15.78:0/2679074573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0538028020 con 0x556aa0a124d0 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 10 client.18162 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 20 client.18162 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09f9f60 0x556aa0df5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 4 client.18162 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa0d4f990 con 0x556aa09f9f60 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.252+0000 7f054dfb0640 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09f9f60 0x556aa0df5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.253+0000 7f05317fa640 10 client.18162 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.253+0000 7f054dfb0640 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09f9f60 0x556aa0df5000 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ba2a52df7a780c server_cookie=4b03fae6b88bcdac in_seq=0 out_seq=0 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f05317fa640 1 -- 172.21.15.78:0/2679074573 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556aa0d4f990 con 0x556aa09f9f60 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f05317fa640 10 client.18162 handle_command_reply: tid=0 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 client.18162 shutdown 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a124d0 msgr2=0x556aa0de7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a124d0 0x556aa0de7530 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0538012c60 tx=0x7f0538005d30 comp rx=0 tx=0).stop 2024-04-25T14:08:28.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 shutdown_connections 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a3b390 0x556aa0dee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09f9f60 0x556aa0df5000 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0de7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 --2- 172.21.15.78:0/2679074573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a124d0 0x556aa0de7530 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f05317fa640 0 client.18162 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 >> 172.21.15.78:0/2679074573 conn(0x556aa0a2f4e0 msgr2=0x556aa0ddbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 shutdown_connections 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 1 -- 172.21.15.78:0/2679074573 wait complete. 2024-04-25T14:08:28.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.254+0000 7f055719b740 20 client.18162 trim_cache size 0 max 16384 2024-04-25T14:08:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 Processor -- start 2024-04-25T14:08:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 -- start start 2024-04-25T14:08:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dd8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0dd8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0de5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05481afe50 con 0x556aa0a3b390 2024-04-25T14:08:28.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05481ad720 con 0x556aa0a1a6b0 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.267+0000 7f055719b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05481ab570 con 0x556aa0a124d0 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0de5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0de5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35060/0 (socket says 172.21.15.78:35060) 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 -- 172.21.15.78:0/1346362329 learned_addr learned my addr 172.21.15.78:0/1346362329 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054dfb0640 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0dd8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dd8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 -- 172.21.15.78:0/1346362329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 msgr2=0x556aa0dd8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dd8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 -- 172.21.15.78:0/1346362329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 msgr2=0x556aa0dd8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0dd8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 -- 172.21.15.78:0/1346362329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa0ddf790 con 0x556aa0a3b390 2024-04-25T14:08:28.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054dfb0640 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0dd8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f054d7af640 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0de5cb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0538013010 tx=0x7f0538009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880bc2ec5827734f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f05337fe640 1 -- 172.21.15.78:0/1346362329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538021040 con 0x556aa0a3b390 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f05337fe640 1 -- 172.21.15.78:0/1346362329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0538025040 con 0x556aa0a3b390 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.268+0000 7f05337fe640 1 -- 172.21.15.78:0/1346362329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538013b90 con 0x556aa0a3b390 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 -- 172.21.15.78:0/1346362329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 msgr2=0x556aa0de5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0de5cb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0538013010 tx=0x7f0538009a70 comp rx=0 tx=0).stop 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 -- 172.21.15.78:0/1346362329 shutdown_connections 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0de5cb0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a1a6b0 0x556aa0dd8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 --2- 172.21.15.78:0/1346362329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a124d0 0x556aa0dd8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 -- 172.21.15.78:0/1346362329 >> 172.21.15.78:0/1346362329 conn(0x556aa0a2f4e0 msgr2=0x556aa0de2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 -- 172.21.15.78:0/1346362329 shutdown_connections 2024-04-25T14:08:28.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 -- 172.21.15.78:0/1346362329 wait complete. 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 -- start start 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.269+0000 7f055719b740 1 Processor -- start 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f055719b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a124d0 0x556aa0dead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f055719b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0deb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f055719b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0e5b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f055719b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05481abbe0 con 0x556aa0a3b390 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f055719b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05481ad050 con 0x556aa0a124d0 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f055719b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05481ac2b0 con 0x556aa0a1a6b0 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f055719b740 10 client.0 _ll_register_callbacks cb 0x7f0538009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0e5b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0e5b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35074/0 (socket says 172.21.15.78:35074) 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 -- 172.21.15.78:0/2327103016 learned_addr learned my addr 172.21.15.78:0/2327103016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054dfb0640 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0deb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a124d0 0x556aa0dead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 -- 172.21.15.78:0/2327103016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 msgr2=0x556aa0deb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0deb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 -- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a124d0 msgr2=0x556aa0dead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a124d0 0x556aa0dead80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 -- 172.21.15.78:0/2327103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa0dee280 con 0x556aa0a3b390 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f054d7af640 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0e5b230 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0538013950 tx=0x7f0538003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7319ba636ffa8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f05317fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.270+0000 7f05317fa640 1 -- 172.21.15.78:0/2327103016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538021020 con 0x556aa0a3b390 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f05317fa640 1 -- 172.21.15.78:0/2327103016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0538004440 con 0x556aa0a3b390 2024-04-25T14:08:28.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0538025040 con 0x556aa0a3b390 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f05317fa640 1 -- 172.21.15.78:0/2327103016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0538045080 con 0x556aa0a3b390 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 10 client.18168 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556aa0e5c6a0 con 0x556aa0a3b390 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f05317fa640 1 -- 172.21.15.78:0/2327103016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0538025040 con 0x556aa0a3b390 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f05317fa640 1 -- 172.21.15.78:0/2327103016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0538028070 con 0x556aa0a3b390 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 10 client.18168 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 20 client.18168 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09cf3a0 0x556aa0e5f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 4 client.18168 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556aa0d49800 con 0x556aa09cf3a0 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.271+0000 7f054dfb0640 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09cf3a0 0x556aa0e5f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f054dfb0640 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09cf3a0 0x556aa0e5f000 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b60466b6e1125e server_cookie=47d6063e4910d834 in_seq=0 out_seq=0 2024-04-25T14:08:28.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f05317fa640 10 client.18168 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f05317fa640 1 -- 172.21.15.78:0/2327103016 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556aa0d49800 con 0x556aa09cf3a0 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f05317fa640 10 client.18168 handle_command_reply: tid=0 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 client.18168 shutdown 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 msgr2=0x556aa0e5b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0e5b230 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0538013950 tx=0x7f0538003a30 comp rx=0 tx=0).stop 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 shutdown_connections 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556aa0a3b390 0x556aa0e5b230 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x556aa09cf3a0 0x556aa0e5f000 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556aa0a1a6b0 0x556aa0deb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 --2- 172.21.15.78:0/2327103016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556aa0a124d0 0x556aa0dead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f05317fa640 0 client.18168 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 >> 172.21.15.78:0/2327103016 conn(0x556aa0a2f4e0 msgr2=0x556aa0dd89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 shutdown_connections 2024-04-25T14:08:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 1 -- 172.21.15.78:0/2327103016 wait complete. 2024-04-25T14:08:28.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.272+0000 7f055719b740 20 client.18168 trim_cache size 0 max 16384 2024-04-25T14:08:28.280 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05340afb70 msgr2=0x7f05340b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05340afb70 0x7f05340b2080 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f05400dcd10 tx=0x7f054004b040 comp rx=0 tx=0).stop 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548109790 msgr2=0x7f0548102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548109790 0x7f0548102680 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f054000eb10 tx=0x7f054000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 shutdown_connections 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f05340afb70 0x7f05340b2080 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0548109790 0x7f0548102680 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0548106c20 0x7f0548102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 --2- 172.21.15.78:0/176631749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0548104230 0x7f0548103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 >> 172.21.15.78:0/176631749 conn(0x7f05480fdbf0 msgr2=0x7f0548106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 shutdown_connections 2024-04-25T14:08:28.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:28.273+0000 7f0550214640 1 -- 172.21.15.78:0/176631749 wait complete. 2024-04-25T14:08:28.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:28 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3038311196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:28.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:28 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3038311196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:29.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:28 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3038311196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:08:29.284 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:08:29.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:08:29.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:08:29.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 -- 172.21.15.78:0/2203187439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a41045f0 msgr2=0x7fc4a4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 --2- 172.21.15.78:0/2203187439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a41045f0 0x7fc4a4106a10 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009900 tx=0x7fc4a002a410 comp rx=0 tx=0).stop 2024-04-25T14:08:29.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 -- 172.21.15.78:0/2203187439 shutdown_connections 2024-04-25T14:08:29.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 --2- 172.21.15.78:0/2203187439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a41098b0 0x7fc4a410bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 --2- 172.21.15.78:0/2203187439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a4106f50 0x7fc4a4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 --2- 172.21.15.78:0/2203187439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a41045f0 0x7fc4a4106a10 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 -- 172.21.15.78:0/2203187439 >> 172.21.15.78:0/2203187439 conn(0x7fc4a40fdbf0 msgr2=0x7fc4a41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 -- 172.21.15.78:0/2203187439 shutdown_connections 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.406+0000 7fc4ac020640 1 -- 172.21.15.78:0/2203187439 wait complete. 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 Processor -- start 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 -- start start 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a41045f0 0x7fc4a419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a4106f50 0x7fc4a419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a41098b0 0x7fc4a41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a410f5e0 con 0x7fc4a4106f50 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a410f460 con 0x7fc4a41045f0 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4ac020640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a410f760 con 0x7fc4a41098b0 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a4106f50 0x7fc4a419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a4106f50 0x7fc4a419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35098/0 (socket says 172.21.15.78:35098) 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 -- 172.21.15.78:0/312656214 learned_addr learned my addr 172.21.15.78:0/312656214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a41045f0 0x7fc4a419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a41098b0 0x7fc4a41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 -- 172.21.15.78:0/312656214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a41098b0 msgr2=0x7fc4a41a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a41098b0 0x7fc4a41a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 -- 172.21.15.78:0/312656214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a41045f0 msgr2=0x7fc4a419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a41045f0 0x7fc4a419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 -- 172.21.15.78:0/312656214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a41a1d90 con 0x7fc4a4106f50 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a41098b0 0x7fc4a41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.407+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a4106f50 0x7fc4a419a790 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc49400d950 tx=0x7fc49400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.408+0000 7fc49affd640 1 -- 172.21.15.78:0/312656214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494017030 con 0x7fc4a4106f50 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.408+0000 7fc49affd640 1 -- 172.21.15.78:0/312656214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc494004020 con 0x7fc4a4106f50 2024-04-25T14:08:29.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.408+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a41a2080 con 0x7fc4a4106f50 2024-04-25T14:08:29.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.408+0000 7fc49affd640 1 -- 172.21.15.78:0/312656214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940047a0 con 0x7fc4a4106f50 2024-04-25T14:08:29.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.408+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a41a25d0 con 0x7fc4a4106f50 2024-04-25T14:08:29.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.410+0000 7fc4b2fad740 1 Processor -- start 2024-04-25T14:08:29.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.410+0000 7fc49affd640 1 -- 172.21.15.78:0/312656214 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc494009a30 con 0x7fc4a4106f50 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.410+0000 7fc4b2fad740 1 -- start start 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7f9410 0x56045ab9a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045ab9aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045aba1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56045aaffb00 con 0x56045a7d8730 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56045aa9b920 con 0x56045a7d0550 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56045aafbac0 con 0x56045a7f9410 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045aba1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045aba1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35108/0 (socket says 172.21.15.78:35108) 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 -- 172.21.15.78:0/350346554 learned_addr learned my addr 172.21.15.78:0/350346554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045ab9aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7f9410 0x56045ab9a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 -- 172.21.15.78:0/350346554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7f9410 msgr2=0x56045ab9a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7f9410 0x56045ab9a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 -- 172.21.15.78:0/350346554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 msgr2=0x56045ab9aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045ab9aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 -- 172.21.15.78:0/350346554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56045a8d97e0 con 0x56045a7d8730 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc49affd640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4900afaf0 0x7fc4900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4900afaf0 0x7fc4900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.411+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045aba1920 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc4940e4a90 tx=0x7fc4940e4f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd623eec049bc3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.412+0000 7fc49affd640 1 -- 172.21.15.78:0/312656214 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc4940de710 con 0x7fc4a4106f50 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.412+0000 7fc48ffff640 1 -- 172.21.15.78:0/350346554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940a2a80 con 0x56045a7d8730 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.412+0000 7fc48ffff640 1 -- 172.21.15.78:0/350346554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4940af370 con 0x56045a7d8730 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.412+0000 7fc48ffff640 1 -- 172.21.15.78:0/350346554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940a6330 con 0x56045a7d8730 2024-04-25T14:08:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 -- 172.21.15.78:0/350346554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 msgr2=0x56045aba1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4900afaf0 0x7fc4900b2000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x56045ab9b670 tx=0x7fc49c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045aba1920 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc4940e4a90 tx=0x7fc4940e4f50 comp rx=0 tx=0).stop 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 -- 172.21.15.78:0/350346554 shutdown_connections 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045aba1920 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045ab9aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/350346554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7f9410 0x56045ab9a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 -- 172.21.15.78:0/350346554 >> 172.21.15.78:0/350346554 conn(0x56045a7ed560 msgr2=0x56045ab95e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 -- 172.21.15.78:0/350346554 shutdown_connections 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 -- 172.21.15.78:0/350346554 wait complete. 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 -- start start 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.413+0000 7fc4b2fad740 1 Processor -- start 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045aba42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7f9410 0x56045abab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56045aba2370 con 0x56045a7f9410 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56045aa995d0 con 0x56045a7d0550 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56045aba24f0 con 0x56045a7d8730 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 10 client.0 _ll_register_callbacks cb 0x56045aa87080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7f9410 0x56045abab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7f9410 0x56045abab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35110/0 (socket says 172.21.15.78:35110) 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 -- 172.21.15.78:0/604425762 learned_addr learned my addr 172.21.15.78:0/604425762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045aba42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 -- 172.21.15.78:0/604425762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 msgr2=0x56045aba47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 -- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 msgr2=0x56045aba42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045aba42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 -- 172.21.15.78:0/604425762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045a939f70 con 0x56045a7f9410 2024-04-25T14:08:29.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7f9410 0x56045abab740 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc49c012900 tx=0x7fc49c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7da0c6c5970fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc48d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc48d7fa640 1 -- 172.21.15.78:0/604425762 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c004d20 con 0x56045a7f9410 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc48d7fa640 1 -- 172.21.15.78:0/604425762 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc49c00faf0 con 0x56045a7f9410 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.414+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4940af370 con 0x56045a7f9410 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc48d7fa640 1 -- 172.21.15.78:0/604425762 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc49c00fcf0 con 0x56045a7f9410 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 10 client.18180 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56045a7d1170 con 0x56045a7f9410 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc48d7fa640 1 -- 172.21.15.78:0/604425762 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4940af370 con 0x56045a7f9410 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc48d7fa640 1 -- 172.21.15.78:0/604425762 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc49c027400 con 0x56045a7f9410 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 10 client.18180 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 20 client.18180 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a7b81f0 0x56045abb1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 4 client.18180 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56045ab0d710 con 0x56045a7b81f0 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.415+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a7b81f0 0x56045abb1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.416+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a7b81f0 0x56045abb1d20 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8a7567a41e81a server_cookie=9ea5e5e9c09da1eb in_seq=0 out_seq=0 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.416+0000 7fc48d7fa640 10 client.18180 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.416+0000 7fc48d7fa640 1 -- 172.21.15.78:0/604425762 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56045ab0d710 con 0x56045a7b81f0 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.416+0000 7fc48d7fa640 10 client.18180 handle_command_reply: tid=0 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.416+0000 7fc4b2fad740 1 client.18180 shutdown 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7f9410 msgr2=0x56045abab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7f9410 0x56045abab740 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc49c012900 tx=0x7fc49c004a40 comp rx=0 tx=0).stop 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 shutdown_connections 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7f9410 0x56045abab740 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a7b81f0 0x56045abb1d20 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/604425762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7d0550 0x56045aba42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 >> 172.21.15.78:0/604425762 conn(0x56045a7ed560 msgr2=0x56045ab94ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc48d7fa640 0 client.18180 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 shutdown_connections 2024-04-25T14:08:29.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 1 -- 172.21.15.78:0/604425762 wait complete. 2024-04-25T14:08:29.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.417+0000 7fc4b2fad740 20 client.18180 trim_cache size 0 max 16384 2024-04-25T14:08:29.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 Processor -- start 2024-04-25T14:08:29.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 -- start start 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d0550 0x56045ab9ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045abab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045ababba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41adc40 con 0x56045a7d8730 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41ab510 con 0x56045a7f9410 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41a9360 con 0x56045a7d0550 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045abab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045abab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35116/0 (socket says 172.21.15.78:35116) 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 -- 172.21.15.78:0/178102096 learned_addr learned my addr 172.21.15.78:0/178102096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d0550 0x56045ab9ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 -- 172.21.15.78:0/178102096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d0550 msgr2=0x56045ab9ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d0550 0x56045ab9ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 -- 172.21.15.78:0/178102096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 msgr2=0x56045ababba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045ababba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045ababba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.435+0000 7fc4a95bb640 1 -- 172.21.15.78:0/178102096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56045aba4ea0 con 0x56045a7d8730 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.436+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d0550 0x56045ab9ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.436+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045abab660 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a3e30 tx=0x7fc4940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41712d90d50966fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.436+0000 7fc48f7fe640 1 -- 172.21.15.78:0/178102096 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940a6510 con 0x56045a7d8730 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.436+0000 7fc48f7fe640 1 -- 172.21.15.78:0/178102096 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4940af370 con 0x56045a7d8730 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.436+0000 7fc48f7fe640 1 -- 172.21.15.78:0/178102096 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940a6990 con 0x56045a7d8730 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 -- 172.21.15.78:0/178102096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 msgr2=0x56045abab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045abab660 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc4940a3e30 tx=0x7fc4940046c0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 -- 172.21.15.78:0/178102096 shutdown_connections 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045ababba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d8730 0x56045abab660 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/178102096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d0550 0x56045ab9ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 -- 172.21.15.78:0/178102096 >> 172.21.15.78:0/178102096 conn(0x56045a7ed560 msgr2=0x56045ab9d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 -- 172.21.15.78:0/178102096 shutdown_connections 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 -- 172.21.15.78:0/178102096 wait complete. 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 -- start start 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 Processor -- start 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d0550 0x56045aba9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045abaadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.437+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41a99d0 con 0x56045a7d0550 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41aae40 con 0x56045a7f9410 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4b2fad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a41aa0a0 con 0x56045a7d8730 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d0550 0x56045aba9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4b2fad740 10 client.0 _ll_register_callbacks cb 0x7fc49c0125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:29.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d0550 0x56045aba9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35130/0 (socket says 172.21.15.78:35130) 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 -- 172.21.15.78:0/3980713416 learned_addr learned my addr 172.21.15.78:0/3980713416 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045abaadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 -- 172.21.15.78:0/3980713416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 msgr2=0x56045aba9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 -- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 msgr2=0x56045abaadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045abaadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 -- 172.21.15.78:0/3980713416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045abab390 con 0x56045a7d0550 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4aa5bd640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045abaadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4a9dbc640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d0550 0x56045aba9960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009900 tx=0x7fc4a00315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31c0a701f751949 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc48d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc48d7fa640 1 -- 172.21.15.78:0/3980713416 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a004a020 con 0x56045a7d0550 2024-04-25T14:08:29.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc48d7fa640 1 -- 172.21.15.78:0/3980713416 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc4a0004240 con 0x56045a7d0550 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4940af370 con 0x56045a7d0550 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc48d7fa640 1 -- 172.21.15.78:0/3980713416 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0055050 con 0x56045a7d0550 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4b2fad740 10 client.18186 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56045ac1a300 con 0x56045a7d0550 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.438+0000 7fc48d7fa640 1 -- 172.21.15.78:0/3980713416 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4940af370 con 0x56045a7d0550 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc48d7fa640 1 -- 172.21.15.78:0/3980713416 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc4a0003790 con 0x56045a7d0550 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4b2fad740 10 client.18186 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4b2fad740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4b2fad740 20 client.18186 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a78d630 0x56045ac1cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4b2fad740 4 client.18186 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56045ab07580 con 0x56045a78d630 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a78d630 0x56045ac1cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc48d7fa640 10 client.18186 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.439+0000 7fc4a95bb640 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a78d630 0x56045ac1cd20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ba598941affbbc server_cookie=55c20c2c9653a8ed in_seq=0 out_seq=0 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.440+0000 7fc48d7fa640 1 -- 172.21.15.78:0/3980713416 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56045ab07580 con 0x56045a78d630 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.440+0000 7fc48d7fa640 10 client.18186 handle_command_reply: tid=0 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.440+0000 7fc4b2fad740 1 client.18186 shutdown 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.440+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d0550 msgr2=0x56045aba9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.440+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d0550 0x56045aba9960 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009900 tx=0x7fc4a00315a0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 shutdown_connections 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56045a7f9410 0x56045abaadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56045a78d630 0x56045ac1cd20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56045a7d8730 0x56045aba9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 --2- 172.21.15.78:0/3980713416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56045a7d0550 0x56045aba9960 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc48d7fa640 0 client.18186 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 >> 172.21.15.78:0/3980713416 conn(0x56045a7ed560 msgr2=0x56045ab98c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 shutdown_connections 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 1 -- 172.21.15.78:0/3980713416 wait complete. 2024-04-25T14:08:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4b2fad740 20 client.18186 trim_cache size 0 max 16384 2024-04-25T14:08:29.447 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:08:29.447 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:08:29.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "2f680f62-af29-4353-abe1-c7c265d2b497", 2024-04-25T14:08:29.447 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:08:29.447 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4900afaf0 msgr2=0x7fc4900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4900afaf0 0x7fc4900b2000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x56045ab9b670 tx=0x7fc49c009680 comp rx=0 tx=0).stop 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a4106f50 msgr2=0x7fc4a419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a4106f50 0x7fc4a419a790 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc49400d950 tx=0x7fc49400de10 comp rx=0 tx=0).stop 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 shutdown_connections 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a41098b0 0x7fc4a41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4900afaf0 0x7fc4900b2000 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a4106f50 0x7fc4a419a790 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 --2- 172.21.15.78:0/312656214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a41045f0 0x7fc4a419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 >> 172.21.15.78:0/312656214 conn(0x7fc4a40fdbf0 msgr2=0x7fc4a41065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 shutdown_connections 2024-04-25T14:08:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.441+0000 7fc4ac020640 1 -- 172.21.15.78:0/312656214 wait complete. 2024-04-25T14:08:29.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:08:29.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:08:29.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 -- 172.21.15.78:0/940414057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288102a00 msgr2=0x7f3288103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 --2- 172.21.15.78:0/940414057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288102a00 0x7f3288103700 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3284009900 tx=0x7f328402a410 comp rx=0 tx=0).stop 2024-04-25T14:08:29.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 -- 172.21.15.78:0/940414057 shutdown_connections 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 --2- 172.21.15.78:0/940414057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3288108940 0x7f328810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 --2- 172.21.15.78:0/940414057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3288103c40 0x7f3288104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 --2- 172.21.15.78:0/940414057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288102a00 0x7f3288103700 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 -- 172.21.15.78:0/940414057 >> 172.21.15.78:0/940414057 conn(0x7f32880fdbf0 msgr2=0x7f32881000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 -- 172.21.15.78:0/940414057 shutdown_connections 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.562+0000 7f328faf9640 1 -- 172.21.15.78:0/940414057 wait complete. 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 Processor -- start 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 -- start start 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3288102a00 0x7f3288075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3288103c40 0x7f328819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288108940 0x7f32881a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f328810e550 con 0x7f3288108940 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f328810e3d0 con 0x7f3288102a00 2024-04-25T14:08:29.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328faf9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f328810e6d0 con 0x7f3288103c40 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288108940 0x7f32881a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288108940 0x7f32881a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35152/0 (socket says 172.21.15.78:35152) 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 -- 172.21.15.78:0/3775698740 learned_addr learned my addr 172.21.15.78:0/3775698740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328d094640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3288103c40 0x7f328819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328d895640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3288102a00 0x7f3288075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 -- 172.21.15.78:0/3775698740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3288103c40 msgr2=0x7f328819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3288103c40 0x7f328819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 -- 172.21.15.78:0/3775698740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3288102a00 msgr2=0x7f3288075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3288102a00 0x7f3288075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.563+0000 7f328e096640 1 -- 172.21.15.78:0/3775698740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32881a1d90 con 0x7f3288108940 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.564+0000 7f328e096640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288108940 0x7f32881a1570 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f328000cad0 tx=0x7f328000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.564+0000 7f327effd640 1 -- 172.21.15.78:0/3775698740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000ae20 con 0x7f3288108940 2024-04-25T14:08:29.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.564+0000 7f327effd640 1 -- 172.21.15.78:0/3775698740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3280005020 con 0x7f3288108940 2024-04-25T14:08:29.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.564+0000 7f327effd640 1 -- 172.21.15.78:0/3775698740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000f6b0 con 0x7f3288108940 2024-04-25T14:08:29.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.564+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32881a2080 con 0x7f3288108940 2024-04-25T14:08:29.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.564+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32881a2580 con 0x7f3288108940 2024-04-25T14:08:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.566+0000 7f32969a9740 1 Processor -- start 2024-04-25T14:08:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f327effd640 1 -- 172.21.15.78:0/3775698740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f32800051d0 con 0x7f3288108940 2024-04-25T14:08:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f32969a9740 1 -- start start 2024-04-25T14:08:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f32969a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4284a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f32969a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a4284fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f32969a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4285c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f32969a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654a41e9d80 con 0x5654a3ee3390 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f32969a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654a41a38b0 con 0x5654a3ec26b0 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f32969a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5654a41e5d40 con 0x5654a3eba4d0 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f328d094640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4284a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f328d094640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4284a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35156/0 (socket says 172.21.15.78:35156) 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f328d094640 1 -- 172.21.15.78:0/3470345661 learned_addr learned my addr 172.21.15.78:0/3470345661 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f327effd640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f32740afaf0 0x7f32740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f328d094640 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4285c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f328d895640 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a4284fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.567+0000 7f328e096640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f32740afaf0 0x7f32740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f328d094640 1 -- 172.21.15.78:0/3470345661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 msgr2=0x5654a4284fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f328d094640 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a4284fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f327effd640 1 -- 172.21.15.78:0/3775698740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f32800dec80 con 0x7f3288108940 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f328d094640 1 -- 172.21.15.78:0/3470345661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 msgr2=0x5654a4285c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f328d094640 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4285c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f328d094640 1 -- 172.21.15.78:0/3470345661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a4174a40 con 0x5654a3ee3390 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f328d094640 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4284a60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f327800b620 tx=0x7f3278004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba22adab8ce7c231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f328e096640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f32740afaf0 0x7f32740b2000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f32800a6e50 tx=0x7f3280075ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f3273fff640 1 -- 172.21.15.78:0/3470345661 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278003e80 con 0x5654a3ee3390 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f3273fff640 1 -- 172.21.15.78:0/3470345661 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32780054d0 con 0x5654a3ee3390 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.568+0000 7f3273fff640 1 -- 172.21.15.78:0/3470345661 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278004e50 con 0x5654a3ee3390 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 -- 172.21.15.78:0/3470345661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 msgr2=0x5654a4284a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4284a60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f327800b620 tx=0x7f3278004270 comp rx=0 tx=0).stop 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 -- 172.21.15.78:0/3470345661 shutdown_connections 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4285c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a4284fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 --2- 172.21.15.78:0/3470345661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4284a60 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 -- 172.21.15.78:0/3470345661 >> 172.21.15.78:0/3470345661 conn(0x5654a3ed74e0 msgr2=0x5654a4280270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 -- 172.21.15.78:0/3470345661 shutdown_connections 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 -- 172.21.15.78:0/3470345661 wait complete. 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 -- start start 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.569+0000 7f32969a9740 1 Processor -- start 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f32969a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f32969a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a428ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f32969a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4295d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f32969a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654a4286720 con 0x5654a3ee3390 2024-04-25T14:08:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f32969a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654a4170d10 con 0x5654a3ec26b0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f32969a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5654a42868a0 con 0x5654a3eba4d0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f32969a9740 10 client.0 _ll_register_callbacks cb 0x5654a419d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4295d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4295d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35170/0 (socket says 172.21.15.78:35170) 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 -- 172.21.15.78:0/1311562442 learned_addr learned my addr 172.21.15.78:0/1311562442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d895640 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a428ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 -- 172.21.15.78:0/1311562442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 msgr2=0x5654a428e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 -- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 msgr2=0x5654a428ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a428ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.570+0000 7f328d094640 1 -- 172.21.15.78:0/1311562442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a3fc57d0 con 0x5654a3ee3390 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f328d094640 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4295d40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3278005cd0 tx=0x7f32780058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ae768abf18d621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32717fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:29.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32717fa640 1 -- 172.21.15.78:0/1311562442 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278024060 con 0x5654a3ee3390 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32717fa640 1 -- 172.21.15.78:0/1311562442 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3278004de0 con 0x5654a3ee3390 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32780054d0 con 0x5654a3ee3390 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32717fa640 1 -- 172.21.15.78:0/1311562442 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3278004fe0 con 0x5654a3ee3390 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 10 client.18198 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5654a3efb3f0 con 0x5654a3ee3390 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32717fa640 1 -- 172.21.15.78:0/1311562442 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32780054d0 con 0x5654a3ee3390 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32717fa640 1 -- 172.21.15.78:0/1311562442 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f327801e3d0 con 0x5654a3ee3390 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 10 client.18198 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 20 client.18198 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3ea1f60 0x5654a429bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 4 client.18198 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:29.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.571+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654a41f7990 con 0x5654a3ea1f60 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.572+0000 7f328d895640 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3ea1f60 0x5654a429bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.572+0000 7f32717fa640 10 client.18198 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.572+0000 7f328d895640 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3ea1f60 0x5654a429bf30 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe16ffbeb7bc4e0 server_cookie=77b5a9c610652845 in_seq=0 out_seq=0 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32717fa640 1 -- 172.21.15.78:0/1311562442 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654a41f7990 con 0x5654a3ea1f60 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32717fa640 10 client.18198 handle_command_reply: tid=0 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 client.18198 shutdown 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 msgr2=0x5654a4295d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4295d40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3278005cd0 tx=0x7f32780058f0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 shutdown_connections 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a4295d40 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a428ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3ea1f60 0x5654a429bf30 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 --2- 172.21.15.78:0/1311562442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 >> 172.21.15.78:0/1311562442 conn(0x5654a3ed74e0 msgr2=0x5654a4283180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32717fa640 0 client.18198 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 shutdown_connections 2024-04-25T14:08:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 1 -- 172.21.15.78:0/1311562442 wait complete. 2024-04-25T14:08:29.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.573+0000 7f32969a9740 20 client.18198 trim_cache size 0 max 16384 2024-04-25T14:08:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 Processor -- start 2024-04-25T14:08:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 -- start start 2024-04-25T14:08:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4295c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a42961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32881adbf0 con 0x5654a3ee3390 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32881ab4c0 con 0x5654a3ec26b0 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.591+0000 7f32969a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32881a9310 con 0x5654a3eba4d0 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a42961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a42961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35176/0 (socket says 172.21.15.78:35176) 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 -- 172.21.15.78:0/4136890735 learned_addr learned my addr 172.21.15.78:0/4136890735 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d895640 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4295c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 -- 172.21.15.78:0/4136890735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 msgr2=0x5654a428d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 -- 172.21.15.78:0/4136890735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 msgr2=0x5654a4295c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4295c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 -- 172.21.15.78:0/4136890735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a3ebb0f0 con 0x5654a3ee3390 2024-04-25T14:08:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f328d094640 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a42961a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f3278005fd0 tx=0x7f3278003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adb32af2e4d34e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f32737fe640 1 -- 172.21.15.78:0/4136890735 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278033040 con 0x5654a3ee3390 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f32737fe640 1 -- 172.21.15.78:0/4136890735 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3278004de0 con 0x5654a3ee3390 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.592+0000 7f32737fe640 1 -- 172.21.15.78:0/4136890735 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278024040 con 0x5654a3ee3390 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 -- 172.21.15.78:0/4136890735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 msgr2=0x5654a42961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a42961a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f3278005fd0 tx=0x7f3278003f00 comp rx=0 tx=0).stop 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 -- 172.21.15.78:0/4136890735 shutdown_connections 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ee3390 0x5654a42961a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3ec26b0 0x5654a4295c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 --2- 172.21.15.78:0/4136890735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3eba4d0 0x5654a428d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 -- 172.21.15.78:0/4136890735 >> 172.21.15.78:0/4136890735 conn(0x5654a3ed74e0 msgr2=0x5654a428cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 -- 172.21.15.78:0/4136890735 shutdown_connections 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 -- 172.21.15.78:0/4136890735 wait complete. 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.593+0000 7f32969a9740 1 -- start start 2024-04-25T14:08:29.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 1 Processor -- start 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3eba4d0 0x5654a42937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ec26b0 0x5654a4293cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3ee3390 0x5654a4294c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32881a9980 con 0x5654a3ec26b0 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32881aadf0 con 0x5654a3eba4d0 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32881aa050 con 0x5654a3ee3390 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f32969a9740 10 client.0 _ll_register_callbacks cb 0x7f327800b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ec26b0 0x5654a4293cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ec26b0 0x5654a4293cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35190/0 (socket says 172.21.15.78:35190) 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 -- 172.21.15.78:0/3565854171 learned_addr learned my addr 172.21.15.78:0/3565854171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d094640 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3ee3390 0x5654a4294c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d094640 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3eba4d0 0x5654a42937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 -- 172.21.15.78:0/3565854171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3ee3390 msgr2=0x5654a4294c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3ee3390 0x5654a4294c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 -- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3eba4d0 msgr2=0x5654a42937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3eba4d0 0x5654a42937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.594+0000 7f328d895640 1 -- 172.21.15.78:0/3565854171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a41aa4b0 con 0x5654a3ec26b0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f328d895640 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ec26b0 0x5654a4293cf0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f3284031440 tx=0x7f3284005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc4365262874ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32717fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32717fa640 1 -- 172.21.15.78:0/3565854171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284035040 con 0x5654a3ec26b0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32717fa640 1 -- 172.21.15.78:0/3565854171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f32840046c0 con 0x5654a3ec26b0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3278004de0 con 0x5654a3ec26b0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32717fa640 1 -- 172.21.15.78:0/3565854171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3284005590 con 0x5654a3ec26b0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 10 client.18204 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5654a42963a0 con 0x5654a3ec26b0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32717fa640 1 -- 172.21.15.78:0/3565854171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3278004de0 con 0x5654a3ec26b0 2024-04-25T14:08:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32717fa640 1 -- 172.21.15.78:0/3565854171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f32840458d0 con 0x5654a3ec26b0 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 10 client.18204 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 10 client.18204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 20 client.18204 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3e773a0 0x5654a4306600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 4 client.18204 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654a41f1800 con 0x5654a3e773a0 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.595+0000 7f328d094640 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3e773a0 0x5654a4306600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32717fa640 10 client.18204 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f328d094640 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3e773a0 0x5654a4306600 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=280e87db03bb0ff server_cookie=afd727a5ed7e298b in_seq=0 out_seq=0 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32717fa640 1 -- 172.21.15.78:0/3565854171 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5654a41f1800 con 0x5654a3e773a0 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32717fa640 10 client.18204 handle_command_reply: tid=0 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 client.18204 shutdown 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ec26b0 msgr2=0x5654a4293cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ec26b0 0x5654a4293cf0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f3284031440 tx=0x7f3284005f20 comp rx=0 tx=0).stop 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 shutdown_connections 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5654a3e773a0 0x5654a4306600 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5654a3ee3390 0x5654a4294c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32717fa640 0 client.18204 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654a3ec26b0 0x5654a4293cf0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 --2- 172.21.15.78:0/3565854171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654a3eba4d0 0x5654a42937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 >> 172.21.15.78:0/3565854171 conn(0x5654a3ed74e0 msgr2=0x5654a4283070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 shutdown_connections 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.596+0000 7f32969a9740 1 -- 172.21.15.78:0/3565854171 wait complete. 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f32969a9740 20 client.18204 trim_cache size 0 max 16384 2024-04-25T14:08:29.603 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: "2f680f62-af29-4353-abe1-c7c265d2b497": { 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "2f680f62-af29-4353-abe1-c7c265d2b497", 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:08:29.604 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f32740afaf0 msgr2=0x7f32740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f32740afaf0 0x7f32740b2000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f32800a6e50 tx=0x7f3280075ca0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288108940 msgr2=0x7f32881a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288108940 0x7f32881a1570 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f328000cad0 tx=0x7f328000cf90 comp rx=0 tx=0).stop 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 shutdown_connections 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f32740afaf0 0x7f32740b2000 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3288108940 0x7f32881a1570 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3288103c40 0x7f328819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 --2- 172.21.15.78:0/3775698740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3288102a00 0x7f3288075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 >> 172.21.15.78:0/3775698740 conn(0x7f32880fdbf0 msgr2=0x7f3288104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 shutdown_connections 2024-04-25T14:08:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:29.597+0000 7f328faf9640 1 -- 172.21.15.78:0/3775698740 wait complete. 2024-04-25T14:08:29.608 INFO:tasks.cephfs.filesystem:scrub status for tag:2f680f62-af29-4353-abe1-c7c265d2b497 - {'path': '/', 'tag': '2f680f62-af29-4353-abe1-c7c265d2b497', 'options': 'recursive,force'} 2024-04-25T14:08:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 -- 172.21.15.78:0/3040428112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 msgr2=0x7f3dfc103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 --2- 172.21.15.78:0/3040428112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc103720 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009900 tx=0x7f3de402a410 comp rx=0 tx=0).stop 2024-04-25T14:08:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 -- 172.21.15.78:0/3040428112 shutdown_connections 2024-04-25T14:08:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 --2- 172.21.15.78:0/3040428112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc108960 0x7f3dfc10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 --2- 172.21.15.78:0/3040428112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc103c60 0x7f3dfc104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 --2- 172.21.15.78:0/3040428112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc103720 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 -- 172.21.15.78:0/3040428112 >> 172.21.15.78:0/3040428112 conn(0x7f3dfc0fdc10 msgr2=0x7f3dfc1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 -- 172.21.15.78:0/3040428112 shutdown_connections 2024-04-25T14:08:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 -- 172.21.15.78:0/3040428112 wait complete. 2024-04-25T14:08:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.888+0000 7f3e01b27640 1 Processor -- start 2024-04-25T14:08:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3e01b27640 1 -- start start 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3e01b27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3e01b27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc103c60 0x7f3dfc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3e01b27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc108960 0x7f3dfc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3e01b27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc10e520 con 0x7f3dfc102a20 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3e01b27640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc10e3a0 con 0x7f3dfc103c60 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3e01b27640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc10e6a0 con 0x7f3dfc108960 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35210/0 (socket says 172.21.15.78:35210) 2024-04-25T14:08:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/1694426593 learned_addr learned my addr 172.21.15.78:0/1694426593 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc103c60 0x7f3dfc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc108960 0x7f3dfc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/1694426593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc108960 msgr2=0x7f3dfc1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc108960 0x7f3dfc1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/1694426593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc103c60 msgr2=0x7f3dfc1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc103c60 0x7f3dfc1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/1694426593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc1a0cd0 con 0x7f3dfc102a20 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc108960 0x7f3dfc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.889+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc199190 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3de4006010 tx=0x7f3de4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.890+0000 7f3df8ff9640 1 -- 172.21.15.78:0/1694426593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de403f030 con 0x7f3dfc102a20 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.890+0000 7f3df8ff9640 1 -- 172.21.15.78:0/1694426593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de4003aa0 con 0x7f3dfc102a20 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.890+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc1a0f60 con 0x7f3dfc102a20 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.890+0000 7f3df8ff9640 1 -- 172.21.15.78:0/1694426593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4035070 con 0x7f3dfc102a20 2024-04-25T14:08:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.890+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc1a1360 con 0x7f3dfc102a20 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 Processor -- start 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3df8ff9640 1 -- 172.21.15.78:0/1694426593 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3de4042030 con 0x7f3dfc102a20 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 -- start start 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e00c4d0 0x557c3e3d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c3e333460 con 0x557c3e00c4d0 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c3e2ed590 con 0x557c3e02d1b0 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3e089d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c3e32f420 con 0x557c3e0042f0 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3dfaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e00c4d0 0x557c3e3d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.892+0000 7f3dfaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e00c4d0 0x557c3e3d7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35216/0 (socket says 172.21.15.78:35216) 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfaffd640 1 -- 172.21.15.78:0/1436131331 learned_addr learned my addr 172.21.15.78:0/1436131331 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3df8ff9640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3de80afae0 0x7f3de80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfaffd640 1 -- 172.21.15.78:0/1436131331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 msgr2=0x557c3e3d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfaffd640 1 -- 172.21.15.78:0/1436131331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 msgr2=0x557c3e3cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfaffd640 1 -- 172.21.15.78:0/1436131331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3e2fa0d0 con 0x557c3e00c4d0 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3de80afae0 0x7f3de80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3df8ff9640 1 -- 172.21.15.78:0/1694426593 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3de40ff250 con 0x7f3dfc102a20 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e00c4d0 0x557c3e3d7000 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00bc90 tx=0x7f3dec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de7a32eed76fb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3df1ffb640 1 -- 172.21.15.78:0/1436131331 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec015070 con 0x557c3e00c4d0 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3df1ffb640 1 -- 172.21.15.78:0/1436131331 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dec01a910 con 0x557c3e00c4d0 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.893+0000 7f3df1ffb640 1 -- 172.21.15.78:0/1436131331 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0118e0 con 0x557c3e00c4d0 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 -- 172.21.15.78:0/1436131331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e00c4d0 msgr2=0x557c3e3d7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3de80afae0 0x7f3de80b1ff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x557c3e3d0d50 tx=0x7f3df4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e00c4d0 0x557c3e3d7000 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00bc90 tx=0x7f3dec00f730 comp rx=0 tx=0).stop 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 -- 172.21.15.78:0/1436131331 shutdown_connections 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e00c4d0 0x557c3e3d7000 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1436131331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 -- 172.21.15.78:0/1436131331 >> 172.21.15.78:0/1436131331 conn(0x557c3e021300 msgr2=0x557c3e3cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 -- 172.21.15.78:0/1436131331 shutdown_connections 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 -- 172.21.15.78:0/1436131331 wait complete. 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.894+0000 7f3e089d7740 1 -- start start 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 1 Processor -- start 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e00c4d0 0x557c3e3d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c3e3d7a50 con 0x557c3e02d1b0 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c3e2bab60 con 0x557c3e00c4d0 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c3e3d7bd0 con 0x557c3e0042f0 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3e089d7740 10 client.0 _ll_register_callbacks cb 0x557c3e2df2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3e0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35230/0 (socket says 172.21.15.78:35230) 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1065500652 learned_addr learned my addr 172.21.15.78:0/1065500652 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e00c4d0 0x557c3e3d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1065500652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 msgr2=0x557c3e3d9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e00c4d0 msgr2=0x557c3e3d9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e00c4d0 0x557c3e3d9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.895+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1065500652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3e045210 con 0x557c3e02d1b0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3e0d00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3df400cfd0 tx=0x7f3df4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48890332143ed374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3ddaffd640 1 -- 172.21.15.78:0/1065500652 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4012b70 con 0x557c3e02d1b0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3ddaffd640 1 -- 172.21.15.78:0/1065500652 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3df4012d10 con 0x557c3e02d1b0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dec01a910 con 0x557c3e02d1b0 2024-04-25T14:08:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3ddaffd640 1 -- 172.21.15.78:0/1065500652 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3df401b450 con 0x557c3e02d1b0 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 10 client.18216 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557c3e111620 con 0x557c3e02d1b0 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3ddaffd640 1 -- 172.21.15.78:0/1065500652 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dec01a910 con 0x557c3e02d1b0 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3ddaffd640 1 -- 172.21.15.78:0/1065500652 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3df402f050 con 0x557c3e02d1b0 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 10 client.18216 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 10 client.18216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 20 client.18216 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfebd80 0x557c3e3e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 4 client.18216 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c3e341070 con 0x557c3dfebd80 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.896+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfebd80 0x557c3e3e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.897+0000 7f3ddaffd640 10 client.18216 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.897+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfebd80 0x557c3e3e76b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a2357aa442e2bd server_cookie=fb6a87e0b0573e11 in_seq=0 out_seq=0 2024-04-25T14:08:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3ddaffd640 1 -- 172.21.15.78:0/1065500652 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c3e341070 con 0x557c3dfebd80 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3ddaffd640 10 client.18216 handle_command_reply: tid=0 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 client.18216 shutdown 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 msgr2=0x557c3e3e0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3e0d00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3df400cfd0 tx=0x7f3df4009680 comp rx=0 tx=0).stop 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 shutdown_connections 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3e0d00 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e00c4d0 0x557c3e3d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfebd80 0x557c3e3e76b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1065500652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e0042f0 0x557c3e3d9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3ddaffd640 0 client.18216 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 >> 172.21.15.78:0/1065500652 conn(0x557c3e021300 msgr2=0x557c3e3ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 shutdown_connections 2024-04-25T14:08:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 1 -- 172.21.15.78:0/1065500652 wait complete. 2024-04-25T14:08:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.898+0000 7f3e089d7740 20 client.18216 trim_cache size 0 max 16384 2024-04-25T14:08:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 Processor -- start 2024-04-25T14:08:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 -- start start 2024-04-25T14:08:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e0042f0 0x557c3e3cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3e0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1ab920 con 0x557c3e02d1b0 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1a91f0 con 0x557c3e0042f0 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3e089d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1a7040 con 0x557c3e00c4d0 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35234/0 (socket says 172.21.15.78:35234) 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1354222071 learned_addr learned my addr 172.21.15.78:0/1354222071 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3e0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e0042f0 0x557c3e3cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1354222071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 msgr2=0x557c3e3e0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3e0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1354222071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e0042f0 msgr2=0x557c3e3cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e0042f0 0x557c3e3cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfbfff640 1 -- 172.21.15.78:0/1354222071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3e3d46e0 con 0x557c3e02d1b0 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.913+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e0042f0 0x557c3e3cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3ce310 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3df401d470 tx=0x7f3df401d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadea3b55223adc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3df17fa640 1 -- 172.21.15.78:0/1354222071 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4026c80 con 0x557c3e02d1b0 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3df17fa640 1 -- 172.21.15.78:0/1354222071 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df4026e50 con 0x557c3e02d1b0 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3df17fa640 1 -- 172.21.15.78:0/1354222071 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df401bd40 con 0x557c3e02d1b0 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 -- 172.21.15.78:0/1354222071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 msgr2=0x557c3e3ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3ce310 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3df401d470 tx=0x7f3df401d5b0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 -- 172.21.15.78:0/1354222071 shutdown_connections 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e02d1b0 0x557c3e3ce310 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3e0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 --2- 172.21.15.78:0/1354222071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e0042f0 0x557c3e3cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 -- 172.21.15.78:0/1354222071 >> 172.21.15.78:0/1354222071 conn(0x557c3e021300 msgr2=0x557c3e3caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 -- 172.21.15.78:0/1354222071 shutdown_connections 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.914+0000 7f3e089d7740 1 -- 172.21.15.78:0/1354222071 wait complete. 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 -- start start 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 Processor -- start 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e0042f0 0x557c3e3de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3dede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3dfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1a76b0 con 0x557c3e0042f0 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1a8b20 con 0x557c3e02d1b0 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1a7d80 con 0x557c3e00c4d0 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3e089d7740 10 client.0 _ll_register_callbacks cb 0x7f3df4009f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e0042f0 0x557c3e3de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3dfb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e0042f0 0x557c3e3de8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35236/0 (socket says 172.21.15.78:35236) 2024-04-25T14:08:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/2034741466 learned_addr learned my addr 172.21.15.78:0/2034741466 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.915+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3dfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/2034741466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 msgr2=0x557c3e3dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 msgr2=0x557c3e3dfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3dfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3dfb7fe640 1 -- 172.21.15.78:0/2034741466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3e3e1160 con 0x557c3e0042f0 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3dfbfff640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3dfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3dfb7fe640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e0042f0 0x557c3e3de8a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3de40c76b0 tx=0x7f3de4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764ddae20984acec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3ddaffd640 1 -- 172.21.15.78:0/2034741466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4049070 con 0x557c3e0042f0 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3ddaffd640 1 -- 172.21.15.78:0/2034741466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3de4003c50 con 0x557c3e0042f0 2024-04-25T14:08:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df4026e50 con 0x557c3e0042f0 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3ddaffd640 1 -- 172.21.15.78:0/2034741466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3de40c7720 con 0x557c3e0042f0 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 10 client.18222 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557c3e44fc80 con 0x557c3e0042f0 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3ddaffd640 1 -- 172.21.15.78:0/2034741466 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df4026e50 con 0x557c3e0042f0 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3ddaffd640 1 -- 172.21.15.78:0/2034741466 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3de40b00b0 con 0x557c3e0042f0 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 10 client.18222 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 10 client.18222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 20 client.18222 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfc11c0 0x557c3e4526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 4 client.18222 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.916+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c3e33aee0 con 0x557c3dfc11c0 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.917+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfc11c0 0x557c3e4526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.917+0000 7f3ddaffd640 10 client.18222 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.917+0000 7f3dfaffd640 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfc11c0 0x557c3e4526c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1487008304498593 server_cookie=164f87f9a5925328 in_seq=0 out_seq=0 2024-04-25T14:08:29.923 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:29 smithi129 ceph-mon[32569]: pgmap v617: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3ddaffd640 1 -- 172.21.15.78:0/2034741466 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c3e33aee0 con 0x557c3dfc11c0 2024-04-25T14:08:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3ddaffd640 10 client.18222 handle_command_reply: tid=0 2024-04-25T14:08:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 client.18222 shutdown 2024-04-25T14:08:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e0042f0 msgr2=0x557c3e3de8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e0042f0 0x557c3e3de8a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3de40c76b0 tx=0x7f3de4003f90 comp rx=0 tx=0).stop 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 shutdown_connections 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c3e02d1b0 0x557c3e3dfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x557c3dfc11c0 0x557c3e4526c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c3e00c4d0 0x557c3e3dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 --2- 172.21.15.78:0/2034741466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c3e0042f0 0x557c3e3de8a0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3ddaffd640 0 client.18222 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 >> 172.21.15.78:0/2034741466 conn(0x557c3e021300 msgr2=0x557c3e3c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 shutdown_connections 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 1 -- 172.21.15.78:0/2034741466 wait complete. 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.931+0000 7f3e089d7740 20 client.18222 trim_cache size 0 max 16384 2024-04-25T14:08:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:08:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:08:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:08:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3de80afae0 msgr2=0x7f3de80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3de80afae0 0x7f3de80b1ff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x557c3e3d0d50 tx=0x7f3df4004a40 comp rx=0 tx=0).stop 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 msgr2=0x7f3dfc199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc199190 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3de4006010 tx=0x7f3de4031a50 comp rx=0 tx=0).stop 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 shutdown_connections 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc108960 0x7f3dfc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3dfc103c60 0x7f3dfc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3de80afae0 0x7f3de80b1ff0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 --2- 172.21.15.78:0/1694426593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3dfc102a20 0x7f3dfc199190 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 >> 172.21.15.78:0/1694426593 conn(0x7f3dfc0fdc10 msgr2=0x7f3dfc104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 shutdown_connections 2024-04-25T14:08:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:29.932+0000 7f3e01b27640 1 -- 172.21.15.78:0/1694426593 wait complete. 2024-04-25T14:08:29.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:29 smithi078 ceph-mon[29163]: pgmap v617: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:30.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:29 smithi112 ceph-mon[32617]: pgmap v617: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:30.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:30 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:08:30.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:30 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:08:30.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:30 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:08:30.914 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:30 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:08:30.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:30 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:08:30.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:30 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:08:30.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:30 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:08:30.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:30 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:08:31.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:30 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:08:31.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:30 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:08:31.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:30 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:08:31.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:30 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:08:31.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:31 smithi078 ceph-mon[29163]: pgmap v618: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:08:31.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:31 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:08:31.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:31 smithi129 ceph-mon[32569]: pgmap v618: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:08:31.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:31 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:08:32.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:31 smithi112 ceph-mon[32617]: pgmap v618: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:08:32.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:31 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:08:33.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:33 smithi129 ceph-mon[32569]: pgmap v619: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:08:33.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:33 smithi078 ceph-mon[29163]: pgmap v619: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:08:34.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:33 smithi112 ceph-mon[32617]: pgmap v619: 97 pgs: 97 active+clean; 2.3 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:08:35.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:35 smithi129 ceph-mon[32569]: pgmap v620: 97 pgs: 97 active+clean; 2.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:35.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:35 smithi078 ceph-mon[29163]: pgmap v620: 97 pgs: 97 active+clean; 2.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:36.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:35 smithi112 ceph-mon[32617]: pgmap v620: 97 pgs: 97 active+clean; 2.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:08:37.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:37 smithi129 ceph-mon[32569]: pgmap v621: 97 pgs: 97 active+clean; 2.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:37.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:37 smithi078 ceph-mon[29163]: pgmap v621: 97 pgs: 97 active+clean; 2.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:38.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:37 smithi112 ceph-mon[32617]: pgmap v621: 97 pgs: 97 active+clean; 2.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:39.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:39 smithi129 ceph-mon[32569]: pgmap v622: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:08:39.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:39 smithi078 ceph-mon[29163]: pgmap v622: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:08:40.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:39 smithi112 ceph-mon[32617]: pgmap v622: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:08:41.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:41 smithi129 ceph-mon[32569]: pgmap v623: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:41.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:41 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:41.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:41 smithi078 ceph-mon[29163]: pgmap v623: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:41.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:41 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:42.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:41 smithi112 ceph-mon[32617]: pgmap v623: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:42.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:41 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:08:42.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:42 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:42.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:42 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:42.913 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:42 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:08:42.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:42 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:42.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:42 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:42.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:42 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:08:43.015 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:42 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:08:43.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:42 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:08:43.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:42 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2532769106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 msgr2=0x7fcb88104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2532769106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb88104070 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009900 tx=0x7fcb7c02a410 comp rx=0 tx=0).stop 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2532769106 shutdown_connections 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2532769106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88108960 0x7fcb8810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2532769106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb88104070 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2532769106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb88102a20 0x7fcb88103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2532769106 >> 172.21.15.78:0/2532769106 conn(0x7fcb880fdc10 msgr2=0x7fcb881000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2532769106 shutdown_connections 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2532769106 wait complete. 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 Processor -- start 2024-04-25T14:08:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.030+0000 7fcb8e4a7640 1 -- start start 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8e4a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88102a20 0x7fcb88199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8e4a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb881996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8e4a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb88108960 0x7fcb881a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8e4a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8810e5c0 con 0x7fcb88103c60 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8e4a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8810e440 con 0x7fcb88102a20 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8e4a7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8810e740 con 0x7fcb88108960 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb881996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb881996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37528/0 (socket says 172.21.15.78:37528) 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 -- 172.21.15.78:0/2304459809 learned_addr learned my addr 172.21.15.78:0/2304459809 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb88108960 0x7fcb881a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb87fff640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88102a20 0x7fcb88199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 -- 172.21.15.78:0/2304459809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb88108960 msgr2=0x7fcb881a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb88108960 0x7fcb881a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 -- 172.21.15.78:0/2304459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88102a20 msgr2=0x7fcb88199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88102a20 0x7fcb88199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 -- 172.21.15.78:0/2304459809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb881a0cc0 con 0x7fcb88103c60 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb88108960 0x7fcb881a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb877fe640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb881996c0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb857fa640 1 -- 172.21.15.78:0/2304459809 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c03f030 con 0x7fcb88103c60 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb857fa640 1 -- 172.21.15.78:0/2304459809 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7c003cb0 con 0x7fcb88103c60 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb881a0f50 con 0x7fcb88103c60 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.031+0000 7fcb857fa640 1 -- 172.21.15.78:0/2304459809 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c035070 con 0x7fcb88103c60 2024-04-25T14:08:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.032+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb881a13e0 con 0x7fcb88103c60 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.034+0000 7fcb95431740 1 Processor -- start 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.034+0000 7fcb95431740 1 -- start start 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb95431740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bbf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb95431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bbfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb95431740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb95431740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560865b25460 con 0x5608657fe4d0 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb95431740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560865adf590 con 0x56086581f1b0 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb95431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560865b21420 con 0x5608657f62f0 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bbfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb857fa640 1 -- 172.21.15.78:0/2304459809 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcb7c042030 con 0x7fcb88103c60 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bbfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.78:54860/0 (socket says 172.21.15.78:54860) 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 -- 172.21.15.78:0/1336293073 learned_addr learned my addr 172.21.15.78:0/1336293073 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bbf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb877fe640 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 -- 172.21.15.78:0/1336293073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 msgr2=0x560865bbf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bbf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 -- 172.21.15.78:0/1336293073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 msgr2=0x560865bc6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 -- 172.21.15.78:0/1336293073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560865aec0d0 con 0x5608657f62f0 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.035+0000 7fcb87fff640 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bbfef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800bc40 tx=0x7fcb7800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a8134670756b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.036+0000 7fcb857fa640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb700afae0 0x7fcb700b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.036+0000 7fcb6e7fc640 1 -- 172.21.15.78:0/1336293073 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78015070 con 0x5608657f62f0 2024-04-25T14:08:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.036+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb700afae0 0x7fcb700b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.036+0000 7fcb857fa640 1 -- 172.21.15.78:0/2304459809 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fcb7c0ff3c0 con 0x7fcb88103c60 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.036+0000 7fcb6e7fc640 1 -- 172.21.15.78:0/1336293073 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7801a910 con 0x5608657f62f0 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.036+0000 7fcb6e7fc640 1 -- 172.21.15.78:0/1336293073 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78011830 con 0x5608657f62f0 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- 172.21.15.78:0/1336293073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 msgr2=0x560865bbfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb700afae0 0x7fcb700b1ff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x560865bc0b40 tx=0x7fcb8000e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bbfef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800bc40 tx=0x7fcb7800f730 comp rx=0 tx=0).stop 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- 172.21.15.78:0/1336293073 shutdown_connections 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bbfef0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 --2- 172.21.15.78:0/1336293073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bbf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- 172.21.15.78:0/1336293073 >> 172.21.15.78:0/1336293073 conn(0x560865813300 msgr2=0x560865bbb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- 172.21.15.78:0/1336293073 shutdown_connections 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- 172.21.15.78:0/1336293073 wait complete. 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- start start 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 Processor -- start 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bc9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bd0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560865bc7840 con 0x5608657fe4d0 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560865aacb60 con 0x56086581f1b0 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560865bc79c0 con 0x5608657f62f0 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.037+0000 7fcb95431740 10 client.0 _ll_register_callbacks cb 0x560865ad12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.044 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37552/0 (socket says 172.21.15.78:37552) 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 -- 172.21.15.78:0/2562225984 learned_addr learned my addr 172.21.15.78:0/2562225984 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb87fff640 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bc9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 -- 172.21.15.78:0/2562225984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 msgr2=0x560865bc9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bc9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 -- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 msgr2=0x560865bd0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bd0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bd0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 -- 172.21.15.78:0/2562225984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560865837210 con 0x5608657fe4d0 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb877fe640 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc9b90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0fceb0 tx=0x7fcb7c102a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440322a843a548ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb677fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb677fe640 1 -- 172.21.15.78:0/2562225984 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c049030 con 0x5608657fe4d0 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb677fe640 1 -- 172.21.15.78:0/2562225984 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fcb7c0c3b20 con 0x5608657fe4d0 2024-04-25T14:08:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb7801a910 con 0x5608657fe4d0 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb677fe640 1 -- 172.21.15.78:0/2562225984 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcb7c0c3750 con 0x5608657fe4d0 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb95431740 10 client.18234 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560865903620 con 0x5608657fe4d0 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.038+0000 7fcb677fe640 1 -- 172.21.15.78:0/2562225984 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7801a910 con 0x5608657fe4d0 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb677fe640 1 -- 172.21.15.78:0/2562225984 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fcb7c0dcce0 con 0x5608657fe4d0 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb95431740 10 client.18234 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb95431740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb95431740 20 client.18234 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb95431740 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657ddd80 0x560865bd7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb95431740 4 client.18234 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560865b33070 con 0x5608657ddd80 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb87fff640 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657ddd80 0x560865bd7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb87fff640 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657ddd80 0x560865bd7360 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786fc382015bcd5c server_cookie=5d57e4b7f573ab88 in_seq=0 out_seq=0 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.039+0000 7fcb677fe640 10 client.18234 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb677fe640 1 -- 172.21.15.78:0/2562225984 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560865b33070 con 0x5608657ddd80 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb677fe640 10 client.18234 handle_command_reply: tid=0 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 client.18234 shutdown 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 msgr2=0x560865bc9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc9b90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0fceb0 tx=0x7fcb7c102a90 comp rx=0 tx=0).stop 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 shutdown_connections 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bd0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bc9b90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657ddd80 0x560865bd7360 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 --2- 172.21.15.78:0/2562225984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bc9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb677fe640 0 client.18234 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.040+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 >> 172.21.15.78:0/2562225984 conn(0x560865813300 msgr2=0x560865bbe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.041+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 shutdown_connections 2024-04-25T14:08:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.041+0000 7fcb95431740 1 -- 172.21.15.78:0/2562225984 wait complete. 2024-04-25T14:08:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.041+0000 7fcb95431740 20 client.18234 trim_cache size 0 max 16384 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.055+0000 7fcb95431740 1 Processor -- start 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.055+0000 7fcb95431740 1 -- start start 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.055+0000 7fcb95431740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657f62f0 0x560865bc3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.055+0000 7fcb95431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657fe4d0 0x560865bc4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.055+0000 7fcb95431740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bbac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb95431740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881aca20 con 0x5608657f62f0 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb95431740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881aa2f0 con 0x56086581f1b0 2024-04-25T14:08:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb95431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881a8140 con 0x5608657fe4d0 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657f62f0 0x560865bc3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657f62f0 0x560865bc3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37560/0 (socket says 172.21.15.78:37560) 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 -- 172.21.15.78:0/560315604 learned_addr learned my addr 172.21.15.78:0/560315604 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 -- 172.21.15.78:0/560315604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657fe4d0 msgr2=0x560865bc4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 --2- 172.21.15.78:0/560315604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657fe4d0 0x560865bc4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 -- 172.21.15.78:0/560315604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 msgr2=0x560865bbac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 --2- 172.21.15.78:0/560315604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bbac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 -- 172.21.15.78:0/560315604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560865bbf480 con 0x5608657f62f0 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb87fff640 1 --2- 172.21.15.78:0/560315604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657f62f0 0x560865bc3ad0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800bc40 tx=0x7fcb7800b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdfc5af9b0241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb6dffb640 1 -- 172.21.15.78:0/560315604 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78015040 con 0x5608657f62f0 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb6dffb640 1 -- 172.21.15.78:0/560315604 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7802b070 con 0x5608657f62f0 2024-04-25T14:08:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.056+0000 7fcb6dffb640 1 -- 172.21.15.78:0/560315604 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801a4e0 con 0x5608657f62f0 2024-04-25T14:08:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 -- 172.21.15.78:0/560315604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657f62f0 msgr2=0x560865bc3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 --2- 172.21.15.78:0/560315604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657f62f0 0x560865bc3ad0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800bc40 tx=0x7fcb7800b750 comp rx=0 tx=0).stop 2024-04-25T14:08:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 -- 172.21.15.78:0/560315604 shutdown_connections 2024-04-25T14:08:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 --2- 172.21.15.78:0/560315604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bbac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 --2- 172.21.15.78:0/560315604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657fe4d0 0x560865bc4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 --2- 172.21.15.78:0/560315604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657f62f0 0x560865bc3ad0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 -- 172.21.15.78:0/560315604 >> 172.21.15.78:0/560315604 conn(0x560865813300 msgr2=0x560865bbf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 -- 172.21.15.78:0/560315604 shutdown_connections 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 -- 172.21.15.78:0/560315604 wait complete. 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 -- start start 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.057+0000 7fcb95431740 1 Processor -- start 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bcda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bcdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881a87b0 con 0x5608657fe4d0 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881a9c20 con 0x56086581f1b0 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb881a8e80 con 0x5608657f62f0 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 10 client.0 _ll_register_callbacks cb 0x7fcb7c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bcdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bcdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37564/0 (socket says 172.21.15.78:37564) 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 -- 172.21.15.78:0/3775651649 learned_addr learned my addr 172.21.15.78:0/3775651649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 -- 172.21.15.78:0/3775651649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 msgr2=0x560865bcda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb87fff640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bcda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bcda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 -- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 msgr2=0x560865bceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 -- 172.21.15.78:0/3775651649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560865bcf460 con 0x5608657fe4d0 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb87fff640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bcda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb8ca44640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb877fe640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bcdf70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c031440 tx=0x7fcb7c0ff750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23922ce09538822e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb677fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb677fe640 1 -- 172.21.15.78:0/3775651649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c049070 con 0x5608657fe4d0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb677fe640 1 -- 172.21.15.78:0/3775651649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fcb7c102a70 con 0x5608657fe4d0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.058+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb7802b070 con 0x5608657fe4d0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb677fe640 1 -- 172.21.15.78:0/3775651649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcb7c0c7000 con 0x5608657fe4d0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 10 client.18240 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560865c3f840 con 0x5608657fe4d0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb677fe640 1 -- 172.21.15.78:0/3775651649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7802b070 con 0x5608657fe4d0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb677fe640 1 -- 172.21.15.78:0/3775651649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fcb7c0c7af0 con 0x5608657fe4d0 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 10 client.18240 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 10 client.18240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 20 client.18240 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657b31c0 0x560865c42000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 4 client.18240 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:08:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560865b2cee0 con 0x5608657b31c0 2024-04-25T14:08:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.059+0000 7fcb87fff640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657b31c0 0x560865c42000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.060+0000 7fcb677fe640 10 client.18240 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.060+0000 7fcb87fff640 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657b31c0 0x560865c42000 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3fbc9805ee8cd0c server_cookie=f4abb6e4183d5703 in_seq=0 out_seq=0 2024-04-25T14:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb677fe640 1 -- 172.21.15.78:0/3775651649 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560865b2cee0 con 0x5608657b31c0 2024-04-25T14:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb677fe640 10 client.18240 handle_command_reply: tid=0 2024-04-25T14:08:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 client.18240 shutdown 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 msgr2=0x560865bcdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bcdf70 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c031440 tx=0x7fcb7c0ff750 comp rx=0 tx=0).stop 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 shutdown_connections 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56086581f1b0 0x560865bceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608657fe4d0 0x560865bcdf70 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608657b31c0 0x560865c42000 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 --2- 172.21.15.78:0/3775651649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608657f62f0 0x560865bcda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb677fe640 0 client.18240 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.074+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 >> 172.21.15.78:0/3775651649 conn(0x560865813300 msgr2=0x560865bbf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 shutdown_connections 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb95431740 1 -- 172.21.15.78:0/3775651649 wait complete. 2024-04-25T14:08:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb95431740 20 client.18240 trim_cache size 0 max 16384 2024-04-25T14:08:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:08:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:08:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:08:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb700afae0 msgr2=0x7fcb700b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb700afae0 0x7fcb700b1ff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x560865bc0b40 tx=0x7fcb8000e040 comp rx=0 tx=0).stop 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 msgr2=0x7fcb881996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb881996c0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c004340 comp rx=0 tx=0).stop 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 shutdown_connections 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb88108960 0x7fcb881a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fcb700afae0 0x7fcb700b1ff0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb88103c60 0x7fcb881996c0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 --2- 172.21.15.78:0/2304459809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb88102a20 0x7fcb88199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 >> 172.21.15.78:0/2304459809 conn(0x7fcb880fdc10 msgr2=0x7fcb88104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 shutdown_connections 2024-04-25T14:08:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:43.075+0000 7fcb8e4a7640 1 -- 172.21.15.78:0/2304459809 wait complete. 2024-04-25T14:08:44.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:44 smithi129 ceph-mon[32569]: pgmap v624: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:44.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:44 smithi078 ceph-mon[29163]: pgmap v624: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:44.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:44 smithi112 ceph-mon[32617]: pgmap v624: 97 pgs: 97 active+clean; 2.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:08:46.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:46 smithi129 ceph-mon[32569]: pgmap v625: 97 pgs: 97 active+clean; 2.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:08:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:46 smithi078 ceph-mon[29163]: pgmap v625: 97 pgs: 97 active+clean; 2.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:08:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:46 smithi112 ceph-mon[32617]: pgmap v625: 97 pgs: 97 active+clean; 2.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:08:48.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:48 smithi129 ceph-mon[32569]: pgmap v626: 97 pgs: 97 active+clean; 2.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:08:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:48 smithi078 ceph-mon[29163]: pgmap v626: 97 pgs: 97 active+clean; 2.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:08:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:48 smithi112 ceph-mon[32617]: pgmap v626: 97 pgs: 97 active+clean; 2.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:08:50.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:50 smithi129 ceph-mon[32569]: pgmap v627: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:50.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:50 smithi078 ceph-mon[29163]: pgmap v627: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:50 smithi112 ceph-mon[32617]: pgmap v627: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:08:52.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:52 smithi129 ceph-mon[32569]: pgmap v628: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:08:52.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:52 smithi078 ceph-mon[29163]: pgmap v628: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:08:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:52 smithi112 ceph-mon[32617]: pgmap v628: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:08:54.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:54 smithi129 ceph-mon[32569]: pgmap v629: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:08:54.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:54 smithi078 ceph-mon[29163]: pgmap v629: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:08:54.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:54 smithi112 ceph-mon[32617]: pgmap v629: 97 pgs: 97 active+clean; 2.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:08:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 -- 172.21.15.78:0/1579286934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94103b40 msgr2=0x7f5b94104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 --2- 172.21.15.78:0/1579286934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94103b40 0x7f5b94104840 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009900 tx=0x7f5b8402a410 comp rx=0 tx=0).stop 2024-04-25T14:08:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 -- 172.21.15.78:0/1579286934 shutdown_connections 2024-04-25T14:08:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 --2- 172.21.15.78:0/1579286934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b94109a80 0x7f5b9410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 --2- 172.21.15.78:0/1579286934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94104d80 0x7f5b94105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 --2- 172.21.15.78:0/1579286934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94103b40 0x7f5b94104840 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 -- 172.21.15.78:0/1579286934 >> 172.21.15.78:0/1579286934 conn(0x7f5b940fed30 msgr2=0x7f5b941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 -- 172.21.15.78:0/1579286934 shutdown_connections 2024-04-25T14:08:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.187+0000 7f5b9886e640 1 -- 172.21.15.78:0/1579286934 wait complete. 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 Processor -- start 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 -- start start 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94103b40 0x7f5b9419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94104d80 0x7f5b9419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b94109a80 0x7f5b941a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410f6a0 con 0x7f5b94104d80 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410f520 con 0x7f5b94103b40 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b9886e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410f820 con 0x7f5b94109a80 2024-04-25T14:08:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b91d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94104d80 0x7f5b9419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b91d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94104d80 0x7f5b9419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37610/0 (socket says 172.21.15.78:37610) 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.188+0000 7f5b91d9b640 1 -- 172.21.15.78:0/2531763312 learned_addr learned my addr 172.21.15.78:0/2531763312 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b94109a80 0x7f5b941a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b9259c640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94103b40 0x7f5b9419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b91d9b640 1 -- 172.21.15.78:0/2531763312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b94109a80 msgr2=0x7f5b941a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b94109a80 0x7f5b941a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b91d9b640 1 -- 172.21.15.78:0/2531763312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94103b40 msgr2=0x7f5b9419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94103b40 0x7f5b9419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b91d9b640 1 -- 172.21.15.78:0/2531763312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b941a1e10 con 0x7f5b94104d80 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94104d80 0x7f5b9419a810 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b4a0 tx=0x7f5b8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b837fe640 1 -- 172.21.15.78:0/2531763312 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88014070 con 0x7f5b94104d80 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b941a2100 con 0x7f5b94104d80 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b837fe640 1 -- 172.21.15.78:0/2531763312 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b88010ac0 con 0x7f5b94104d80 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b837fe640 1 -- 172.21.15.78:0/2531763312 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88017c30 con 0x7f5b94104d80 2024-04-25T14:08:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.189+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b941a2720 con 0x7f5b94104d80 2024-04-25T14:08:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.192+0000 7f5b9f800740 1 Processor -- start 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.192+0000 7f5b9f800740 1 -- start start 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b837fe640 1 -- 172.21.15.78:0/2531763312 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5b88017070 con 0x7f5b94104d80 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5bddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9f800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7dc51c460 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9f800740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7dc4d6590 con 0x55a7dc1f54d0 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9f800740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a7dc518420 con 0x55a7dc2161b0 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37616/0 (socket says 172.21.15.78:37616) 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 -- 172.21.15.78:0/4132004693 learned_addr learned my addr 172.21.15.78:0/4132004693 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5bddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 -- 172.21.15.78:0/4132004693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 msgr2=0x55a7dc5b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 -- 172.21.15.78:0/4132004693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1f54d0 msgr2=0x55a7dc5bddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5bddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 -- 172.21.15.78:0/4132004693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dc4e30d0 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b9259c640 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5b6ef0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009230 tx=0x7f5b84031c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2266b75c8185fcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.193+0000 7f5b837fe640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5b7c0afae0 0x7f5b7c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.194+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5b7c0afae0 0x7f5b7c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.194+0000 7f5b80ff9640 1 -- 172.21.15.78:0/4132004693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84035030 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.194+0000 7f5b837fe640 1 -- 172.21.15.78:0/2531763312 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5b880de280 con 0x7f5b94104d80 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.194+0000 7f5b80ff9640 1 -- 172.21.15.78:0/4132004693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b84040070 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.194+0000 7f5b80ff9640 1 -- 172.21.15.78:0/4132004693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84003bb0 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- 172.21.15.78:0/4132004693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 msgr2=0x55a7dc5b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5b6ef0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009230 tx=0x7f5b84031c50 comp rx=0 tx=0).stop 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5b7c0afae0 0x7f5b7c0b1ff0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55a7dc5b7b40 tx=0x7f5b8c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- 172.21.15.78:0/4132004693 shutdown_connections 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5bddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5b6ef0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 --2- 172.21.15.78:0/4132004693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- 172.21.15.78:0/4132004693 >> 172.21.15.78:0/4132004693 conn(0x55a7dc20a300 msgr2=0x55a7dc5b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- 172.21.15.78:0/4132004693 shutdown_connections 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- 172.21.15.78:0/4132004693 wait complete. 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- start start 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 Processor -- start 2024-04-25T14:08:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7dc5be7e0 con 0x55a7dc1f54d0 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a7dc4a3b60 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a7dc5be960 con 0x55a7dc2161b0 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b9f800740 10 client.0 _ll_register_callbacks cb 0x55a7dc4c82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.195+0000 7f5b91d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37622/0 (socket says 172.21.15.78:37622) 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 -- 172.21.15.78:0/67805781 learned_addr learned my addr 172.21.15.78:0/67805781 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b9259c640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 -- 172.21.15.78:0/67805781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 msgr2=0x55a7dc5c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 -- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1ed2f0 msgr2=0x55a7dc5c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 -- 172.21.15.78:0/67805781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dc22e210 con 0x55a7dc1f54d0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b9259c640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c0b30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f5b880a2880 tx=0x7f5b880a6150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bea7ce3afb23b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b79ffb640 1 -- 172.21.15.78:0/67805781 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8801e070 con 0x55a7dc1f54d0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b79ffb640 1 -- 172.21.15.78:0/67805781 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5b880a2a90 con 0x55a7dc1f54d0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b84040070 con 0x55a7dc1f54d0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b79ffb640 1 -- 172.21.15.78:0/67805781 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5b880a6310 con 0x55a7dc1f54d0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b9f800740 10 client.18252 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a7dc2fa620 con 0x55a7dc1f54d0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.196+0000 7f5b79ffb640 1 -- 172.21.15.78:0/67805781 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b84040070 con 0x55a7dc1f54d0 2024-04-25T14:08:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b79ffb640 1 -- 172.21.15.78:0/67805781 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5b8808f0d0 con 0x55a7dc1f54d0 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b9f800740 10 client.18252 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b9f800740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b9f800740 20 client.18252 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b9f800740 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc1d4d80 0x55a7dc5ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b9f800740 4 client.18252 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7dc52a070 con 0x55a7dc1d4d80 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.197+0000 7f5b9259c640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc1d4d80 0x55a7dc5ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b79ffb640 10 client.18252 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9259c640 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc1d4d80 0x55a7dc5ce360 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1927130a45b991 server_cookie=8e3d4efe7d276e83 in_seq=0 out_seq=0 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b79ffb640 1 -- 172.21.15.78:0/67805781 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7dc52a070 con 0x55a7dc1d4d80 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b79ffb640 10 client.18252 handle_command_reply: tid=0 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 client.18252 shutdown 2024-04-25T14:08:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 msgr2=0x55a7dc5c0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c0b30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f5b880a2880 tx=0x7f5b880a6150 comp rx=0 tx=0).stop 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 shutdown_connections 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc1d4d80 0x55a7dc5ce360 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc2161b0 0x55a7dc5c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b79ffb640 0 client.18252 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c0b30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 --2- 172.21.15.78:0/67805781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.198+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 >> 172.21.15.78:0/67805781 conn(0x55a7dc20a300 msgr2=0x55a7dc5b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.199+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 shutdown_connections 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.199+0000 7f5b9f800740 1 -- 172.21.15.78:0/67805781 wait complete. 2024-04-25T14:08:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.199+0000 7f5b9f800740 20 client.18252 trim_cache size 0 max 16384 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 Processor -- start 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 -- start start 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 0x55a7dc5baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941adc90 con 0x55a7dc1f54d0 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941ab560 con 0x55a7dc20a300 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b9f800740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941a93b0 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37632/0 (socket says 172.21.15.78:37632) 2024-04-25T14:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 -- 172.21.15.78:0/426797276 learned_addr learned my addr 172.21.15.78:0/426797276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 -- 172.21.15.78:0/426797276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1ed2f0 msgr2=0x55a7dc5bb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5bb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 -- 172.21.15.78:0/426797276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 msgr2=0x55a7dc5baad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 0x55a7dc5baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.214+0000 7f5b92d9d640 1 -- 172.21.15.78:0/426797276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dc5b6480 con 0x55a7dc1f54d0 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5b1c50 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0092f0 tx=0x7f5b8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fc5ab60b8361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b7bfff640 1 -- 172.21.15.78:0/426797276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c004530 con 0x55a7dc1f54d0 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b7bfff640 1 -- 172.21.15.78:0/426797276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b8c026030 con 0x55a7dc1f54d0 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b7bfff640 1 -- 172.21.15.78:0/426797276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c022650 con 0x55a7dc1f54d0 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 -- 172.21.15.78:0/426797276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 msgr2=0x55a7dc5b1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5b1c50 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0092f0 tx=0x7f5b8c016000 comp rx=0 tx=0).stop 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 -- 172.21.15.78:0/426797276 shutdown_connections 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5b1c50 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 --2- 172.21.15.78:0/426797276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 0x55a7dc5baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 -- 172.21.15.78:0/426797276 >> 172.21.15.78:0/426797276 conn(0x55a7dc5ce490 msgr2=0x55a7dc5b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 -- 172.21.15.78:0/426797276 shutdown_connections 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.215+0000 7f5b9f800740 1 -- 172.21.15.78:0/426797276 wait complete. 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 -- start start 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 Processor -- start 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 0x55a7dc5c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941a9a20 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941aae90 con 0x55a7dc20a300 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941aa0f0 con 0x55a7dc1f54d0 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9f800740 10 client.0 _ll_register_callbacks cb 0x7f5b88017550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37638/0 (socket says 172.21.15.78:37638) 2024-04-25T14:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b9259c640 1 -- 172.21.15.78:0/3576565334 learned_addr learned my addr 172.21.15.78:0/3576565334 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.216+0000 7f5b92d9d640 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 0x55a7dc5c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9259c640 1 -- 172.21.15.78:0/3576565334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1f54d0 msgr2=0x55a7dc5c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9259c640 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9259c640 1 -- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 msgr2=0x55a7dc5c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9259c640 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 0x55a7dc5c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9259c640 1 -- 172.21.15.78:0/3576565334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dc5c6470 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9259c640 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c4a40 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005ab0 tx=0x7f5b84005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aab06ddd9f4453d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b79ffb640 1 -- 172.21.15.78:0/3576565334 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84056060 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b79ffb640 1 -- 172.21.15.78:0/3576565334 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5b84004240 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b8c026030 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b79ffb640 1 -- 172.21.15.78:0/3576565334 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5b84035040 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9f800740 10 client.18258 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a7dc636830 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.217+0000 7f5b79ffb640 1 -- 172.21.15.78:0/3576565334 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b8c026030 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b79ffb640 1 -- 172.21.15.78:0/3576565334 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5b8404f470 con 0x55a7dc1ed2f0 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b9f800740 10 client.18258 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b9f800740 10 client.18258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b9f800740 20 client.18258 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b9f800740 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc2161b0 0x55a7dc638db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b9f800740 4 client.18258 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7dc523ee0 con 0x55a7dc2161b0 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc2161b0 0x55a7dc638db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b91d9b640 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc2161b0 0x55a7dc638db0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa92826f471b3d3d server_cookie=28f567ad6afe6df7 in_seq=0 out_seq=0 2024-04-25T14:08:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.218+0000 7f5b79ffb640 10 client.18258 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b79ffb640 1 -- 172.21.15.78:0/3576565334 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7dc523ee0 con 0x55a7dc2161b0 2024-04-25T14:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b79ffb640 10 client.18258 handle_command_reply: tid=0 2024-04-25T14:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 client.18258 shutdown 2024-04-25T14:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 msgr2=0x55a7dc5c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c4a40 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005ab0 tx=0x7f5b84005e10 comp rx=0 tx=0).stop 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 shutdown_connections 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a7dc20a300 0x55a7dc5c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a7dc2161b0 0x55a7dc638db0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7dc1f54d0 0x55a7dc5c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b79ffb640 0 client.18258 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 --2- 172.21.15.78:0/3576565334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7dc1ed2f0 0x55a7dc5c4a40 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 >> 172.21.15.78:0/3576565334 conn(0x55a7dc5ce490 msgr2=0x55a7dc5b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 shutdown_connections 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 1 -- 172.21.15.78:0/3576565334 wait complete. 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.236+0000 7f5b9f800740 20 client.18258 trim_cache size 0 max 16384 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5b7c0afae0 msgr2=0x7f5b7c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5b7c0afae0 0x7f5b7c0b1ff0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55a7dc5b7b40 tx=0x7f5b8c009600 comp rx=0 tx=0).stop 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94104d80 msgr2=0x7f5b9419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94104d80 0x7f5b9419a810 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b4a0 tx=0x7f5b8800b960 comp rx=0 tx=0).stop 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 shutdown_connections 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b94109a80 0x7f5b941a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5b7c0afae0 0x7f5b7c0b1ff0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b94104d80 0x7f5b9419a810 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 --2- 172.21.15.78:0/2531763312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b94103b40 0x7f5b9419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 >> 172.21.15.78:0/2531763312 conn(0x7f5b940fed30 msgr2=0x7f5b94105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 shutdown_connections 2024-04-25T14:08:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:08:56.237+0000 7f5b9886e640 1 -- 172.21.15.78:0/2531763312 wait complete. 2024-04-25T14:08:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:56 smithi112 ceph-mon[32617]: pgmap v630: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:08:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:56 smithi129 ceph-mon[32569]: pgmap v630: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:08:56.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:56 smithi078 ceph-mon[29163]: pgmap v630: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:08:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:08:58 smithi112 ceph-mon[32617]: pgmap v631: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:08:58 smithi129 ceph-mon[32569]: pgmap v631: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:58.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:08:58 smithi078 ceph-mon[29163]: pgmap v631: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:08:59.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:08:59.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:08:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.736+0000 7fa847334640 1 -- 172.21.15.78:0/1171167674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840104d30 msgr2=0x7fa840105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 --2- 172.21.15.78:0/1171167674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840104d30 0x7fa840105140 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa830009900 tx=0x7fa83002a410 comp rx=0 tx=0).stop 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 -- 172.21.15.78:0/1171167674 shutdown_connections 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 --2- 172.21.15.78:0/1171167674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa840109a30 0x7fa84010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 --2- 172.21.15.78:0/1171167674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840104d30 0x7fa840105140 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 --2- 172.21.15.78:0/1171167674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa840103af0 0x7fa8401047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 -- 172.21.15.78:0/1171167674 >> 172.21.15.78:0/1171167674 conn(0x7fa8400fece0 msgr2=0x7fa840101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 -- 172.21.15.78:0/1171167674 shutdown_connections 2024-04-25T14:08:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 -- 172.21.15.78:0/1171167674 wait complete. 2024-04-25T14:08:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 Processor -- start 2024-04-25T14:08:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.737+0000 7fa847334640 1 -- start start 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840103af0 0x7fa84019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa840104d30 0x7fa84019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa840109a30 0x7fa8401a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa84010f620 con 0x7fa840103af0 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa84010f4a0 con 0x7fa840104d30 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa84010f7a0 con 0x7fa840109a30 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840103af0 0x7fa84019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840103af0 0x7fa84019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46380/0 (socket says 172.21.15.78:46380) 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 -- 172.21.15.78:0/4083595465 learned_addr learned my addr 172.21.15.78:0/4083595465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8448cf640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa840104d30 0x7fa84019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 -- 172.21.15.78:0/4083595465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa840109a30 msgr2=0x7fa8401a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8458d1640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa840109a30 0x7fa8401a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa840109a30 0x7fa8401a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 -- 172.21.15.78:0/4083595465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa840104d30 msgr2=0x7fa84019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa840104d30 0x7fa84019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 -- 172.21.15.78:0/4083595465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8401a1c70 con 0x7fa840103af0 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8458d1640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa840109a30 0x7fa8401a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8448cf640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa840104d30 0x7fa84019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8450d0640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840103af0 0x7fa84019a1e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c970 tx=0x7fa83c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8367fc640 1 -- 172.21.15.78:0/4083595465 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007c70 con 0x7fa840103af0 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8367fc640 1 -- 172.21.15.78:0/4083595465 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa83c007e10 con 0x7fa840103af0 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8401a1f60 con 0x7fa840103af0 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa8367fc640 1 -- 172.21.15.78:0/4083595465 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0187b0 con 0x7fa840103af0 2024-04-25T14:08:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.738+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8401a23c0 con 0x7fa840103af0 2024-04-25T14:08:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa8367fc640 1 -- 172.21.15.78:0/4083595465 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fa83c00fab0 con 0x7fa840103af0 2024-04-25T14:08:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 Processor -- start 2024-04-25T14:08:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 -- start start 2024-04-25T14:08:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f661390 0x56145fa02a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa02fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6406b0 0x56145fa03c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56145f967d80 con 0x56145f661390 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56145f9218b0 con 0x56145f6406b0 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56145f963d40 con 0x56145f6384d0 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa8458d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f661390 0x56145fa02a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa8458d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f661390 0x56145fa02a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46388/0 (socket says 172.21.15.78:46388) 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa8458d1640 1 -- 172.21.15.78:0/3722041051 learned_addr learned my addr 172.21.15.78:0/3722041051 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.742+0000 7fa8448cf640 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6406b0 0x56145fa03c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8450d0640 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa02fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8367fc640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa82c0afaf0 0x7fa82c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 -- 172.21.15.78:0/3722041051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 msgr2=0x56145fa02fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa02fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 -- 172.21.15.78:0/3722041051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6406b0 msgr2=0x56145fa03c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6406b0 0x56145fa03c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 -- 172.21.15.78:0/3722041051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145f8f2a40 con 0x56145f661390 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa82c0afaf0 0x7fa82c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8367fc640 1 -- 172.21.15.78:0/4083595465 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa83c0dece0 con 0x7fa840103af0 2024-04-25T14:08:59.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f661390 0x56145fa02a60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa83800f390 tx=0x7fa83800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cead8c540cfcf0c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa82b7fe640 1 -- 172.21.15.78:0/3722041051 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83801a610 con 0x56145f661390 2024-04-25T14:08:59.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa82b7fe640 1 -- 172.21.15.78:0/3722041051 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa83801ac60 con 0x56145f661390 2024-04-25T14:08:59.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa82b7fe640 1 -- 172.21.15.78:0/3722041051 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838010af0 con 0x56145f661390 2024-04-25T14:08:59.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.743+0000 7fa8458d1640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa82c0afaf0 0x7fa82c0b2000 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa838020000 tx=0x7fa83801d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3722041051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f661390 msgr2=0x56145fa02a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f661390 0x56145fa02a60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa83800f390 tx=0x7fa83800f850 comp rx=0 tx=0).stop 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3722041051 shutdown_connections 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6406b0 0x56145fa03c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa02fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3722041051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f661390 0x56145fa02a60 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3722041051 >> 172.21.15.78:0/3722041051 conn(0x56145f6554e0 msgr2=0x56145f9fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3722041051 shutdown_connections 2024-04-25T14:08:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.744+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3722041051 wait complete. 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 -- start start 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 Processor -- start 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6384d0 0x56145fa0c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6406b0 0x56145fa0cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa13d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56145fa04780 con 0x56145f6384d0 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56145f8eed10 con 0x56145f661390 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56145fa04900 con 0x56145f6406b0 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa84e1e4740 10 client.0 _ll_register_callbacks cb 0x56145f91b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa8450d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6384d0 0x56145fa0c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa8450d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6384d0 0x56145fa0c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46398/0 (socket says 172.21.15.78:46398) 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa8450d0640 1 -- 172.21.15.78:0/541888640 learned_addr learned my addr 172.21.15.78:0/541888640 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa8458d1640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa13d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.745+0000 7fa8448cf640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6406b0 0x56145fa0cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa8450d0640 1 -- 172.21.15.78:0/541888640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6406b0 msgr2=0x56145fa0cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa8450d0640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6406b0 0x56145fa0cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa8450d0640 1 -- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 msgr2=0x56145fa13d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa8450d0640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa13d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa8450d0640 1 -- 172.21.15.78:0/541888640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145f7437d0 con 0x56145f6384d0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa8448cf640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6406b0 0x56145fa0cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa8450d0640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6384d0 0x56145fa0c960 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa83c020000 tx=0x7fa83c0a77d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4719fa0cda2aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa828ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa828ff9640 1 -- 172.21.15.78:0/541888640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0a6920 con 0x56145f6384d0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa828ff9640 1 -- 172.21.15.78:0/541888640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa83c0a2d30 con 0x56145f6384d0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa83801ac60 con 0x56145f6384d0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa828ff9640 1 -- 172.21.15.78:0/541888640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa83c0a6cc0 con 0x56145f6384d0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa84e1e4740 10 client.18270 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56145f6793f0 con 0x56145f6384d0 2024-04-25T14:08:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa828ff9640 1 -- 172.21.15.78:0/541888640 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa83801ac60 con 0x56145f6384d0 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa828ff9640 1 -- 172.21.15.78:0/541888640 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa83c020030 con 0x56145f6384d0 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa84e1e4740 10 client.18270 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa84e1e4740 10 client.18270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa84e1e4740 20 client.18270 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.746+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f61ff60 0x56145fa1a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.747+0000 7fa84e1e4740 4 client.18270 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.747+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145f975990 con 0x56145f61ff60 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.747+0000 7fa8448cf640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f61ff60 0x56145fa1a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.747+0000 7fa828ff9640 10 client.18270 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.747+0000 7fa8448cf640 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f61ff60 0x56145fa1a280 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0639485d20e1b server_cookie=65e74d6ad17d1b0 in_seq=0 out_seq=0 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa828ff9640 1 -- 172.21.15.78:0/541888640 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56145f975990 con 0x56145f61ff60 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa828ff9640 10 client.18270 handle_command_reply: tid=0 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 client.18270 shutdown 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6384d0 msgr2=0x56145fa0c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6384d0 0x56145fa0c960 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fa83c020000 tx=0x7fa83c0a77d0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 shutdown_connections 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa13d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f61ff60 0x56145fa1a280 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6406b0 0x56145fa0cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/541888640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6384d0 0x56145fa0c960 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa828ff9640 0 client.18270 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.748+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 >> 172.21.15.78:0/541888640 conn(0x56145f6554e0 msgr2=0x56145f9fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.749+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 shutdown_connections 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.749+0000 7fa84e1e4740 1 -- 172.21.15.78:0/541888640 wait complete. 2024-04-25T14:08:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.749+0000 7fa84e1e4740 20 client.18270 trim_cache size 0 max 16384 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 Processor -- start 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 -- start start 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa0b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa1ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401adb30 con 0x56145f6406b0 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ab400 con 0x56145f661390 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401a9250 con 0x56145f6384d0 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.766+0000 7fa8448cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa1ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46408/0 (socket says 172.21.15.78:46408) 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 -- 172.21.15.78:0/3847997383 learned_addr learned my addr 172.21.15.78:0/3847997383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8450d0640 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa0b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8458d1640 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa1ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 -- 172.21.15.78:0/3847997383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 msgr2=0x56145fa0b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa0b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 -- 172.21.15.78:0/3847997383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 msgr2=0x56145fa1ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa1ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 -- 172.21.15.78:0/3847997383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145f6390f0 con 0x56145f6406b0 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa8448cf640 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa1ea00 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa83002a930 tx=0x7fa830005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee214bffdf8906b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa82affd640 1 -- 172.21.15.78:0/3847997383 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830035040 con 0x56145f6406b0 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa82affd640 1 -- 172.21.15.78:0/3847997383 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa830049070 con 0x56145f6406b0 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.767+0000 7fa82affd640 1 -- 172.21.15.78:0/3847997383 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004920 con 0x56145f6406b0 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3847997383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 msgr2=0x56145fa1ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa1ea00 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa83002a930 tx=0x7fa830005a20 comp rx=0 tx=0).stop 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3847997383 shutdown_connections 2024-04-25T14:08:59.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f661390 0x56145fa1ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa1ea00 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/3847997383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f6384d0 0x56145fa0b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3847997383 >> 172.21.15.78:0/3847997383 conn(0x56145f6554e0 msgr2=0x56145fa0af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3847997383 shutdown_connections 2024-04-25T14:08:59.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 -- 172.21.15.78:0/3847997383 wait complete. 2024-04-25T14:08:59.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 -- start start 2024-04-25T14:08:59.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.768+0000 7fa84e1e4740 1 Processor -- start 2024-04-25T14:08:59.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6384d0 0x56145fa10770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa10cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa84e1e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f661390 0x56145fa11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401a98c0 con 0x56145f6406b0 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401aad30 con 0x56145f6384d0 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa84e1e4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401a9f90 con 0x56145f661390 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa84e1e4740 10 client.0 _ll_register_callbacks cb 0x7fa83c0e22d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa10cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa10cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46412/0 (socket says 172.21.15.78:46412) 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 -- 172.21.15.78:0/2250625046 learned_addr learned my addr 172.21.15.78:0/2250625046 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8450d0640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6384d0 0x56145fa10770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 -- 172.21.15.78:0/2250625046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f661390 msgr2=0x56145fa11be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8458d1640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f661390 0x56145fa11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f661390 0x56145fa11be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 -- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6384d0 msgr2=0x56145fa10770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6384d0 0x56145fa10770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 -- 172.21.15.78:0/2250625046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145f9284b0 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8450d0640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6384d0 0x56145fa10770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa8448cf640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa10cb0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa830006010 tx=0x7fa8300328f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741163f3f32e7fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa828ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.769+0000 7fa828ff9640 1 -- 172.21.15.78:0/2250625046 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830035050 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa828ff9640 1 -- 172.21.15.78:0/2250625046 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fa830032040 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa830049070 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa828ff9640 1 -- 172.21.15.78:0/2250625046 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa830032240 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 10 client.18276 fetch_fsmap learned FSMap version 22 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56145fa13290 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa828ff9640 1 -- 172.21.15.78:0/2250625046 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa830049070 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa828ff9640 1 -- 172.21.15.78:0/2250625046 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa830044330 con 0x56145f6406b0 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 10 client.18276 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:08:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 20 client.18276 populate_metadata read hostname 'smithi078' 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f5f53a0 0x56145fa848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 4 client.18276 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56145f96f800 con 0x56145f5f53a0 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.770+0000 7fa8450d0640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f5f53a0 0x56145fa848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa828ff9640 10 client.18276 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa8450d0640 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f5f53a0 0x56145fa848c0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04f2e6a7ba7b2eb server_cookie=71279847f8636b5 in_seq=0 out_seq=0 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa828ff9640 1 -- 172.21.15.78:0/2250625046 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56145f96f800 con 0x56145f5f53a0 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa828ff9640 10 client.18276 handle_command_reply: tid=0 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 client.18276 shutdown 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 msgr2=0x56145fa10cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa10cb0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa830006010 tx=0x7fa8300328f0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 shutdown_connections 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56145f5f53a0 0x56145fa848c0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56145f661390 0x56145fa11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56145f6406b0 0x56145fa10cb0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa828ff9640 0 client.18276 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 --2- 172.21.15.78:0/2250625046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56145f6384d0 0x56145fa10770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 >> 172.21.15.78:0/2250625046 conn(0x56145f6554e0 msgr2=0x56145fa010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 shutdown_connections 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 1 -- 172.21.15.78:0/2250625046 wait complete. 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.771+0000 7fa84e1e4740 20 client.18276 trim_cache size 0 max 16384 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:08:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa82c0afaf0 msgr2=0x7fa82c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa82c0afaf0 0x7fa82c0b2000 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa838020000 tx=0x7fa83801d000 comp rx=0 tx=0).stop 2024-04-25T14:08:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840103af0 msgr2=0x7fa84019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840103af0 0x7fa84019a1e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c970 tx=0x7fa83c00ce30 comp rx=0 tx=0).stop 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 shutdown_connections 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa840109a30 0x7fa8401a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa840104d30 0x7fa84019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fa82c0afaf0 0x7fa82c0b2000 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 --2- 172.21.15.78:0/4083595465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa840103af0 0x7fa84019a1e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 >> 172.21.15.78:0/4083595465 conn(0x7fa8400fece0 msgr2=0x7fa840105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 shutdown_connections 2024-04-25T14:08:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.772+0000 7fa847334640 1 -- 172.21.15.78:0/4083595465 wait complete. 2024-04-25T14:08:59.782 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:08:59.783 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:08:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 -- 172.21.15.78:0/3256013711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec104d50 msgr2=0x7f2eec105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/3256013711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec104d50 0x7f2eec105160 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009900 tx=0x7f2ee002a410 comp rx=0 tx=0).stop 2024-04-25T14:08:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 -- 172.21.15.78:0/3256013711 shutdown_connections 2024-04-25T14:08:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/3256013711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eec109a50 0x7f2eec10be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/3256013711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec104d50 0x7f2eec105160 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/3256013711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eec103b10 0x7f2eec104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 -- 172.21.15.78:0/3256013711 >> 172.21.15.78:0/3256013711 conn(0x7f2eec0fed40 msgr2=0x7f2eec1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:08:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 -- 172.21.15.78:0/3256013711 shutdown_connections 2024-04-25T14:08:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.890+0000 7f2ef1cee640 1 -- 172.21.15.78:0/3256013711 wait complete. 2024-04-25T14:08:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 Processor -- start 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 -- start start 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec103b10 0x7f2eec077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eec104d50 0x7f2eec076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eec109a50 0x7f2eec076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f640 con 0x7f2eec103b10 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f4c0 con 0x7f2eec104d50 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2ef1cee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f7c0 con 0x7f2eec109a50 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec103b10 0x7f2eec077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec103b10 0x7f2eec077960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46422/0 (socket says 172.21.15.78:46422) 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2eeb7fe640 1 -- 172.21.15.78:0/1370027723 learned_addr learned my addr 172.21.15.78:0/1370027723 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:08:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.891+0000 7f2eebfff640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eec109a50 0x7f2eec076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeb7fe640 1 -- 172.21.15.78:0/1370027723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eec109a50 msgr2=0x7f2eec076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeaffd640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eec104d50 0x7f2eec076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeb7fe640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eec109a50 0x7f2eec076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeb7fe640 1 -- 172.21.15.78:0/1370027723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eec104d50 msgr2=0x7f2eec076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeb7fe640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eec104d50 0x7f2eec076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeb7fe640 1 -- 172.21.15.78:0/1370027723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec076c70 con 0x7f2eec103b10 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeaffd640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eec104d50 0x7f2eec076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2eeb7fe640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec103b10 0x7f2eec077960 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00d740 tx=0x7f2edc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2ee8ff9640 1 -- 172.21.15.78:0/1370027723 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc017070 con 0x7f2eec103b10 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2ee8ff9640 1 -- 172.21.15.78:0/1370027723 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2edc00de50 con 0x7f2eec103b10 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec1a4300 con 0x7f2eec103b10 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2ee8ff9640 1 -- 172.21.15.78:0/1370027723 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc003c20 con 0x7f2eec103b10 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.892+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec1a47c0 con 0x7f2eec103b10 2024-04-25T14:08:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.893+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec108860 con 0x7f2eec103b10 2024-04-25T14:08:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.894+0000 7f2ee8ff9640 1 -- 172.21.15.78:0/1370027723 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2edc003dc0 con 0x7f2eec103b10 2024-04-25T14:08:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.895+0000 7f2ee8ff9640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2ed40afaf0 0x7f2ed40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:08:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.895+0000 7f2eeaffd640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2ed40afaf0 0x7f2ed40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:08:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.895+0000 7f2ee8ff9640 1 -- 172.21.15.78:0/1370027723 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2edc0de6f0 con 0x7f2eec103b10 2024-04-25T14:08:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.896+0000 7f2eeaffd640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2ed40afaf0 0x7f2ed40b2000 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006010 tx=0x7f2ee0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:08:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:08:59.900+0000 7f2ee8ff9640 1 -- 172.21.15.78:0/1370027723 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2edc0a72c0 con 0x7f2eec103b10 2024-04-25T14:09:00.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.113+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eec02db50 con 0x7f2eec103b10 2024-04-25T14:09:00.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.114+0000 7f2ee8ff9640 1 -- 172.21.15.78:0/1370027723 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f2eec02db50 con 0x7f2eec103b10 2024-04-25T14:09:00.114 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:09:00.114 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:09:00.114 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2ed40afaf0 msgr2=0x7f2ed40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2ed40afaf0 0x7f2ed40b2000 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006010 tx=0x7f2ee0000bc0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec103b10 msgr2=0x7f2eec077960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec103b10 0x7f2eec077960 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00d740 tx=0x7f2edc00dc00 comp rx=0 tx=0).stop 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 shutdown_connections 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2eec109a50 0x7f2eec076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eec104d50 0x7f2eec076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2ed40afaf0 0x7f2ed40b2000 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 --2- 172.21.15.78:0/1370027723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2eec103b10 0x7f2eec077960 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 >> 172.21.15.78:0/1370027723 conn(0x7f2eec0fed40 msgr2=0x7f2eec105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 shutdown_connections 2024-04-25T14:09:00.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.116+0000 7f2ef1cee640 1 -- 172.21.15.78:0/1370027723 wait complete. 2024-04-25T14:09:00.127 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 -- 172.21.15.78:0/393039608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 msgr2=0x7fd5c0103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/393039608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0103750 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009900 tx=0x7fd5b402a410 comp rx=0 tx=0).stop 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 -- 172.21.15.78:0/393039608 shutdown_connections 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/393039608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0108990 0x7fd5c010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/393039608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c0103c90 0x7fd5c01040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/393039608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0103750 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 -- 172.21.15.78:0/393039608 >> 172.21.15.78:0/393039608 conn(0x7fd5c00fdc40 msgr2=0x7fd5c01000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 -- 172.21.15.78:0/393039608 shutdown_connections 2024-04-25T14:09:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 -- 172.21.15.78:0/393039608 wait complete. 2024-04-25T14:09:00.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.238+0000 7fd5c5bce640 1 Processor -- start 2024-04-25T14:09:00.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5c5bce640 1 -- start start 2024-04-25T14:09:00.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5c5bce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5c5bce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c0103c90 0x7fd5c019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5c5bce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0108990 0x7fd5c01a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5c5bce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010e4c0 con 0x7fd5c0102a50 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5c5bce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010e340 con 0x7fd5c0103c90 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5c5bce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010e640 con 0x7fd5c0108990 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46442/0 (socket says 172.21.15.78:46442) 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 -- 172.21.15.78:0/224664654 learned_addr learned my addr 172.21.15.78:0/224664654 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bffff640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0108990 0x7fd5c01a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5beffd640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c0103c90 0x7fd5c019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 -- 172.21.15.78:0/224664654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0108990 msgr2=0x7fd5c01a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0108990 0x7fd5c01a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 -- 172.21.15.78:0/224664654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c0103c90 msgr2=0x7fd5c019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c0103c90 0x7fd5c019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5bf7fe640 1 -- 172.21.15.78:0/224664654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c01a1d50 con 0x7fd5c0102a50 2024-04-25T14:09:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.239+0000 7fd5beffd640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c0103c90 0x7fd5c019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.240+0000 7fd5bf7fe640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0075860 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4006010 tx=0x7fd5b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.240+0000 7fd5bcff9640 1 -- 172.21.15.78:0/224664654 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b403f030 con 0x7fd5c0102a50 2024-04-25T14:09:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.240+0000 7fd5bcff9640 1 -- 172.21.15.78:0/224664654 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b4003aa0 con 0x7fd5c0102a50 2024-04-25T14:09:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.240+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c01a1fe0 con 0x7fd5c0102a50 2024-04-25T14:09:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.240+0000 7fd5bcff9640 1 -- 172.21.15.78:0/224664654 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4035070 con 0x7fd5c0102a50 2024-04-25T14:09:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.240+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c01a23e0 con 0x7fd5c0102a50 2024-04-25T14:09:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 Processor -- start 2024-04-25T14:09:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 -- start start 2024-04-25T14:09:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb429c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5bcff9640 1 -- 172.21.15.78:0/224664654 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd5b4042030 con 0x7fd5c0102a50 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed14d0 0x562fb429cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed96b0 0x562fb429da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562fb4200d80 con 0x562fb3efa390 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562fb41baa20 con 0x562fb3ed14d0 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5cca83740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562fb41fcd40 con 0x562fb3ed96b0 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.242+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb429c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb429c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46452/0 (socket says 172.21.15.78:46452) 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 -- 172.21.15.78:0/4159918601 learned_addr learned my addr 172.21.15.78:0/4159918601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bf7fe640 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed14d0 0x562fb429cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5beffd640 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed96b0 0x562fb429da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 -- 172.21.15.78:0/4159918601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed96b0 msgr2=0x562fb429da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed96b0 0x562fb429da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 -- 172.21.15.78:0/4159918601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed14d0 msgr2=0x562fb429cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed14d0 0x562fb429cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 -- 172.21.15.78:0/4159918601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb41c5c30 con 0x562fb3efa390 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bffff640 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb429c810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009ee0 tx=0x7fd5b8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627045e26838868b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bcff9640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd5a80afaf0 0x7fd5a80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5a5ffb640 1 -- 172.21.15.78:0/4159918601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8016070 con 0x562fb3efa390 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5a5ffb640 1 -- 172.21.15.78:0/4159918601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b8004d20 con 0x562fb3efa390 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5a5ffb640 1 -- 172.21.15.78:0/4159918601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8012780 con 0x562fb3efa390 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bf7fe640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd5a80afaf0 0x7fd5a80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.243+0000 7fd5bcff9640 1 -- 172.21.15.78:0/224664654 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd5b40ff320 con 0x7fd5c0102a50 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 -- 172.21.15.78:0/4159918601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 msgr2=0x562fb429c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5bf7fe640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd5a80afaf0 0x7fd5a80b2000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4102fd0 tx=0x7fd5b4102e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb429c810 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009ee0 tx=0x7fd5b8010740 comp rx=0 tx=0).stop 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 -- 172.21.15.78:0/4159918601 shutdown_connections 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed96b0 0x562fb429da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed14d0 0x562fb429cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 --2- 172.21.15.78:0/4159918601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb429c810 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 -- 172.21.15.78:0/4159918601 >> 172.21.15.78:0/4159918601 conn(0x562fb3eee4e0 msgr2=0x562fb4298020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 -- 172.21.15.78:0/4159918601 shutdown_connections 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.244+0000 7fd5cca83740 1 -- 172.21.15.78:0/4159918601 wait complete. 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 -- start start 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 Processor -- start 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed14d0 0x562fb42a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb42ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562fb429e530 con 0x562fb3efa390 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562fb4187dc0 con 0x562fb3ed96b0 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562fb429e6b0 con 0x562fb3ed14d0 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5cca83740 10 client.0 _ll_register_callbacks cb 0x562fb41a3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb42ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb42ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46454/0 (socket says 172.21.15.78:46454) 2024-04-25T14:09:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 -- 172.21.15.78:0/3798628632 learned_addr learned my addr 172.21.15.78:0/3798628632 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed14d0 0x562fb42a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5beffd640 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 -- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed14d0 msgr2=0x562fb42a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed14d0 0x562fb42a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 -- 172.21.15.78:0/3798628632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 msgr2=0x562fb42a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.245+0000 7fd5bffff640 1 -- 172.21.15.78:0/3798628632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb3f123f0 con 0x562fb3efa390 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5bffff640 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb42ad9d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8019000 tx=0x7fd5b8005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfb32b975956cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd59effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd59effd640 1 -- 172.21.15.78:0/3798628632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8016040 con 0x562fb3efa390 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd59effd640 1 -- 172.21.15.78:0/3798628632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd5b8012810 con 0x562fb3efa390 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b8004d20 con 0x562fb3efa390 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd59effd640 1 -- 172.21.15.78:0/3798628632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd5b8031050 con 0x562fb3efa390 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 10 client.18294 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562fb3fdc7d0 con 0x562fb3efa390 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd59effd640 1 -- 172.21.15.78:0/3798628632 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b8004d20 con 0x562fb3efa390 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd59effd640 1 -- 172.21.15.78:0/3798628632 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd5b800f040 con 0x562fb3efa390 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 10 client.18294 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 10 client.18294 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 20 client.18294 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3eb8f60 0x562fb42b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 4 client.18294 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.246+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fb420e990 con 0x562fb3eb8f60 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.247+0000 7fd5beffd640 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3eb8f60 0x562fb42b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.247+0000 7fd59effd640 10 client.18294 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.247+0000 7fd5beffd640 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3eb8f60 0x562fb42b3ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53e6ad29cb4be8cd server_cookie=f266180bcfd4e23f in_seq=0 out_seq=0 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd59effd640 1 -- 172.21.15.78:0/3798628632 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562fb420e990 con 0x562fb3eb8f60 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd59effd640 10 client.18294 handle_command_reply: tid=0 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 client.18294 shutdown 2024-04-25T14:09:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 msgr2=0x562fb42ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb42ad9d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8019000 tx=0x7fd5b8005c40 comp rx=0 tx=0).stop 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 shutdown_connections 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3eb8f60 0x562fb42b3ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3efa390 0x562fb42ad9d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.248+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3798628632 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3ed14d0 0x562fb42a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.249+0000 7fd59effd640 0 client.18294 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.249+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 >> 172.21.15.78:0/3798628632 conn(0x562fb3eee4e0 msgr2=0x562fb429af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.249+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 shutdown_connections 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.249+0000 7fd5cca83740 1 -- 172.21.15.78:0/3798628632 wait complete. 2024-04-25T14:09:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.249+0000 7fd5cca83740 20 client.18294 trim_cache size 0 max 16384 2024-04-25T14:09:00.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 Processor -- start 2024-04-25T14:09:00.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 -- start start 2024-04-25T14:09:00.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb429c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 0x562fb429cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01adb60 con 0x562fb3ed14d0 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01ab430 con 0x562fb3ed96b0 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5cca83740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01a9280 con 0x562fb3efa390 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb429c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb429c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46466/0 (socket says 172.21.15.78:46466) 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5bffff640 1 -- 172.21.15.78:0/2822597627 learned_addr learned my addr 172.21.15.78:0/2822597627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.262+0000 7fd5bffff640 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 0x562fb429cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5beffd640 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5bffff640 1 -- 172.21.15.78:0/2822597627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 msgr2=0x562fb429cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5bffff640 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 0x562fb429cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5bffff640 1 -- 172.21.15.78:0/2822597627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 msgr2=0x562fb42ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5bffff640 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5bffff640 1 -- 172.21.15.78:0/2822597627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb42a7130 con 0x562fb3ed14d0 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5bffff640 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb429c7d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8006010 tx=0x7fd5b8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218cd8344559c0be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5a57fa640 1 -- 172.21.15.78:0/2822597627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8016040 con 0x562fb3ed14d0 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5a57fa640 1 -- 172.21.15.78:0/2822597627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b80049e0 con 0x562fb3ed14d0 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.263+0000 7fd5a57fa640 1 -- 172.21.15.78:0/2822597627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8004f80 con 0x562fb3ed14d0 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 -- 172.21.15.78:0/2822597627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 msgr2=0x562fb429c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb429c7d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8006010 tx=0x7fd5b8010740 comp rx=0 tx=0).stop 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 -- 172.21.15.78:0/2822597627 shutdown_connections 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 0x562fb429cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 --2- 172.21.15.78:0/2822597627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb429c7d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 -- 172.21.15.78:0/2822597627 >> 172.21.15.78:0/2822597627 conn(0x562fb3eee4e0 msgr2=0x562fb42a12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 -- 172.21.15.78:0/2822597627 shutdown_connections 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 -- 172.21.15.78:0/2822597627 wait complete. 2024-04-25T14:09:00.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 -- start start 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 Processor -- start 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb42ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.264+0000 7fd5cca83740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 0x562fb42aa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5cca83740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01a98f0 con 0x562fb3ed14d0 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5cca83740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01aad60 con 0x562fb3ed96b0 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5cca83740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01a9fc0 con 0x562fb3efa390 2024-04-25T14:09:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5cca83740 10 client.0 _ll_register_callbacks cb 0x7fd5b4102cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb42ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb42ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46482/0 (socket says 172.21.15.78:46482) 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 -- 172.21.15.78:0/3860739038 learned_addr learned my addr 172.21.15.78:0/3860739038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5beffd640 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 -- 172.21.15.78:0/3860739038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 msgr2=0x562fb42aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 0x562fb42aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 -- 172.21.15.78:0/3860739038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 msgr2=0x562fb42aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 -- 172.21.15.78:0/3860739038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb431e7a0 con 0x562fb3ed14d0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5bffff640 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb42ad760 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd5b801a4e0 tx=0x7fd5b80056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abca6a5a3cf2901 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd59effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd59effd640 1 -- 172.21.15.78:0/3860739038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8039070 con 0x562fb3ed14d0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd59effd640 1 -- 172.21.15.78:0/3860739038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fd5b801aa50 con 0x562fb3ed14d0 2024-04-25T14:09:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b80049e0 con 0x562fb3ed14d0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd59effd640 1 -- 172.21.15.78:0/3860739038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd5b8012810 con 0x562fb3ed14d0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5cca83740 10 client.18300 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.265+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562fb431f890 con 0x562fb3ed14d0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd59effd640 1 -- 172.21.15.78:0/3860739038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b80049e0 con 0x562fb3ed14d0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd59effd640 1 -- 172.21.15.78:0/3860739038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd5b802a430 con 0x562fb3ed14d0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd5cca83740 10 client.18300 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd5cca83740 10 client.18300 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd5cca83740 20 client.18300 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3e8e3a0 0x562fb43221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd5cca83740 4 client.18300 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fb4208800 con 0x562fb3e8e3a0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.266+0000 7fd5beffd640 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3e8e3a0 0x562fb43221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd59effd640 10 client.18300 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5beffd640 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3e8e3a0 0x562fb43221f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64bbc142322c52a6 server_cookie=cc6110283f5685b1 in_seq=0 out_seq=0 2024-04-25T14:09:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd59effd640 1 -- 172.21.15.78:0/3860739038 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fb4208800 con 0x562fb3e8e3a0 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd59effd640 10 client.18300 handle_command_reply: tid=0 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 client.18300 shutdown 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 msgr2=0x562fb42ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb42ad760 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd5b801a4e0 tx=0x7fd5b80056b0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 shutdown_connections 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x562fb3e8e3a0 0x562fb43221f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fb3efa390 0x562fb42aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562fb3ed96b0 0x562fb42aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd59effd640 0 client.18300 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 --2- 172.21.15.78:0/3860739038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562fb3ed14d0 0x562fb42ad760 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 >> 172.21.15.78:0/3860739038 conn(0x562fb3eee4e0 msgr2=0x562fb42a12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 shutdown_connections 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 1 -- 172.21.15.78:0/3860739038 wait complete. 2024-04-25T14:09:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.267+0000 7fd5cca83740 20 client.18300 trim_cache size 0 max 16384 2024-04-25T14:09:00.273 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:09:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd5a80afaf0 msgr2=0x7fd5a80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd5a80afaf0 0x7fd5a80b2000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4102fd0 tx=0x7fd5b4102e00 comp rx=0 tx=0).stop 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 msgr2=0x7fd5c0075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0075860 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4006010 tx=0x7fd5b4004270 comp rx=0 tx=0).stop 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 shutdown_connections 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5c0108990 0x7fd5c01a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c0103c90 0x7fd5c019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fd5a80afaf0 0x7fd5a80b2000 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 --2- 172.21.15.78:0/224664654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c0102a50 0x7fd5c0075860 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 >> 172.21.15.78:0/224664654 conn(0x7fd5c00fdc40 msgr2=0x7fd5c0104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 shutdown_connections 2024-04-25T14:09:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.268+0000 7fd5c5bce640 1 -- 172.21.15.78:0/224664654 wait complete. 2024-04-25T14:09:00.278 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 -- 172.21.15.78:0/4208175677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54541050b0 msgr2=0x7f54541054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 --2- 172.21.15.78:0/4208175677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54541050b0 0x7f54541054c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5444009900 tx=0x7f544402a410 comp rx=0 tx=0).stop 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 -- 172.21.15.78:0/4208175677 shutdown_connections 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 --2- 172.21.15.78:0/4208175677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5454102260 0x7f545410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 --2- 172.21.15.78:0/4208175677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5454101760 0x7f5454101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 --2- 172.21.15.78:0/4208175677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54541050b0 0x7f54541054c0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 -- 172.21.15.78:0/4208175677 >> 172.21.15.78:0/4208175677 conn(0x7f54540fd380 msgr2=0x7f54540ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 -- 172.21.15.78:0/4208175677 shutdown_connections 2024-04-25T14:09:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.388+0000 7f54591ce640 1 -- 172.21.15.78:0/4208175677 wait complete. 2024-04-25T14:09:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 Processor -- start 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 -- start start 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5454101760 0x7f5454104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5454102260 0x7f5454102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54541050b0 0x7f5454102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f545410f600 con 0x7f5454102260 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f545410f480 con 0x7f54541050b0 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f54591ce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f545410f780 con 0x7f5454101760 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f545259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5454102260 0x7f5454102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f545259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5454102260 0x7f5454102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46492/0 (socket says 172.21.15.78:46492) 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.389+0000 7f545259c640 1 -- 172.21.15.78:0/2405042814 learned_addr learned my addr 172.21.15.78:0/2405042814 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f545359e640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54541050b0 0x7f5454102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f5452d9d640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5454101760 0x7f5454104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f545259c640 1 -- 172.21.15.78:0/2405042814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5454101760 msgr2=0x7f5454104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f545259c640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5454101760 0x7f5454104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f545259c640 1 -- 172.21.15.78:0/2405042814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54541050b0 msgr2=0x7f5454102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f545259c640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54541050b0 0x7f5454102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f545259c640 1 -- 172.21.15.78:0/2405042814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54541034e0 con 0x7f5454102260 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f545259c640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5454102260 0x7f5454102900 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f544800c890 tx=0x7f544800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f543bfff640 1 -- 172.21.15.78:0/2405042814 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448004220 con 0x7f5454102260 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f543bfff640 1 -- 172.21.15.78:0/2405042814 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54480043c0 con 0x7f5454102260 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54541a42f0 con 0x7f5454102260 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f543bfff640 1 -- 172.21.15.78:0/2405042814 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448004a00 con 0x7f5454102260 2024-04-25T14:09:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.390+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54541a4700 con 0x7f5454102260 2024-04-25T14:09:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 Processor -- start 2024-04-25T14:09:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 -- start start 2024-04-25T14:09:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f543bfff640 1 -- 172.21.15.78:0/2405042814 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5448004ba0 con 0x7f5454102260 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366202b390 0x5636623cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563662331d80 con 0x56366202b390 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636622eba20 con 0x56366200a6b0 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f5460080740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56366232dd40 con 0x5636620024d0 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f545359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366202b390 0x5636623cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f545359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366202b390 0x5636623cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46500/0 (socket says 172.21.15.78:46500) 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.393+0000 7f545359e640 1 -- 172.21.15.78:0/4079206548 learned_addr learned my addr 172.21.15.78:0/4079206548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f545259c640 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f5452d9d640 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f545359e640 1 -- 172.21.15.78:0/4079206548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 msgr2=0x5636623cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f545359e640 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f545359e640 1 -- 172.21.15.78:0/4079206548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 msgr2=0x5636623cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f545359e640 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f545359e640 1 -- 172.21.15.78:0/4079206548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636622f6c30 con 0x56366202b390 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f5452d9d640 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f545359e640 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366202b390 0x5636623cd810 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f544c007f40 tx=0x7f544c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adfea9199c5d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f543bfff640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f543c0afaf0 0x7f543c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f54397fa640 1 -- 172.21.15.78:0/4079206548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c010900 con 0x56366202b390 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f54397fa640 1 -- 172.21.15.78:0/4079206548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f544c0183f0 con 0x56366202b390 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f54397fa640 1 -- 172.21.15.78:0/4079206548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c017600 con 0x56366202b390 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f5452d9d640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f543c0afaf0 0x7f543c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.394+0000 7f543bfff640 1 -- 172.21.15.78:0/2405042814 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f54480de9d0 con 0x7f5454102260 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 -- 172.21.15.78:0/4079206548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366202b390 msgr2=0x5636623cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5452d9d640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f543c0afaf0 0x7f543c0b2000 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5444009230 tx=0x7f5444005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366202b390 0x5636623cd810 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f544c007f40 tx=0x7f544c00f850 comp rx=0 tx=0).stop 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 -- 172.21.15.78:0/4079206548 shutdown_connections 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 --2- 172.21.15.78:0/4079206548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366202b390 0x5636623cd810 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 -- 172.21.15.78:0/4079206548 >> 172.21.15.78:0/4079206548 conn(0x56366201f4e0 msgr2=0x5636623c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 -- 172.21.15.78:0/4079206548 shutdown_connections 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 -- 172.21.15.78:0/4079206548 wait complete. 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 -- start start 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.395+0000 7f5460080740 1 Processor -- start 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623d7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366202b390 0x5636623deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636623cf530 con 0x56366200a6b0 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636622b8dc0 con 0x56366202b390 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5636623cf6b0 con 0x5636620024d0 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 10 client.0 _ll_register_callbacks cb 0x5636622d4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46514/0 (socket says 172.21.15.78:46514) 2024-04-25T14:09:00.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 -- 172.21.15.78:0/2102833149 learned_addr learned my addr 172.21.15.78:0/2102833149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5452d9d640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623d7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545359e640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366202b390 0x5636623deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 -- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 msgr2=0x5636623d7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623d7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 -- 172.21.15.78:0/2102833149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366202b390 msgr2=0x5636623deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366202b390 0x5636623deac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 -- 172.21.15.78:0/2102833149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636620433f0 con 0x56366200a6b0 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545359e640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366202b390 0x5636623deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f545259c640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d7b60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f54480a6e30 tx=0x7f544800a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efd2e3483d5a0b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5436ff5640 1 -- 172.21.15.78:0/2102833149 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544801d040 con 0x56366200a6b0 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5436ff5640 1 -- 172.21.15.78:0/2102833149 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f54480a3490 con 0x56366200a6b0 2024-04-25T14:09:00.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544c0183f0 con 0x56366200a6b0 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5436ff5640 1 -- 172.21.15.78:0/2102833149 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f54480a6610 con 0x56366200a6b0 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 10 client.18312 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.396+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56366210d7d0 con 0x56366200a6b0 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5436ff5640 1 -- 172.21.15.78:0/2102833149 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f544c0183f0 con 0x56366200a6b0 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5436ff5640 1 -- 172.21.15.78:0/2102833149 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f54480bd060 con 0x56366200a6b0 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5460080740 10 client.18312 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5460080740 10 client.18312 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5460080740 20 client.18312 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5460080740 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563661fe9f60 0x5636623e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5460080740 4 client.18312 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56366233f990 con 0x563661fe9f60 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5452d9d640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563661fe9f60 0x5636623e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5436ff5640 10 client.18312 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.397+0000 7f5452d9d640 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563661fe9f60 0x5636623e5350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=839b01335c6dc447 server_cookie=7d6f8dcc57d06b86 in_seq=0 out_seq=0 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5436ff5640 1 -- 172.21.15.78:0/2102833149 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56366233f990 con 0x563661fe9f60 2024-04-25T14:09:00.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5436ff5640 10 client.18312 handle_command_reply: tid=0 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 client.18312 shutdown 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 msgr2=0x5636623d7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d7b60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f54480a6e30 tx=0x7f544800a5e0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 shutdown_connections 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366202b390 0x5636623deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d7b60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x563661fe9f60 0x5636623e5350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 --2- 172.21.15.78:0/2102833149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636620024d0 0x5636623d7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5436ff5640 0 client.18312 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 >> 172.21.15.78:0/2102833149 conn(0x56366201f4e0 msgr2=0x5636623c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 shutdown_connections 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.398+0000 7f5460080740 1 -- 172.21.15.78:0/2102833149 wait complete. 2024-04-25T14:09:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.399+0000 7f5460080740 20 client.18312 trim_cache size 0 max 16384 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 Processor -- start 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 -- start start 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636620024d0 0x5636623d61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54541afcc0 con 0x56366200a6b0 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54541ad590 con 0x5636620024d0 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5460080740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54541ab3e0 con 0x563661fe9f60 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46528/0 (socket says 172.21.15.78:46528) 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 -- 172.21.15.78:0/1585833960 learned_addr learned my addr 172.21.15.78:0/1585833960 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f5452d9d640 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545259c640 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636620024d0 0x5636623d61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 -- 172.21.15.78:0/1585833960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 msgr2=0x5636623d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 -- 172.21.15.78:0/1585833960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636620024d0 msgr2=0x5636623d61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636620024d0 0x5636623d61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.412+0000 7f545359e640 1 -- 172.21.15.78:0/1585833960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636623cf7a0 con 0x56366200a6b0 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f545359e640 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d2370 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f544c01a040 tx=0x7f544c007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d928d926acf336c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5438ff9640 1 -- 172.21.15.78:0/1585833960 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c026020 con 0x56366200a6b0 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5438ff9640 1 -- 172.21.15.78:0/1585833960 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f544c010bb0 con 0x56366200a6b0 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5438ff9640 1 -- 172.21.15.78:0/1585833960 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c005350 con 0x56366200a6b0 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5460080740 1 -- 172.21.15.78:0/1585833960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 msgr2=0x5636623d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5460080740 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d2370 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f544c01a040 tx=0x7f544c007940 comp rx=0 tx=0).stop 2024-04-25T14:09:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5460080740 1 -- 172.21.15.78:0/1585833960 shutdown_connections 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5460080740 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366200a6b0 0x5636623d2370 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5460080740 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636620024d0 0x5636623d61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5460080740 1 --2- 172.21.15.78:0/1585833960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.413+0000 7f5460080740 1 -- 172.21.15.78:0/1585833960 >> 172.21.15.78:0/1585833960 conn(0x56366201f4e0 msgr2=0x5636623cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 -- 172.21.15.78:0/1585833960 shutdown_connections 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 -- 172.21.15.78:0/1585833960 wait complete. 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 -- start start 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 Processor -- start 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623dde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636620024d0 0x5636623daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623daf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54541aba50 con 0x5636620024d0 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54541acec0 con 0x56366200a6b0 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54541ac120 con 0x563661fe9f60 2024-04-25T14:09:00.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5460080740 10 client.0 _ll_register_callbacks cb 0x7f54480058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636620024d0 0x5636623daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636620024d0 0x5636623daa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46544/0 (socket says 172.21.15.78:46544) 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 -- 172.21.15.78:0/9423558 learned_addr learned my addr 172.21.15.78:0/9423558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f5452d9d640 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623dde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545359e640 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623daf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 -- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 msgr2=0x5636623dde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623dde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 -- 172.21.15.78:0/9423558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 msgr2=0x5636623daf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623daf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.414+0000 7f545259c640 1 -- 172.21.15.78:0/9423558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636623db590 con 0x5636620024d0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f545259c640 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636620024d0 0x5636623daa10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f54480a6ba0 tx=0x7f54480a6330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1057fe59c0f918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5436ff5640 1 -- 172.21.15.78:0/9423558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544801d040 con 0x5636620024d0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5436ff5640 1 -- 172.21.15.78:0/9423558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f54480a3490 con 0x5636620024d0 2024-04-25T14:09:00.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544c010bb0 con 0x5636620024d0 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5436ff5640 1 -- 172.21.15.78:0/9423558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5448089050 con 0x5636620024d0 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 10 client.18318 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56366244c6e0 con 0x5636620024d0 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5436ff5640 1 -- 172.21.15.78:0/9423558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f544c010bb0 con 0x5636620024d0 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5436ff5640 1 -- 172.21.15.78:0/9423558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f54480ac3e0 con 0x5636620024d0 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 10 client.18318 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 10 client.18318 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 20 client.18318 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56366202b390 0x56366244edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 4 client.18318 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.415+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563662339800 con 0x56366202b390 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5452d9d640 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56366202b390 0x56366244edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5436ff5640 10 client.18318 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5452d9d640 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56366202b390 0x56366244edd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa395008888837c1 server_cookie=e496f1ab5150190a in_seq=0 out_seq=0 2024-04-25T14:09:00.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5436ff5640 1 -- 172.21.15.78:0/9423558 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563662339800 con 0x56366202b390 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5436ff5640 10 client.18318 handle_command_reply: tid=0 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 client.18318 shutdown 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636620024d0 msgr2=0x5636623daa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636620024d0 0x5636623daa10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f54480a6ba0 tx=0x7f54480a6330 comp rx=0 tx=0).stop 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 shutdown_connections 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56366200a6b0 0x5636623daf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636620024d0 0x5636623daa10 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x56366202b390 0x56366244edd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 --2- 172.21.15.78:0/9423558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563661fe9f60 0x5636623dde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 >> 172.21.15.78:0/9423558 conn(0x56366201f4e0 msgr2=0x5636623d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5436ff5640 0 client.18318 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 shutdown_connections 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.416+0000 7f5460080740 1 -- 172.21.15.78:0/9423558 wait complete. 2024-04-25T14:09:00.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f5460080740 20 client.18318 trim_cache size 0 max 16384 2024-04-25T14:09:00.423 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:09:00.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f543c0afaf0 msgr2=0x7f543c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f543c0afaf0 0x7f543c0b2000 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5444009230 tx=0x7f5444005de0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5454102260 msgr2=0x7f5454102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5454102260 0x7f5454102900 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f544800c890 tx=0x7f544800cd50 comp rx=0 tx=0).stop 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 shutdown_connections 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54541050b0 0x7f5454102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f543c0afaf0 0x7f543c0b2000 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5454102260 0x7f5454102900 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 --2- 172.21.15.78:0/2405042814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5454101760 0x7f5454104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 >> 172.21.15.78:0/2405042814 conn(0x7f54540fd380 msgr2=0x7f5454105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 shutdown_connections 2024-04-25T14:09:00.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.417+0000 7f54591ce640 1 -- 172.21.15.78:0/2405042814 wait complete. 2024-04-25T14:09:00.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:00 smithi078 ceph-mon[29163]: pgmap v632: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:00.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:00 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1370027723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:09:00.427 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:09:00.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:00 smithi112 ceph-mon[32617]: pgmap v632: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:00.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:00 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1370027723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 -- 172.21.15.78:0/1433201866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 msgr2=0x7fe1381047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 --2- 172.21.15.78:0/1433201866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe1381047f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe134009900 tx=0x7fe13402a410 comp rx=0 tx=0).stop 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 -- 172.21.15.78:0/1433201866 shutdown_connections 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 --2- 172.21.15.78:0/1433201866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe138109a30 0x7fe13810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 --2- 172.21.15.78:0/1433201866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe138104d30 0x7fe138105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 --2- 172.21.15.78:0/1433201866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe1381047f0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 -- 172.21.15.78:0/1433201866 >> 172.21.15.78:0/1433201866 conn(0x7fe1380fece0 msgr2=0x7fe138101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 -- 172.21.15.78:0/1433201866 shutdown_connections 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.537+0000 7fe13f30d640 1 -- 172.21.15.78:0/1433201866 wait complete. 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 Processor -- start 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 -- start start 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe13819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe138104d30 0x7fe13819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe138109a30 0x7fe1381a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe13810f5f0 con 0x7fe138103af0 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe13810f470 con 0x7fe138104d30 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13f30d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe13810f770 con 0x7fe138109a30 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe13819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe13819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46560/0 (socket says 172.21.15.78:46560) 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 -- 172.21.15.78:0/420522191 learned_addr learned my addr 172.21.15.78:0/420522191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d8aa640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe138109a30 0x7fe1381a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 -- 172.21.15.78:0/420522191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe138109a30 msgr2=0x7fe1381a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe138104d30 0x7fe13819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe138109a30 0x7fe1381a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 -- 172.21.15.78:0/420522191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe138104d30 msgr2=0x7fe13819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe138104d30 0x7fe13819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.538+0000 7fe13d0a9640 1 -- 172.21.15.78:0/420522191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1381a1d80 con 0x7fe138103af0 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.539+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe13819a2b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe134006010 tx=0x7fe134004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.539+0000 7fe12e7fc640 1 -- 172.21.15.78:0/420522191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13403f030 con 0x7fe138103af0 2024-04-25T14:09:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.539+0000 7fe12e7fc640 1 -- 172.21.15.78:0/420522191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe134003aa0 con 0x7fe138103af0 2024-04-25T14:09:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.539+0000 7fe12e7fc640 1 -- 172.21.15.78:0/420522191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134035070 con 0x7fe138103af0 2024-04-25T14:09:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.539+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1381a2010 con 0x7fe138103af0 2024-04-25T14:09:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.539+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1381a2550 con 0x7fe138103af0 2024-04-25T14:09:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.541+0000 7fe1461c4740 1 Processor -- start 2024-04-25T14:09:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.541+0000 7fe1461c4740 1 -- start start 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.541+0000 7fe12e7fc640 1 -- 172.21.15.78:0/420522191 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe134042030 con 0x7fe138103af0 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 0x560222b82810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b83a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe1461c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560222ae6d80 con 0x5602227b74d0 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe1461c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560222aa0a20 con 0x5602227bf6b0 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe1461c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560222ae2d40 con 0x5602227e0390 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b82d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46572/0 (socket says 172.21.15.78:46572) 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 -- 172.21.15.78:0/792836555 learned_addr learned my addr 172.21.15.78:0/792836555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d8aa640 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 0x560222b82810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b83a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 -- 172.21.15.78:0/792836555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 msgr2=0x560222b82810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 0x560222b82810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 -- 172.21.15.78:0/792836555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 msgr2=0x560222b83a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b83a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 -- 172.21.15.78:0/792836555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560222aabc30 con 0x5602227b74d0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe12e7fc640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1240afb70 0x7fe1240b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d8aa640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1240afb70 0x7fe1240b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b82d90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe1341074e0 tx=0x7fe1341075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417d1cbc73dfcc1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.542+0000 7fe12e7fc640 1 -- 172.21.15.78:0/420522191 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe1340ff2e0 con 0x7fe138103af0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.543+0000 7fe1237fe640 1 -- 172.21.15.78:0/792836555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1340c3670 con 0x5602227b74d0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.543+0000 7fe1237fe640 1 -- 172.21.15.78:0/792836555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1340c72c0 con 0x5602227b74d0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.543+0000 7fe1237fe640 1 -- 172.21.15.78:0/792836555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1340c6f20 con 0x5602227b74d0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 -- 172.21.15.78:0/792836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 msgr2=0x560222b82d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe13d8aa640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1240afb70 0x7fe1240b2080 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x560222b83860 tx=0x7fe130009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b82d90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe1341074e0 tx=0x7fe1341075d0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 -- 172.21.15.78:0/792836555 shutdown_connections 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b83a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b82d90 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 --2- 172.21.15.78:0/792836555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 0x560222b82810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 -- 172.21.15.78:0/792836555 >> 172.21.15.78:0/792836555 conn(0x5602227d44e0 msgr2=0x560222b7e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 -- 172.21.15.78:0/792836555 shutdown_connections 2024-04-25T14:09:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 -- 172.21.15.78:0/792836555 wait complete. 2024-04-25T14:09:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 -- start start 2024-04-25T14:09:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.544+0000 7fe1461c4740 1 Processor -- start 2024-04-25T14:09:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b8c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 0x560222b93730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560222b842f0 con 0x5602227b74d0 2024-04-25T14:09:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560222a6ddc0 con 0x5602227bf6b0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560222b84470 con 0x5602227e0390 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 10 client.0 _ll_register_callbacks cb 0x560222a89600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b8c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46578/0 (socket says 172.21.15.78:46578) 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 -- 172.21.15.78:0/22120532 learned_addr learned my addr 172.21.15.78:0/22120532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b8c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 -- 172.21.15.78:0/22120532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 msgr2=0x560222b93730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 0x560222b93730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 -- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 msgr2=0x560222b8c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b8c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 -- 172.21.15.78:0/22120532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602227f83f0 con 0x5602227b74d0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b8c290 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe1340fce20 tx=0x7fe134003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37032085008c312c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe120ff9640 1 -- 172.21.15.78:0/22120532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134049040 con 0x5602227b74d0 2024-04-25T14:09:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe120ff9640 1 -- 172.21.15.78:0/22120532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe13403bd90 con 0x5602227b74d0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1340c72c0 con 0x5602227b74d0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe120ff9640 1 -- 172.21.15.78:0/22120532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe1340c7100 con 0x5602227b74d0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 10 client.18330 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.545+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5602228c27d0 con 0x5602227b74d0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe120ff9640 1 -- 172.21.15.78:0/22120532 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1340c72c0 con 0x5602227b74d0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe120ff9640 1 -- 172.21.15.78:0/22120532 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe134003c50 con 0x5602227b74d0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe1461c4740 10 client.18330 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe1461c4740 10 client.18330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe1461c4740 20 client.18330 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe1461c4740 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56022279ef60 0x560222b99cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe1461c4740 4 client.18330 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560222af4990 con 0x56022279ef60 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.546+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56022279ef60 0x560222b99cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.547+0000 7fe120ff9640 10 client.18330 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.547+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56022279ef60 0x560222b99cf0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7cd6bda7ff305 server_cookie=6679584bb743d9ae in_seq=0 out_seq=0 2024-04-25T14:09:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.547+0000 7fe120ff9640 1 -- 172.21.15.78:0/22120532 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560222af4990 con 0x56022279ef60 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.547+0000 7fe120ff9640 10 client.18330 handle_command_reply: tid=0 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.547+0000 7fe1461c4740 1 client.18330 shutdown 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 msgr2=0x560222b8c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b8c290 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe1340fce20 tx=0x7fe134003f90 comp rx=0 tx=0).stop 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 shutdown_connections 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56022279ef60 0x560222b99cf0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227e0390 0x560222b93730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe120ff9640 0 client.18330 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227bf6b0 0x560222b8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 --2- 172.21.15.78:0/22120532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b8c290 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 >> 172.21.15.78:0/22120532 conn(0x5602227d44e0 msgr2=0x560222b80f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 shutdown_connections 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 1 -- 172.21.15.78:0/22120532 wait complete. 2024-04-25T14:09:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.548+0000 7fe1461c4740 20 client.18330 trim_cache size 0 max 16384 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.561+0000 7fe1461c4740 1 Processor -- start 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.561+0000 7fe1461c4740 1 -- start start 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.561+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227b74d0 0x560222b8be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.561+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227bf6b0 0x560222b8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.561+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b9e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.561+0000 7fe1461c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381adbe0 con 0x5602227bf6b0 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe1461c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381ab4b0 con 0x5602227e0390 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe1461c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381a9300 con 0x5602227b74d0 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227bf6b0 0x560222b8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227bf6b0 0x560222b8c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46582/0 (socket says 172.21.15.78:46582) 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 -- 172.21.15.78:0/1951069737 learned_addr learned my addr 172.21.15.78:0/1951069737 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13d8aa640 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b9e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227b74d0 0x560222b8be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 -- 172.21.15.78:0/1951069737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227b74d0 msgr2=0x560222b8be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227b74d0 0x560222b8be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 -- 172.21.15.78:0/1951069737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 msgr2=0x560222b9e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b9e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 -- 172.21.15.78:0/1951069737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560222b87ba0 con 0x5602227bf6b0 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227bf6b0 0x560222b8c3a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x560222b7edb0 tx=0x7fe128005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a9b907a6c26377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe122ffd640 1 -- 172.21.15.78:0/1951069737 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12801c390 con 0x5602227bf6b0 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe122ffd640 1 -- 172.21.15.78:0/1951069737 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe128021070 con 0x5602227bf6b0 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.562+0000 7fe122ffd640 1 -- 172.21.15.78:0/1951069737 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128011b20 con 0x5602227bf6b0 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 -- 172.21.15.78:0/1951069737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227bf6b0 msgr2=0x560222b8c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227bf6b0 0x560222b8c3a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x560222b7edb0 tx=0x7fe128005f30 comp rx=0 tx=0).stop 2024-04-25T14:09:00.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 -- 172.21.15.78:0/1951069737 shutdown_connections 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b9e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227bf6b0 0x560222b8c3a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 --2- 172.21.15.78:0/1951069737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227b74d0 0x560222b8be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 -- 172.21.15.78:0/1951069737 >> 172.21.15.78:0/1951069737 conn(0x5602227d44e0 msgr2=0x560222b8a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 -- 172.21.15.78:0/1951069737 shutdown_connections 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 -- 172.21.15.78:0/1951069737 wait complete. 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 -- start start 2024-04-25T14:09:00.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.563+0000 7fe1461c4740 1 Processor -- start 2024-04-25T14:09:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227bf6b0 0x560222b91bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe1461c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b92110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe1461c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381a9970 con 0x5602227b74d0 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe1461c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381aade0 con 0x5602227e0390 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe1461c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1381aa040 con 0x5602227bf6b0 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe1461c4740 10 client.0 _ll_register_callbacks cb 0x7fe1340c3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b91690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46586/0 (socket says 172.21.15.78:46586) 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 -- 172.21.15.78:0/3687727241 learned_addr learned my addr 172.21.15.78:0/3687727241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d8aa640 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b92110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227bf6b0 0x560222b91bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 -- 172.21.15.78:0/3687727241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227bf6b0 msgr2=0x560222b91bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227bf6b0 0x560222b91bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 -- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 msgr2=0x560222b92110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b92110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 -- 172.21.15.78:0/3687727241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560222c00080 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe13d0a9640 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b91690 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe1340c4a20 tx=0x7fe1340fce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5325f572780455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.564+0000 7fe120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe120ff9640 1 -- 172.21.15.78:0/3687727241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134049070 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe120ff9640 1 -- 172.21.15.78:0/3687727241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fe1340b00c0 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe128021070 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe120ff9640 1 -- 172.21.15.78:0/3687727241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe1340e5070 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 10 client.18336 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560222c01300 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe120ff9640 1 -- 172.21.15.78:0/3687727241 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe128021070 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe120ff9640 1 -- 172.21.15.78:0/3687727241 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe1340d0180 con 0x5602227b74d0 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 10 client.18336 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 10 client.18336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 20 client.18336 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5602227743a0 0x560222c038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 4 client.18336 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560222aee800 con 0x5602227743a0 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.565+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5602227743a0 0x560222c038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe120ff9640 10 client.18336 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe13c8a8640 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5602227743a0 0x560222c038a0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713aacf64179df25 server_cookie=1d44dde233736aa6 in_seq=0 out_seq=0 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe120ff9640 1 -- 172.21.15.78:0/3687727241 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560222aee800 con 0x5602227743a0 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe120ff9640 10 client.18336 handle_command_reply: tid=0 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 client.18336 shutdown 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 msgr2=0x560222b91690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b91690 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe1340c4a20 tx=0x7fe1340fce70 comp rx=0 tx=0).stop 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 shutdown_connections 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5602227e0390 0x560222b92110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5602227743a0 0x560222c038a0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5602227bf6b0 0x560222b91bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 --2- 172.21.15.78:0/3687727241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602227b74d0 0x560222b91690 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe120ff9640 0 client.18336 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 >> 172.21.15.78:0/3687727241 conn(0x5602227d44e0 msgr2=0x560222b8be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 shutdown_connections 2024-04-25T14:09:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 1 -- 172.21.15.78:0/3687727241 wait complete. 2024-04-25T14:09:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.566+0000 7fe1461c4740 20 client.18336 trim_cache size 0 max 16384 2024-04-25T14:09:00.572 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:09:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1240afb70 msgr2=0x7fe1240b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1240afb70 0x7fe1240b2080 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x560222b83860 tx=0x7fe130009500 comp rx=0 tx=0).stop 2024-04-25T14:09:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 msgr2=0x7fe13819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe13819a2b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe134006010 tx=0x7fe134004270 comp rx=0 tx=0).stop 2024-04-25T14:09:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 shutdown_connections 2024-04-25T14:09:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe138109a30 0x7fe1381a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe138104d30 0x7fe13819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fe1240afb70 0x7fe1240b2080 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 --2- 172.21.15.78:0/420522191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe138103af0 0x7fe13819a2b0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 >> 172.21.15.78:0/420522191 conn(0x7fe1380fece0 msgr2=0x7fe138105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 shutdown_connections 2024-04-25T14:09:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:00.567+0000 7fe13f30d640 1 -- 172.21.15.78:0/420522191 wait complete. 2024-04-25T14:09:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:00 smithi129 ceph-mon[32569]: pgmap v632: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:00 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1370027723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:09:01.578 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:09:01.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:01.579 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 -- 172.21.15.78:0/188328413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 msgr2=0x7f5afc103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 --2- 172.21.15.78:0/188328413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc103750 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009900 tx=0x7f5aec02a410 comp rx=0 tx=0).stop 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 -- 172.21.15.78:0/188328413 shutdown_connections 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 --2- 172.21.15.78:0/188328413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5afc108990 0x7f5afc10adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 --2- 172.21.15.78:0/188328413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5afc103c90 0x7f5afc1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 --2- 172.21.15.78:0/188328413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc103750 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 -- 172.21.15.78:0/188328413 >> 172.21.15.78:0/188328413 conn(0x7f5afc0fdc40 msgr2=0x7f5afc1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 -- 172.21.15.78:0/188328413 shutdown_connections 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 -- 172.21.15.78:0/188328413 wait complete. 2024-04-25T14:09:01.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.702+0000 7f5b012f2640 1 Processor -- start 2024-04-25T14:09:01.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5b012f2640 1 -- start start 2024-04-25T14:09:01.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5b012f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5b012f2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5afc103c90 0x7f5afc1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5b012f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5afc108990 0x7f5afc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5b012f2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10e4e0 con 0x7f5afc102a50 2024-04-25T14:09:01.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5b012f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10e360 con 0x7f5afc108990 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5b012f2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10e660 con 0x7f5afc103c90 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46600/0 (socket says 172.21.15.78:46600) 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 -- 172.21.15.78:0/3066046084 learned_addr learned my addr 172.21.15.78:0/3066046084 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5afc103c90 0x7f5afc1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afb7fe640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5afc108990 0x7f5afc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 -- 172.21.15.78:0/3066046084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5afc103c90 msgr2=0x7f5afc1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5afc103c90 0x7f5afc1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 -- 172.21.15.78:0/3066046084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5afc108990 msgr2=0x7f5afc1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5afc108990 0x7f5afc1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.703+0000 7f5afaffd640 1 -- 172.21.15.78:0/3066046084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc1a0cc0 con 0x7f5afc102a50 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.704+0000 7f5afaffd640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc199180 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006010 tx=0x7f5aec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.704+0000 7f5ae3fff640 1 -- 172.21.15.78:0/3066046084 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec03f030 con 0x7f5afc102a50 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.704+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc1a0f50 con 0x7f5afc102a50 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.704+0000 7f5ae3fff640 1 -- 172.21.15.78:0/3066046084 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec003aa0 con 0x7f5afc102a50 2024-04-25T14:09:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.704+0000 7f5ae3fff640 1 -- 172.21.15.78:0/3066046084 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec035070 con 0x7f5afc102a50 2024-04-25T14:09:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.704+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc1a13b0 con 0x7f5afc102a50 2024-04-25T14:09:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.706+0000 7f5b081a2740 1 Processor -- start 2024-04-25T14:09:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.706+0000 7f5ae3fff640 1 -- 172.21.15.78:0/3066046084 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5aec042030 con 0x7f5afc102a50 2024-04-25T14:09:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.706+0000 7f5b081a2740 1 -- start start 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0fbe410 0x561ee135f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f95550 0x561ee135fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1366920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5b081a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ee12c4b00 con 0x561ee0f9d730 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5b081a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ee1260920 con 0x561ee0fbe410 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5b081a2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ee12c0ac0 con 0x561ee0f95550 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1366920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1366920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46610/0 (socket says 172.21.15.78:46610) 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2836725454 learned_addr learned my addr 172.21.15.78:0/2836725454 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afb7fe640 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0fbe410 0x561ee135f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afaffd640 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f95550 0x561ee135fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5ae3fff640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ae40afaf0 0x7f5ae40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2836725454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f95550 msgr2=0x561ee135fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f95550 0x561ee135fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2836725454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0fbe410 msgr2=0x561ee135f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0fbe410 0x561ee135f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2836725454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee109e7e0 con 0x561ee0f9d730 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afb7fe640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ae40afaf0 0x7f5ae40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5ae3fff640 1 -- 172.21.15.78:0/3066046084 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5aec0ff130 con 0x7f5afc102a50 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.707+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1366920 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009c20 tx=0x7f5af00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e44fbed4cdc1e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.708+0000 7f5ae17fa640 1 -- 172.21.15.78:0/2836725454 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0004030 con 0x561ee0f9d730 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.708+0000 7f5ae17fa640 1 -- 172.21.15.78:0/2836725454 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af00035f0 con 0x561ee0f9d730 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.708+0000 7f5ae17fa640 1 -- 172.21.15.78:0/2836725454 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0012650 con 0x561ee0f9d730 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.708+0000 7f5b081a2740 1 -- 172.21.15.78:0/2836725454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 msgr2=0x561ee1366920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.708+0000 7f5afb7fe640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ae40afaf0 0x7f5ae40b2000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x561ee1360670 tx=0x7f5af4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.708+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1366920 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009c20 tx=0x7f5af00117b0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- 172.21.15.78:0/2836725454 shutdown_connections 2024-04-25T14:09:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1366920 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f95550 0x561ee135fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2836725454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0fbe410 0x561ee135f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- 172.21.15.78:0/2836725454 >> 172.21.15.78:0/2836725454 conn(0x561ee0fb2560 msgr2=0x561ee135ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- 172.21.15.78:0/2836725454 shutdown_connections 2024-04-25T14:09:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- 172.21.15.78:0/2836725454 wait complete. 2024-04-25T14:09:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- start start 2024-04-25T14:09:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 Processor -- start 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f95550 0x561ee1369230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1369770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0fbe410 0x561ee13706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ee1367370 con 0x561ee0f9d730 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561ee125e5d0 con 0x561ee0f95550 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ee13674f0 con 0x561ee0fbe410 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5b081a2740 10 client.0 _ll_register_callbacks cb 0x561ee124c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1369770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1369770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46622/0 (socket says 172.21.15.78:46622) 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.709+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2822742865 learned_addr learned my addr 172.21.15.78:0/2822742865 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afaffd640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f95550 0x561ee1369230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2822742865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0fbe410 msgr2=0x561ee13706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afb7fe640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0fbe410 0x561ee13706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0fbe410 0x561ee13706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f95550 msgr2=0x561ee1369230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f95550 0x561ee1369230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2822742865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee10fef70 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afaffd640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f95550 0x561ee1369230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1369770 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005d80 tx=0x7f5af0009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa7581568753815 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5adeff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5adeff5640 1 -- 172.21.15.78:0/2822742865 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0028020 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5adeff5640 1 -- 172.21.15.78:0/2822742865 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5af0004300 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af00035f0 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5adeff5640 1 -- 172.21.15.78:0/2822742865 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5af0033050 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5b081a2740 10 client.18348 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561ee0f96170 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.710+0000 7f5adeff5640 1 -- 172.21.15.78:0/2822742865 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af00035f0 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5adeff5640 1 -- 172.21.15.78:0/2822742865 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5af0024530 con 0x561ee0f9d730 2024-04-25T14:09:01.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5b081a2740 10 client.18348 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5b081a2740 10 client.18348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5b081a2740 20 client.18348 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0f7d1f0 0x561ee1376870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5b081a2740 4 client.18348 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ee12d2710 con 0x561ee0f7d1f0 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5afaffd640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0f7d1f0 0x561ee1376870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5adeff5640 10 client.18348 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.711+0000 7f5afaffd640 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0f7d1f0 0x561ee1376870 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4235d70cb40791 server_cookie=67d73fcda9e04ca9 in_seq=0 out_seq=0 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5adeff5640 1 -- 172.21.15.78:0/2822742865 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ee12d2710 con 0x561ee0f7d1f0 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5adeff5640 10 client.18348 handle_command_reply: tid=0 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 client.18348 shutdown 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 msgr2=0x561ee1369770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1369770 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005d80 tx=0x7f5af0009dd0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 shutdown_connections 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0f7d1f0 0x561ee1376870 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0fbe410 0x561ee13706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f9d730 0x561ee1369770 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5adeff5640 0 client.18348 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2822742865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f95550 0x561ee1369230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.712+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 >> 172.21.15.78:0/2822742865 conn(0x561ee0fb2560 msgr2=0x561ee1359ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.713+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 shutdown_connections 2024-04-25T14:09:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.713+0000 7f5b081a2740 1 -- 172.21.15.78:0/2822742865 wait complete. 2024-04-25T14:09:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.713+0000 7f5b081a2740 20 client.18348 trim_cache size 0 max 16384 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 Processor -- start 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 -- start start 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee1362dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee1370d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1acad0 con 0x561ee0f95550 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1aa3a0 con 0x561ee0f9d730 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5b081a2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1a81f0 con 0x561ee0f7d1f0 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.730+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46628/0 (socket says 172.21.15.78:46628) 2024-04-25T14:09:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2060800240 learned_addr learned my addr 172.21.15.78:0/2060800240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afaffd640 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee1362dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2060800240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 msgr2=0x561ee1362dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee1362dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2060800240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 msgr2=0x561ee1370d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee1370d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afa7fc640 1 -- 172.21.15.78:0/2060800240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee13640e0 con 0x561ee0f95550 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13707f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005d50 tx=0x7f5af00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd18e41d060ad5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5ae0ff9640 1 -- 172.21.15.78:0/2060800240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0028070 con 0x561ee0f95550 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5ae0ff9640 1 -- 172.21.15.78:0/2060800240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af00035f0 con 0x561ee0f95550 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.731+0000 7f5ae0ff9640 1 -- 172.21.15.78:0/2060800240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0003e60 con 0x561ee0f95550 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- 172.21.15.78:0/2060800240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 msgr2=0x561ee13707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13707f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005d50 tx=0x7f5af00117b0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- 172.21.15.78:0/2060800240 shutdown_connections 2024-04-25T14:09:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee1370d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13707f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 --2- 172.21.15.78:0/2060800240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee1362dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- 172.21.15.78:0/2060800240 >> 172.21.15.78:0/2060800240 conn(0x561ee0fb2560 msgr2=0x561ee13626b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- 172.21.15.78:0/2060800240 shutdown_connections 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- 172.21.15.78:0/2060800240 wait complete. 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- start start 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 Processor -- start 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee136ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee13dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1a8860 con 0x561ee0f95550 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1a9cd0 con 0x561ee0f9d730 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.732+0000 7f5b081a2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1a8f30 con 0x561ee0f7d1f0 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5b081a2740 10 client.0 _ll_register_callbacks cb 0x7f5af0003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46630/0 (socket says 172.21.15.78:46630) 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 -- 172.21.15.78:0/124523638 learned_addr learned my addr 172.21.15.78:0/124523638 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afaffd640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee136ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afb7fe640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee13dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 -- 172.21.15.78:0/124523638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 msgr2=0x561ee136ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee136ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 -- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 msgr2=0x561ee13dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee13dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 -- 172.21.15.78:0/124523638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee13df1b0 con 0x561ee0f95550 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afb7fe640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee13dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5afa7fc640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13704c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000c00 tx=0x7f5af0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cd00c8d4ab8ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5adeff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5adeff5640 1 -- 172.21.15.78:0/124523638 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0028040 con 0x561ee0f95550 2024-04-25T14:09:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5adeff5640 1 -- 172.21.15.78:0/124523638 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5af00243f0 con 0x561ee0f95550 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af00035f0 con 0x561ee0f95550 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5adeff5640 1 -- 172.21.15.78:0/124523638 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5af001f040 con 0x561ee0f95550 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5b081a2740 10 client.18354 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.733+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561ee13e0470 con 0x561ee0f95550 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5adeff5640 1 -- 172.21.15.78:0/124523638 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af00035f0 con 0x561ee0f95550 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5adeff5640 1 -- 172.21.15.78:0/124523638 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5af0038440 con 0x561ee0f95550 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5b081a2740 10 client.18354 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5b081a2740 10 client.18354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5b081a2740 20 client.18354 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5b081a2740 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0fbe410 0x561ee13e2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5b081a2740 4 client.18354 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ee12cc580 con 0x561ee0fbe410 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5afaffd640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0fbe410 0x561ee13e2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5adeff5640 10 client.18354 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.734+0000 7f5afaffd640 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0fbe410 0x561ee13e2dd0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62945771e7a208ab server_cookie=d07b491c5a101e14 in_seq=0 out_seq=0 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.735+0000 7f5adeff5640 1 -- 172.21.15.78:0/124523638 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ee12cc580 con 0x561ee0fbe410 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.735+0000 7f5adeff5640 10 client.18354 handle_command_reply: tid=0 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.735+0000 7f5b081a2740 1 client.18354 shutdown 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 msgr2=0x561ee13704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13704c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000c00 tx=0x7f5af0004f80 comp rx=0 tx=0).stop 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 shutdown_connections 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561ee0f9d730 0x561ee13dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ee0f95550 0x561ee13704c0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561ee0fbe410 0x561ee13e2dd0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 --2- 172.21.15.78:0/124523638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ee0f7d1f0 0x561ee136ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 >> 172.21.15.78:0/124523638 conn(0x561ee0fb2560 msgr2=0x561ee135dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5adeff5640 0 client.18354 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 shutdown_connections 2024-04-25T14:09:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 1 -- 172.21.15.78:0/124523638 wait complete. 2024-04-25T14:09:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b081a2740 20 client.18354 trim_cache size 0 max 16384 2024-04-25T14:09:01.742 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:09:01.742 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:09:01.742 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "278bc093-58bc-4623-b771-b5bfdba8ede0", 2024-04-25T14:09:01.742 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:09:01.742 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ae40afaf0 msgr2=0x7f5ae40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ae40afaf0 0x7f5ae40b2000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x561ee1360670 tx=0x7f5af4004a40 comp rx=0 tx=0).stop 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 msgr2=0x7f5afc199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc199180 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006010 tx=0x7f5aec004270 comp rx=0 tx=0).stop 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 shutdown_connections 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5afc108990 0x7f5afc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5afc103c90 0x7f5afc1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5ae40afaf0 0x7f5ae40b2000 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 --2- 172.21.15.78:0/3066046084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5afc102a50 0x7f5afc199180 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.736+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 >> 172.21.15.78:0/3066046084 conn(0x7f5afc0fdc40 msgr2=0x7f5afc104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.737+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 shutdown_connections 2024-04-25T14:09:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.737+0000 7f5b012f2640 1 -- 172.21.15.78:0/3066046084 wait complete. 2024-04-25T14:09:01.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:09:01.747 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:09:01.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.857+0000 7f04d9d46640 1 -- 172.21.15.78:0/463557313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d4102260 msgr2=0x7f04d410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.857+0000 7f04d9d46640 1 --2- 172.21.15.78:0/463557313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d4102260 0x7f04d410bcc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009ca0 tx=0x7f04cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 -- 172.21.15.78:0/463557313 shutdown_connections 2024-04-25T14:09:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 --2- 172.21.15.78:0/463557313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d4102260 0x7f04d410bcc0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 --2- 172.21.15.78:0/463557313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d4101760 0x7f04d4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 --2- 172.21.15.78:0/463557313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04d41050b0 0x7f04d41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 -- 172.21.15.78:0/463557313 >> 172.21.15.78:0/463557313 conn(0x7f04d40fd380 msgr2=0x7f04d40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 -- 172.21.15.78:0/463557313 shutdown_connections 2024-04-25T14:09:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 -- 172.21.15.78:0/463557313 wait complete. 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 Processor -- start 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.858+0000 7f04d9d46640 1 -- start start 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d9d46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04d4101760 0x7f04d4104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d9d46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d4102260 0x7f04d4102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d9d46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d41050b0 0x7f04d4102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d9d46640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d410f620 con 0x7f04d41050b0 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d9d46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04d410f4a0 con 0x7f04d4102260 2024-04-25T14:09:01.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d9d46640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04d410f7a0 con 0x7f04d4101760 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d41050b0 0x7f04d4102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d41050b0 0x7f04d4102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46644/0 (socket says 172.21.15.78:46644) 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 -- 172.21.15.78:0/3935510350 learned_addr learned my addr 172.21.15.78:0/3935510350 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d4102260 0x7f04d4102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d37fe640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04d4101760 0x7f04d4104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 -- 172.21.15.78:0/3935510350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04d4101760 msgr2=0x7f04d4104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04d4101760 0x7f04d4104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 -- 172.21.15.78:0/3935510350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d4102260 msgr2=0x7f04d4102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d4102260 0x7f04d4102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 -- 172.21.15.78:0/3935510350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d4103570 con 0x7f04d41050b0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d3fff640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d41050b0 0x7f04d4102e40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0097b0 tx=0x7f04cc031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d0ff9640 1 -- 172.21.15.78:0/3935510350 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc003bf0 con 0x7f04d41050b0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d41a4250 con 0x7f04d41050b0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.859+0000 7f04d0ff9640 1 -- 172.21.15.78:0/3935510350 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04cc003d90 con 0x7f04d41050b0 2024-04-25T14:09:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.860+0000 7f04d0ff9640 1 -- 172.21.15.78:0/3935510350 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc02fc30 con 0x7f04d41050b0 2024-04-25T14:09:01.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.860+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d41a4710 con 0x7f04d41050b0 2024-04-25T14:09:01.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04d0ff9640 1 -- 172.21.15.78:0/3935510350 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f04cc004040 con 0x7f04d41050b0 2024-04-25T14:09:01.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 Processor -- start 2024-04-25T14:09:01.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 -- start start 2024-04-25T14:09:01.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb56a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb56fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7946b0 0x5647ecb57c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647ecabbd80 con 0x5647ec7b5390 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647eca758b0 con 0x5647ec7946b0 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5647ecab7d40 con 0x5647ec78c4d0 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.862+0000 7f04d0ff9640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04bc0afaf0 0x7f04bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb56a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb56a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46660/0 (socket says 172.21.15.78:46660) 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 -- 172.21.15.78:0/2721260563 learned_addr learned my addr 172.21.15.78:0/2721260563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d37fe640 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb56fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d3fff640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04bc0afaf0 0x7f04bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7946b0 0x5647ecb57c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d0ff9640 1 -- 172.21.15.78:0/3935510350 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f04cc100520 con 0x7f04d41050b0 2024-04-25T14:09:01.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 -- 172.21.15.78:0/2721260563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 msgr2=0x5647ecb56fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb56fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 -- 172.21.15.78:0/2721260563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7946b0 msgr2=0x5647ecb57c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7946b0 0x5647ecb57c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 -- 172.21.15.78:0/2721260563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647eca46a40 con 0x5647ec7b5390 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb56a60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f04c80127d0 tx=0x7f04c800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d69707f5547d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04d3fff640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04bc0afaf0 0x7f04bc0b2000 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f04cc003fd0 tx=0x7f04cc0eb040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04b9ffb640 1 -- 172.21.15.78:0/2721260563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8012d50 con 0x5647ec7b5390 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04b9ffb640 1 -- 172.21.15.78:0/2721260563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c8010c20 con 0x5647ec7b5390 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.863+0000 7f04b9ffb640 1 -- 172.21.15.78:0/2721260563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c801ead0 con 0x5647ec7b5390 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2721260563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 msgr2=0x5647ecb56a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb56a60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f04c80127d0 tx=0x7f04c800b7b0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2721260563 shutdown_connections 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7946b0 0x5647ecb57c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb56fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2721260563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb56a60 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2721260563 >> 172.21.15.78:0/2721260563 conn(0x5647ec7a94e0 msgr2=0x5647ecb52270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2721260563 shutdown_connections 2024-04-25T14:09:01.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.864+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2721260563 wait complete. 2024-04-25T14:09:01.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 -- start start 2024-04-25T14:09:01.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 Processor -- start 2024-04-25T14:09:01.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec78c4d0 0x5647ecb60940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb60e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 0x5647ecb67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647ecb58780 con 0x5647ec78c4d0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647eca42d10 con 0x5647ec7b5390 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5647ecb58900 con 0x5647ec7946b0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04e0bfa740 10 client.0 _ll_register_callbacks cb 0x5647eca6f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec78c4d0 0x5647ecb60940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec78c4d0 0x5647ecb60940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46672/0 (socket says 172.21.15.78:46672) 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.865+0000 7f04d2ffd640 1 -- 172.21.15.78:0/4232780197 learned_addr learned my addr 172.21.15.78:0/4232780197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04d37fe640 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb60e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04d2ffd640 1 -- 172.21.15.78:0/4232780197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 msgr2=0x5647ecb60e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb60e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04d2ffd640 1 -- 172.21.15.78:0/4232780197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 msgr2=0x5647ecb67de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 0x5647ecb67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04d2ffd640 1 -- 172.21.15.78:0/4232780197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647ec8977d0 con 0x5647ec78c4d0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec78c4d0 0x5647ecb60940 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f04c800b390 tx=0x7f04c800b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab3b3c5ecf20f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:01.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04b2ffd640 1 -- 172.21.15.78:0/4232780197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c802c060 con 0x5647ec78c4d0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04b2ffd640 1 -- 172.21.15.78:0/4232780197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f04c8010970 con 0x5647ec78c4d0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c8010c20 con 0x5647ec78c4d0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04b2ffd640 1 -- 172.21.15.78:0/4232780197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f04c8004040 con 0x5647ec78c4d0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04e0bfa740 10 client.18366 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5647ec7cd3f0 con 0x5647ec78c4d0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04b2ffd640 1 -- 172.21.15.78:0/4232780197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c8010c20 con 0x5647ec78c4d0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.866+0000 7f04b2ffd640 1 -- 172.21.15.78:0/4232780197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f04c80035f0 con 0x5647ec78c4d0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04e0bfa740 10 client.18366 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04e0bfa740 10 client.18366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04e0bfa740 20 client.18366 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec773f60 0x5647ecb6e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04e0bfa740 4 client.18366 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647ecac9990 con 0x5647ec773f60 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04d37fe640 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec773f60 0x5647ecb6e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04b2ffd640 10 client.18366 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.867+0000 7f04d37fe640 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec773f60 0x5647ecb6e040 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f32c7cf00eceb8f2 server_cookie=21da0094575216c in_seq=0 out_seq=0 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04b2ffd640 1 -- 172.21.15.78:0/4232780197 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647ecac9990 con 0x5647ec773f60 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04b2ffd640 10 client.18366 handle_command_reply: tid=0 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 client.18366 shutdown 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec78c4d0 msgr2=0x5647ecb60940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec78c4d0 0x5647ecb60940 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f04c800b390 tx=0x7f04c800b3c0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 shutdown_connections 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 0x5647ecb67de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec773f60 0x5647ecb6e040 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb60e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04b2ffd640 0 client.18366 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/4232780197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec78c4d0 0x5647ecb60940 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 >> 172.21.15.78:0/4232780197 conn(0x5647ec7a94e0 msgr2=0x5647ecb55180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 shutdown_connections 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 1 -- 172.21.15.78:0/4232780197 wait complete. 2024-04-25T14:09:01.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.868+0000 7f04e0bfa740 20 client.18366 trim_cache size 0 max 16384 2024-04-25T14:09:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 Processor -- start 2024-04-25T14:09:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 -- start start 2024-04-25T14:09:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb5f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7946b0 0x5647ecb67d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 0x5647ecb5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41afd50 con 0x5647ec7946b0 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41ad620 con 0x5647ec7b5390 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41ab470 con 0x5647ec78c4d0 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7946b0 0x5647ecb67d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7946b0 0x5647ecb67d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46682/0 (socket says 172.21.15.78:46682) 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04d37fe640 1 -- 172.21.15.78:0/2626979536 learned_addr learned my addr 172.21.15.78:0/2626979536 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.886+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb5f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 0x5647ecb5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d37fe640 1 -- 172.21.15.78:0/2626979536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 msgr2=0x5647ecb5f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d37fe640 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb5f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d37fe640 1 -- 172.21.15.78:0/2626979536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 msgr2=0x5647ecb5b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d37fe640 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 0x5647ecb5b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d37fe640 1 -- 172.21.15.78:0/2626979536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ec78d0f0 con 0x5647ec7946b0 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb5f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04d37fe640 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7946b0 0x5647ecb67d70 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x5647ecb56780 tx=0x7f04c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2a6fd4c781c2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04b97fa640 1 -- 172.21.15.78:0/2626979536 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004220 con 0x5647ec7946b0 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04b97fa640 1 -- 172.21.15.78:0/2626979536 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c4024030 con 0x5647ec7946b0 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.887+0000 7f04b97fa640 1 -- 172.21.15.78:0/2626979536 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4020510 con 0x5647ec7946b0 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2626979536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7946b0 msgr2=0x5647ecb67d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7946b0 0x5647ecb67d70 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x5647ecb56780 tx=0x7f04c4005f90 comp rx=0 tx=0).stop 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2626979536 shutdown_connections 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec7b5390 0x5647ecb5b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7946b0 0x5647ecb67d70 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/2626979536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec78c4d0 0x5647ecb5f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2626979536 >> 172.21.15.78:0/2626979536 conn(0x5647ec7a94e0 msgr2=0x5647ecb60510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2626979536 shutdown_connections 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 -- 172.21.15.78:0/2626979536 wait complete. 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 -- start start 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 Processor -- start 2024-04-25T14:09:01.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec78c4d0 0x5647ecb65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.888+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb65650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41abae0 con 0x5647ec7b5390 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41acf50 con 0x5647ec78c4d0 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04d41ac1b0 con 0x5647ec7946b0 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 10 client.0 _ll_register_callbacks cb 0x7f04c800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb66580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46686/0 (socket says 172.21.15.78:46686) 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 -- 172.21.15.78:0/3075675319 learned_addr learned my addr 172.21.15.78:0/3075675319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec78c4d0 0x5647ecb65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d37fe640 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb65650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 -- 172.21.15.78:0/3075675319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 msgr2=0x5647ecb65650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb65650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 -- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec78c4d0 msgr2=0x5647ecb65110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec78c4d0 0x5647ecb65110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 -- 172.21.15.78:0/3075675319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647eca7c4b0 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04d2ffd640 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb66580 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f04c800b620 tx=0x7f04c800b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30e131755da04af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04b2ffd640 1 -- 172.21.15.78:0/3075675319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8036040 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04b2ffd640 1 -- 172.21.15.78:0/3075675319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f04c8003c00 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c4024030 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04b2ffd640 1 -- 172.21.15.78:0/3075675319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f04c802c040 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 10 client.18372 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.889+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5647ecb67c00 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04b2ffd640 1 -- 172.21.15.78:0/3075675319 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c4024030 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04b2ffd640 1 -- 172.21.15.78:0/3075675319 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f04c801e5d0 con 0x5647ec7b5390 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04e0bfa740 10 client.18372 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04e0bfa740 10 client.18372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04e0bfa740 20 client.18372 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec7493a0 0x5647ecbd86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04e0bfa740 4 client.18372 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647ecac3800 con 0x5647ec7493a0 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.890+0000 7f04d37fe640 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec7493a0 0x5647ecbd86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04b2ffd640 10 client.18372 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04d37fe640 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec7493a0 0x5647ecbd86b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a476eb20e4b44ec8 server_cookie=6b06cb4769fe0f32 in_seq=0 out_seq=0 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04b2ffd640 1 -- 172.21.15.78:0/3075675319 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5647ecac3800 con 0x5647ec7493a0 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04b2ffd640 10 client.18372 handle_command_reply: tid=0 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 client.18372 shutdown 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 msgr2=0x5647ecb66580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb66580 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f04c800b620 tx=0x7f04c800b3c0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 shutdown_connections 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647ec7b5390 0x5647ecb66580 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5647ec7493a0 0x5647ecbd86b0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647ec7946b0 0x5647ecb65650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 --2- 172.21.15.78:0/3075675319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647ec78c4d0 0x5647ecb65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04b2ffd640 0 client.18372 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 >> 172.21.15.78:0/3075675319 conn(0x5647ec7a94e0 msgr2=0x5647ecb509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 shutdown_connections 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 1 -- 172.21.15.78:0/3075675319 wait complete. 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.891+0000 7f04e0bfa740 20 client.18372 trim_cache size 0 max 16384 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: "278bc093-58bc-4623-b771-b5bfdba8ede0": { 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "278bc093-58bc-4623-b771-b5bfdba8ede0", 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:09:01.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:09:01.898 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:09:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04bc0afaf0 msgr2=0x7f04bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04bc0afaf0 0x7f04bc0b2000 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f04cc003fd0 tx=0x7f04cc0eb040 comp rx=0 tx=0).stop 2024-04-25T14:09:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d41050b0 msgr2=0x7f04d4102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d41050b0 0x7f04d4102e40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0097b0 tx=0x7f04cc031840 comp rx=0 tx=0).stop 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 shutdown_connections 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04bc0afaf0 0x7f04bc0b2000 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d41050b0 0x7f04d4102e40 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04d4102260 0x7f04d4102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 --2- 172.21.15.78:0/3935510350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04d4101760 0x7f04d4104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 >> 172.21.15.78:0/3935510350 conn(0x7f04d40fd380 msgr2=0x7f04d4107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 shutdown_connections 2024-04-25T14:09:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:01.892+0000 7f04d9d46640 1 -- 172.21.15.78:0/3935510350 wait complete. 2024-04-25T14:09:01.902 INFO:tasks.cephfs.filesystem:scrub status for tag:278bc093-58bc-4623-b771-b5bfdba8ede0 - {'path': '/', 'tag': '278bc093-58bc-4623-b771-b5bfdba8ede0', 'options': 'recursive,force'} 2024-04-25T14:09:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:02 smithi112 ceph-mon[32617]: pgmap v633: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:09:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:02 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:09:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:02 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:02 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:09:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:02 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:02 smithi129 ceph-mon[32569]: pgmap v633: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:09:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:02 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:09:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:02 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:02 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:09:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:02 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:02 smithi078 ceph-mon[29163]: pgmap v633: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:09:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:02 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:09:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:02 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:02 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:09:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:02 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:437/600: open fd = 3 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:438/600: open fd = 3 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:439/600: open fd = 3 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.547 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:440/600: open fd = 3 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:441/600: open fd = 3 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:442/600: open fd = 3 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:443/600: open fd = 3 2024-04-25T14:09:03.548 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:444/600: open fd = 3 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:445/600: open fd = 3 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:446/600: open fd = 3 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.549 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:447/600: open fd = 3 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:448/600: open fd = 3 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:449/600: open fd = 3 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:450/600: open fd = 3 2024-04-25T14:09:03.550 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:451/600: open fd = 3 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:452/600: open fd = 3 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:453/600: open fd = 3 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:454/600: open fd = 3 2024-04-25T14:09:03.551 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:455/600: open fd = 3 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:456/600: open fd = 3 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:457/600: open fd = 3 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:458/600: open fd = 3 2024-04-25T14:09:03.552 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:459/600: open fd = 3 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:460/600: open fd = 3 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:461/600: open fd = 3 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.553 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:462/600: open fd = 3 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:463/600: open fd = 3 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:464/600: open fd = 3 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:465/600: open fd = 3 2024-04-25T14:09:03.554 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:466/600: open fd = 3 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:467/600: open fd = 3 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:468/600: open fd = 3 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.555 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:469/600: open fd = 3 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:470/600: open fd = 3 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:471/600: open fd = 3 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:472/600: open fd = 3 2024-04-25T14:09:03.556 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:473/600: open fd = 3 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:474/600: open fd = 3 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:475/600: open fd = 3 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.557 INFO:tasks.workunit.client.0.smithi078.stdout:476/600: open fd = 3 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:477/600: open fd = 3 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:478/600: open fd = 3 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:479/600: open fd = 3 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.558 INFO:tasks.workunit.client.0.smithi078.stdout:480/600: open fd = 3 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:481/600: open fd = 3 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:482/600: open fd = 3 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:483/600: open fd = 3 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.559 INFO:tasks.workunit.client.0.smithi078.stdout:484/600: open fd = 3 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:485/600: open fd = 3 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:486/600: open fd = 3 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:487/600: open fd = 3 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.560 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:488/600: open fd = 3 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:489/600: open fd = 3 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:490/600: open fd = 3 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:491/600: open fd = 3 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.561 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:492/600: open fd = 3 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:493/600: open fd = 3 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:494/600: open fd = 3 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:495/600: open fd = 3 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.562 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:496/600: open fd = 3 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:497/600: open fd = 3 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:498/600: open fd = 3 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:09:03.563 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:09:04.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:04 smithi112 ceph-mon[32617]: pgmap v634: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:09:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:04 smithi129 ceph-mon[32569]: pgmap v634: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:09:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:04 smithi078 ceph-mon[29163]: pgmap v634: 97 pgs: 97 active+clean; 2.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:09:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:06 smithi112 ceph-mon[32617]: pgmap v635: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:09:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:06 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:09:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:06 smithi129 ceph-mon[32569]: pgmap v635: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:09:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:06 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:09:06.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:06 smithi078 ceph-mon[29163]: pgmap v635: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:09:06.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:06 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:09:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:08 smithi112 ceph-mon[32617]: pgmap v636: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:09:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:08 smithi129 ceph-mon[32569]: pgmap v636: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:09:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:08 smithi078 ceph-mon[29163]: pgmap v636: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:09:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 -- 172.21.15.78:0/163972680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c110280 msgr2=0x7f474c1126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 --2- 172.21.15.78:0/163972680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c110280 0x7f474c1126a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f4744009ca0 tx=0x7f474402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 -- 172.21.15.78:0/163972680 shutdown_connections 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 --2- 172.21.15.78:0/163972680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c110280 0x7f474c1126a0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 --2- 172.21.15.78:0/163972680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f474c0769f0 0x7f474c10fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 --2- 172.21.15.78:0/163972680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f474c0760a0 0x7f474c0764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 -- 172.21.15.78:0/163972680 >> 172.21.15.78:0/163972680 conn(0x7f474c0fed10 msgr2=0x7f474c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 -- 172.21.15.78:0/163972680 shutdown_connections 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.342+0000 7f47532e0640 1 -- 172.21.15.78:0/163972680 wait complete. 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 Processor -- start 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 -- start start 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f474c0760a0 0x7f474c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c0769f0 0x7f474c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f474c110280 0x7f474c1a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f474c115c60 con 0x7f474c0769f0 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f474c115ae0 con 0x7f474c110280 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f47532e0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f474c115de0 con 0x7f474c0760a0 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f475087b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c0769f0 0x7f474c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f475087b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c0769f0 0x7f474c1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48568/0 (socket says 172.21.15.78:48568) 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f475087b640 1 -- 172.21.15.78:0/1937320555 learned_addr learned my addr 172.21.15.78:0/1937320555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.343+0000 7f475187d640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f474c110280 0x7f474c1a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f475107c640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f474c0760a0 0x7f474c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f475087b640 1 -- 172.21.15.78:0/1937320555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f474c0760a0 msgr2=0x7f474c1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f475087b640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f474c0760a0 0x7f474c1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f475087b640 1 -- 172.21.15.78:0/1937320555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f474c110280 msgr2=0x7f474c1a7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f475087b640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f474c110280 0x7f474c1a7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f475087b640 1 -- 172.21.15.78:0/1937320555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c1a8300 con 0x7f474c0769f0 2024-04-25T14:09:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f475087b640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c0769f0 0x7f474c1a0d00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f473c00d950 tx=0x7f473c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f47427fc640 1 -- 172.21.15.78:0/1937320555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c017070 con 0x7f474c0769f0 2024-04-25T14:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f47427fc640 1 -- 172.21.15.78:0/1937320555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c009c30 con 0x7f474c0769f0 2024-04-25T14:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c1a85f0 con 0x7f474c0769f0 2024-04-25T14:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f47427fc640 1 -- 172.21.15.78:0/1937320555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c00f620 con 0x7f474c0769f0 2024-04-25T14:09:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.344+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1a8b90 con 0x7f474c0769f0 2024-04-25T14:09:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 Processor -- start 2024-04-25T14:09:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f47427fc640 1 -- 172.21.15.78:0/1937320555 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f473c010460 con 0x7f474c0769f0 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 -- start start 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84e31b0 0x55e0d8883970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8883ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84c24d0 0x55e0d888adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e0d87e9460 con 0x55e0d84ba2f0 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0d87a3590 con 0x55e0d84e31b0 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475a194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0d87e5420 con 0x55e0d84c24d0 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8883ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8883ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48572/0 (socket says 172.21.15.78:48572) 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 -- 172.21.15.78:0/2898688703 learned_addr learned my addr 172.21.15.78:0/2898688703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:09.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475087b640 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84c24d0 0x55e0d888adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475187d640 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84e31b0 0x55e0d8883970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 -- 172.21.15.78:0/2898688703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84c24d0 msgr2=0x55e0d888adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84c24d0 0x55e0d888adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 -- 172.21.15.78:0/2898688703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84e31b0 msgr2=0x55e0d8883970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84e31b0 0x55e0d8883970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f475107c640 1 -- 172.21.15.78:0/2898688703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d87b00d0 con 0x55e0d84ba2f0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.347+0000 7f47427fc640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47380afb60 0x7f47380b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.348+0000 7f475187d640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47380afb60 0x7f47380b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.348+0000 7f475107c640 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8883ef0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f474800bce0 tx=0x7f474800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39df582c04aebf61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.348+0000 7f47427fc640 1 -- 172.21.15.78:0/1937320555 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f473c0de7a0 con 0x7f474c0769f0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.348+0000 7f47377fe640 1 -- 172.21.15.78:0/2898688703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748015070 con 0x55e0d84ba2f0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.348+0000 7f47377fe640 1 -- 172.21.15.78:0/2898688703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474801a910 con 0x55e0d84ba2f0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.348+0000 7f47377fe640 1 -- 172.21.15.78:0/2898688703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748011900 con 0x55e0d84ba2f0 2024-04-25T14:09:09.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 -- 172.21.15.78:0/2898688703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 msgr2=0x55e0d8883ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475187d640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47380afb60 0x7f47380b2070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4744004af0 tx=0x7f4744009450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8883ef0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f474800bce0 tx=0x7f474800f730 comp rx=0 tx=0).stop 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 -- 172.21.15.78:0/2898688703 shutdown_connections 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84c24d0 0x55e0d888adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8883ef0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 --2- 172.21.15.78:0/2898688703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84e31b0 0x55e0d8883970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 -- 172.21.15.78:0/2898688703 >> 172.21.15.78:0/2898688703 conn(0x55e0d84d7300 msgr2=0x55e0d887f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 -- 172.21.15.78:0/2898688703 shutdown_connections 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 -- 172.21.15.78:0/2898688703 wait complete. 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 -- start start 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.349+0000 7f475a194740 1 Processor -- start 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475a194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84ba2f0 0x55e0d888d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475a194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 0x55e0d888db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475a194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84e31b0 0x55e0d8894ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475a194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e0d888b840 con 0x55e0d84e31b0 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475a194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0d8770b60 con 0x55e0d84c24d0 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475a194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0d888b9c0 con 0x55e0d84ba2f0 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475a194740 10 client.0 _ll_register_callbacks cb 0x55e0d87952b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84e31b0 0x55e0d8894ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84e31b0 0x55e0d8894ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48580/0 (socket says 172.21.15.78:48580) 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 -- 172.21.15.78:0/1707128100 learned_addr learned my addr 172.21.15.78:0/1707128100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475107c640 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84ba2f0 0x55e0d888d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475087b640 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 0x55e0d888db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 -- 172.21.15.78:0/1707128100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84ba2f0 msgr2=0x55e0d888d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84ba2f0 0x55e0d888d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 -- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 msgr2=0x55e0d888db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 0x55e0d888db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 -- 172.21.15.78:0/1707128100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d84fb210 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f475187d640 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84e31b0 0x55e0d8894ae0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4744035000 tx=0x7f474402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c817ba91ff1694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.350+0000 7f4734ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f4734ff9640 1 -- 172.21.15.78:0/1707128100 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440043d0 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f4734ff9640 1 -- 172.21.15.78:0/1707128100 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4744004570 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474801a910 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f4734ff9640 1 -- 172.21.15.78:0/1707128100 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4744004770 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 10 client.18384 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e0d85c7620 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f4734ff9640 1 -- 172.21.15.78:0/1707128100 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474801a910 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f4734ff9640 1 -- 172.21.15.78:0/1707128100 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f474405b050 con 0x55e0d84e31b0 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 10 client.18384 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:09.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 20 client.18384 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84a1d80 0x55e0d889b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 4 client.18384 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.351+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0d87f7070 con 0x55e0d84a1d80 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.352+0000 7f475107c640 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84a1d80 0x55e0d889b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.352+0000 7f4734ff9640 10 client.18384 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.352+0000 7f475107c640 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84a1d80 0x55e0d889b3b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3daefb905ff6da server_cookie=ec1cb667f1a7b664 in_seq=0 out_seq=0 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f4734ff9640 1 -- 172.21.15.78:0/1707128100 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0d87f7070 con 0x55e0d84a1d80 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f4734ff9640 10 client.18384 handle_command_reply: tid=0 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 client.18384 shutdown 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84e31b0 msgr2=0x55e0d8894ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84e31b0 0x55e0d8894ae0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4744035000 tx=0x7f474402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 shutdown_connections 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84e31b0 0x55e0d8894ae0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 0x55e0d888db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84a1d80 0x55e0d889b3b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 --2- 172.21.15.78:0/1707128100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84ba2f0 0x55e0d888d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f4734ff9640 0 client.18384 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 >> 172.21.15.78:0/1707128100 conn(0x55e0d84d7300 msgr2=0x55e0d8882210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 shutdown_connections 2024-04-25T14:09:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 1 -- 172.21.15.78:0/1707128100 wait complete. 2024-04-25T14:09:09.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.353+0000 7f475a194740 20 client.18384 trim_cache size 0 max 16384 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 Processor -- start 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 -- start start 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84ba2f0 0x55e0d887ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84c24d0 0x55e0d887f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d8882100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b41d0 con 0x55e0d84c24d0 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b1aa0 con 0x55e0d84ba2f0 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475a194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1af8f0 con 0x55e0d84e31b0 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475087b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84c24d0 0x55e0d887f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475087b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84c24d0 0x55e0d887f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48586/0 (socket says 172.21.15.78:48586) 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.368+0000 7f475087b640 1 -- 172.21.15.78:0/3783606210 learned_addr learned my addr 172.21.15.78:0/3783606210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475107c640 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84ba2f0 0x55e0d887ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475187d640 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d8882100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475087b640 1 -- 172.21.15.78:0/3783606210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 msgr2=0x55e0d8882100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475087b640 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d8882100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475087b640 1 -- 172.21.15.78:0/3783606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84ba2f0 msgr2=0x55e0d887ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475087b640 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84ba2f0 0x55e0d887ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475087b640 1 -- 172.21.15.78:0/3783606210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0d888ae70 con 0x55e0d84c24d0 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475107c640 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84ba2f0 0x55e0d887ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475187d640 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d8882100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f475087b640 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84c24d0 0x55e0d887f150 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f473c0a6970 tx=0x7f473c0a6680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad97718eacf8b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f4736ffd640 1 -- 172.21.15.78:0/3783606210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c020070 con 0x55e0d84c24d0 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f4736ffd640 1 -- 172.21.15.78:0/3783606210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c0a6780 con 0x55e0d84c24d0 2024-04-25T14:09:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.369+0000 7f4736ffd640 1 -- 172.21.15.78:0/3783606210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c0ab020 con 0x55e0d84c24d0 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 -- 172.21.15.78:0/3783606210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84c24d0 msgr2=0x55e0d887f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84c24d0 0x55e0d887f150 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f473c0a6970 tx=0x7f473c0a6680 comp rx=0 tx=0).stop 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 -- 172.21.15.78:0/3783606210 shutdown_connections 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d8882100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84c24d0 0x55e0d887f150 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 --2- 172.21.15.78:0/3783606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84ba2f0 0x55e0d887ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 -- 172.21.15.78:0/3783606210 >> 172.21.15.78:0/3783606210 conn(0x55e0d84d7300 msgr2=0x55e0d8887ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 -- 172.21.15.78:0/3783606210 shutdown_connections 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 -- 172.21.15.78:0/3783606210 wait complete. 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 -- start start 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.370+0000 7f475a194740 1 Processor -- start 2024-04-25T14:09:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8892070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 0x55e0d88925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d88934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1aff60 con 0x55e0d84ba2f0 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b13d0 con 0x55e0d84c24d0 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1b0630 con 0x55e0d84e31b0 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 10 client.0 _ll_register_callbacks cb 0x7f4744009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8892070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8892070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48596/0 (socket says 172.21.15.78:48596) 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 -- 172.21.15.78:0/165230184 learned_addr learned my addr 172.21.15.78:0/165230184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475187d640 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d88934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 -- 172.21.15.78:0/165230184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 msgr2=0x55e0d88934e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d88934e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 -- 172.21.15.78:0/165230184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 msgr2=0x55e0d88925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 0x55e0d88925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 -- 172.21.15.78:0/165230184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0d8893aa0 con 0x55e0d84ba2f0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475107c640 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8892070 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f474800bc30 tx=0x7f474800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93d053b2553f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f4734ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f4734ff9640 1 -- 172.21.15.78:0/165230184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748015040 con 0x55e0d84ba2f0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f4734ff9640 1 -- 172.21.15.78:0/165230184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4748004440 con 0x55e0d84ba2f0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.371+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473c0a6780 con 0x55e0d84ba2f0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f4734ff9640 1 -- 172.21.15.78:0/165230184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4748005040 con 0x55e0d84ba2f0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 10 client.18390 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e0d8903dd0 con 0x55e0d84ba2f0 2024-04-25T14:09:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f4734ff9640 1 -- 172.21.15.78:0/165230184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c0a6780 con 0x55e0d84ba2f0 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f4734ff9640 1 -- 172.21.15.78:0/165230184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4748004c90 con 0x55e0d84ba2f0 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 10 client.18390 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 10 client.18390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 20 client.18390 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84771c0 0x55e0d8906590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 4 client.18390 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0d87f0ee0 con 0x55e0d84771c0 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.372+0000 7f475087b640 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84771c0 0x55e0d8906590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.373+0000 7f4734ff9640 10 client.18390 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:09.379 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.373+0000 7f475087b640 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84771c0 0x55e0d8906590 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b379b32b27e0141 server_cookie=2bbd60838d21e0ad in_seq=0 out_seq=0 2024-04-25T14:09:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f4734ff9640 1 -- 172.21.15.78:0/165230184 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0d87f0ee0 con 0x55e0d84771c0 2024-04-25T14:09:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f4734ff9640 10 client.18390 handle_command_reply: tid=0 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 client.18390 shutdown 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 msgr2=0x55e0d8892070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8892070 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f474800bc30 tx=0x7f474800b690 comp rx=0 tx=0).stop 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 shutdown_connections 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e0d84771c0 0x55e0d8906590 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0d84e31b0 0x55e0d88934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0d84c24d0 0x55e0d88925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f4734ff9640 0 client.18390 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 --2- 172.21.15.78:0/165230184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0d84ba2f0 0x55e0d8892070 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 >> 172.21.15.78:0/165230184 conn(0x55e0d84d7300 msgr2=0x55e0d8882100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 shutdown_connections 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.388+0000 7f475a194740 1 -- 172.21.15.78:0/165230184 wait complete. 2024-04-25T14:09:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f475a194740 20 client.18390 trim_cache size 0 max 16384 2024-04-25T14:09:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:09:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:09:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:09:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47380afb60 msgr2=0x7f47380b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47380afb60 0x7f47380b2070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4744004af0 tx=0x7f4744009450 comp rx=0 tx=0).stop 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c0769f0 msgr2=0x7f474c1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c0769f0 0x7f474c1a0d00 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f473c00d950 tx=0x7f473c00de10 comp rx=0 tx=0).stop 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 shutdown_connections 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f474c110280 0x7f474c1a7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47380afb60 0x7f47380b2070 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f474c0769f0 0x7f474c1a0d00 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 --2- 172.21.15.78:0/1937320555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f474c0760a0 0x7f474c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 >> 172.21.15.78:0/1937320555 conn(0x7f474c0fed10 msgr2=0x7f474c1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 shutdown_connections 2024-04-25T14:09:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:09.389+0000 7f47532e0640 1 -- 172.21.15.78:0/1937320555 wait complete. 2024-04-25T14:09:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:10 smithi112 ceph-mon[32617]: pgmap v637: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:09:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:10 smithi129 ceph-mon[32569]: pgmap v637: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:09:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:10 smithi078 ceph-mon[29163]: pgmap v637: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:09:12.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:12 smithi112 ceph-mon[32617]: pgmap v638: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:09:12.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:12 smithi129 ceph-mon[32569]: pgmap v638: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:09:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:12 smithi078 ceph-mon[29163]: pgmap v638: 97 pgs: 97 active+clean; 2.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:09:14.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:14 smithi112 ceph-mon[32617]: pgmap v639: 97 pgs: 97 active+clean; 2.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:09:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:14 smithi129 ceph-mon[32569]: pgmap v639: 97 pgs: 97 active+clean; 2.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:09:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:14 smithi078 ceph-mon[29163]: pgmap v639: 97 pgs: 97 active+clean; 2.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-25T14:09:16.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:16 smithi112 ceph-mon[32617]: pgmap v640: 97 pgs: 97 active+clean; 2.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:16.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:16 smithi129 ceph-mon[32569]: pgmap v640: 97 pgs: 97 active+clean; 2.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:16.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:16 smithi078 ceph-mon[29163]: pgmap v640: 97 pgs: 97 active+clean; 2.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:18 smithi112 ceph-mon[32617]: pgmap v641: 97 pgs: 97 active+clean; 2.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:18 smithi129 ceph-mon[32569]: pgmap v641: 97 pgs: 97 active+clean; 2.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:18.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:18 smithi078 ceph-mon[29163]: pgmap v641: 97 pgs: 97 active+clean; 2.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:20.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:20 smithi112 ceph-mon[32617]: pgmap v642: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:20 smithi129 ceph-mon[32569]: pgmap v642: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:20.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:20 smithi078 ceph-mon[29163]: pgmap v642: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.495+0000 7f67c3bb2640 1 -- 172.21.15.78:0/1008971590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0760a0 msgr2=0x7f67bc0764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.495+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/1008971590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0760a0 0x7f67bc0764b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009900 tx=0x7f67b802a410 comp rx=0 tx=0).stop 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.495+0000 7f67c3bb2640 1 -- 172.21.15.78:0/1008971590 shutdown_connections 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.495+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/1008971590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67bc1102a0 0x7f67bc1126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.495+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/1008971590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67bc0769f0 0x7f67bc10fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.495+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/1008971590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0760a0 0x7f67bc0764b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.495+0000 7f67c3bb2640 1 -- 172.21.15.78:0/1008971590 >> 172.21.15.78:0/1008971590 conn(0x7f67bc0fed10 msgr2=0x7f67bc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 -- 172.21.15.78:0/1008971590 shutdown_connections 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 -- 172.21.15.78:0/1008971590 wait complete. 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 Processor -- start 2024-04-25T14:09:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 -- start start 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67bc0760a0 0x7f67bc1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0769f0 0x7f67bc1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67bc1102a0 0x7f67bc1a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc115c80 con 0x7f67bc0769f0 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc115b00 con 0x7f67bc0760a0 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.496+0000 7f67c3bb2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc115e00 con 0x7f67bc1102a0 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0769f0 0x7f67bc1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0769f0 0x7f67bc1a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37100/0 (socket says 172.21.15.78:37100) 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 -- 172.21.15.78:0/3943583831 learned_addr learned my addr 172.21.15.78:0/3943583831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c194e640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67bc0760a0 0x7f67bc1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c214f640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67bc1102a0 0x7f67bc1a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 -- 172.21.15.78:0/3943583831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67bc1102a0 msgr2=0x7f67bc1a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67bc1102a0 0x7f67bc1a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 -- 172.21.15.78:0/3943583831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67bc0760a0 msgr2=0x7f67bc1a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67bc0760a0 0x7f67bc1a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 -- 172.21.15.78:0/3943583831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1a83a0 con 0x7f67bc0769f0 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c214f640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67bc1102a0 0x7f67bc1a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c114d640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0769f0 0x7f67bc1a0da0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9f0 tx=0x7f67ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67b2ffd640 1 -- 172.21.15.78:0/3943583831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00cd80 con 0x7f67bc0769f0 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc1a8690 con 0x7f67bc0769f0 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67b2ffd640 1 -- 172.21.15.78:0/3943583831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac0054d0 con 0x7f67bc0769f0 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67b2ffd640 1 -- 172.21.15.78:0/3943583831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac010690 con 0x7f67bc0769f0 2024-04-25T14:09:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.497+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a8b90 con 0x7f67bc0769f0 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.500+0000 7f67b2ffd640 1 -- 172.21.15.78:0/3943583831 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f67ac004e90 con 0x7f67bc0769f0 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.500+0000 7f67caa63740 1 Processor -- start 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.500+0000 7f67caa63740 1 -- start start 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67caa63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165012970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67caa63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165012ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67caa63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x565165019df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67caa63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565164f78460 con 0x565164c721b0 2024-04-25T14:09:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67caa63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565164f32590 con 0x565164c514d0 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67caa63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565164f74420 con 0x565164c492f0 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67b2ffd640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67a80afae0 0x7f67a80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165012970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165012970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37112/0 (socket says 172.21.15.78:37112) 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 -- 172.21.15.78:0/1488550876 learned_addr learned my addr 172.21.15.78:0/1488550876 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67a80afae0 0x7f67a80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c194e640 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165012ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c114d640 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x565165019df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67b2ffd640 1 -- 172.21.15.78:0/3943583831 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f67ac0df150 con 0x7f67bc0769f0 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 -- 172.21.15.78:0/1488550876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 msgr2=0x565165012ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165012ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 -- 172.21.15.78:0/1488550876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 msgr2=0x565165019df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x565165019df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.501+0000 7f67c214f640 1 -- 172.21.15.78:0/1488550876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565164f3f0d0 con 0x565164c721b0 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.502+0000 7f67c214f640 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165012970 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f67b4014e00 tx=0x7f67b4011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea40138e80a3f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.502+0000 7f67c214f640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67a80afae0 0x7f67a80b1ff0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009270 tx=0x7f67b4016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.502+0000 7f67a7fff640 1 -- 172.21.15.78:0/1488550876 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4022070 con 0x565164c721b0 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.502+0000 7f67a7fff640 1 -- 172.21.15.78:0/1488550876 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b4009bf0 con 0x565164c721b0 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.502+0000 7f67a7fff640 1 -- 172.21.15.78:0/1488550876 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4005430 con 0x565164c721b0 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 -- 172.21.15.78:0/1488550876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 msgr2=0x565165012970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165012970 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f67b4014e00 tx=0x7f67b4011750 comp rx=0 tx=0).stop 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 -- 172.21.15.78:0/1488550876 shutdown_connections 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x565165019df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165012ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 --2- 172.21.15.78:0/1488550876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165012970 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 -- 172.21.15.78:0/1488550876 >> 172.21.15.78:0/1488550876 conn(0x565164c66300 msgr2=0x56516500e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 -- 172.21.15.78:0/1488550876 shutdown_connections 2024-04-25T14:09:22.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 -- 172.21.15.78:0/1488550876 wait complete. 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 -- start start 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.503+0000 7f67caa63740 1 Processor -- start 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c492f0 0x56516501c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x56516501cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c721b0 0x565165023ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56516501a7e0 con 0x565164c514d0 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565164effb60 con 0x565164c492f0 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56516501a960 con 0x565164c721b0 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 10 client.0 _ll_register_callbacks cb 0x565164f242b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x56516501cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x56516501cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37116/0 (socket says 172.21.15.78:37116) 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 -- 172.21.15.78:0/3667098341 learned_addr learned my addr 172.21.15.78:0/3667098341 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c194e640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c492f0 0x56516501c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c214f640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c721b0 0x565165023ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 -- 172.21.15.78:0/3667098341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c721b0 msgr2=0x565165023ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c721b0 0x565165023ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 -- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c492f0 msgr2=0x56516501c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c492f0 0x56516501c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 -- 172.21.15.78:0/3667098341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565164c8a210 con 0x565164c514d0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c194e640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c492f0 0x56516501c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67c114d640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x56516501cb80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0a3650 tx=0x7f67ac005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f3ccdb62504578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67a57fa640 1 -- 172.21.15.78:0/3667098341 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0a7d20 con 0x565164c514d0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67a57fa640 1 -- 172.21.15.78:0/3667098341 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f67ac0a3c10 con 0x565164c514d0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.504+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565164f86070 con 0x565164c514d0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67a57fa640 1 -- 172.21.15.78:0/3667098341 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f67ac0a7620 con 0x565164c514d0 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 10 client.18402 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x565164d56620 con 0x565164c514d0 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67a57fa640 1 -- 172.21.15.78:0/3667098341 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565164f86070 con 0x565164c514d0 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67a57fa640 1 -- 172.21.15.78:0/3667098341 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f67ac0a2f90 con 0x565164c514d0 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 10 client.18402 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 20 client.18402 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c30d80 0x56516502a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 4 client.18402 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565164f7fee0 con 0x565164c30d80 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.505+0000 7f67c194e640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c30d80 0x56516502a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.506+0000 7f67a57fa640 10 client.18402 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.506+0000 7f67c194e640 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c30d80 0x56516502a370 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe5c56e4c72ffd0 server_cookie=60242281c81be8cb in_seq=0 out_seq=0 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.506+0000 7f67a57fa640 1 -- 172.21.15.78:0/3667098341 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565164f7fee0 con 0x565164c30d80 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.506+0000 7f67a57fa640 10 client.18402 handle_command_reply: tid=0 2024-04-25T14:09:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.506+0000 7f67caa63740 1 client.18402 shutdown 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 msgr2=0x56516501cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x56516501cb80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0a3650 tx=0x7f67ac005180 comp rx=0 tx=0).stop 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 shutdown_connections 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c30d80 0x56516502a370 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c721b0 0x565165023ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x56516501cb80 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67a57fa640 0 client.18402 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 --2- 172.21.15.78:0/3667098341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c492f0 0x56516501c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 >> 172.21.15.78:0/3667098341 conn(0x565164c66300 msgr2=0x565165011210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 shutdown_connections 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 1 -- 172.21.15.78:0/3667098341 wait complete. 2024-04-25T14:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.507+0000 7f67caa63740 20 client.18402 trim_cache size 0 max 16384 2024-04-25T14:09:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 Processor -- start 2024-04-25T14:09:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 -- start start 2024-04-25T14:09:22.525 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x56516500dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x56516500e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165011100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b40f0 con 0x565164c721b0 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b19c0 con 0x565164c514d0 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67caa63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1af810 con 0x565164c492f0 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165011100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165011100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37130/0 (socket says 172.21.15.78:37130) 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 -- 172.21.15.78:0/3065176961 learned_addr learned my addr 172.21.15.78:0/3065176961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c114d640 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x56516500e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c194e640 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x56516500dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 -- 172.21.15.78:0/3065176961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 msgr2=0x56516500dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x56516500dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 -- 172.21.15.78:0/3065176961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 msgr2=0x56516500e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x56516500e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 -- 172.21.15.78:0/3065176961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516500e740 con 0x565164c721b0 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.522+0000 7f67c214f640 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165011100 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facc6e859baa5821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67a77fe640 1 -- 172.21.15.78:0/3065176961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4033070 con 0x565164c721b0 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67a77fe640 1 -- 172.21.15.78:0/3065176961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b4014250 con 0x565164c721b0 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67a77fe640 1 -- 172.21.15.78:0/3065176961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4022040 con 0x565164c721b0 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 -- 172.21.15.78:0/3065176961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 msgr2=0x565165011100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165011100 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b4005bd0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 -- 172.21.15.78:0/3065176961 shutdown_connections 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c721b0 0x565165011100 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c514d0 0x56516500e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 --2- 172.21.15.78:0/3065176961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x56516500dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 -- 172.21.15.78:0/3065176961 >> 172.21.15.78:0/3065176961 conn(0x565164c66300 msgr2=0x56516501ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 -- 172.21.15.78:0/3065176961 shutdown_connections 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.523+0000 7f67caa63740 1 -- 172.21.15.78:0/3065176961 wait complete. 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 -- start start 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 Processor -- start 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165020a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x565165020f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c721b0 0x565165021eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1afe80 con 0x565164c514d0 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b12f0 con 0x565164c721b0 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b0550 con 0x565164c492f0 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67caa63740 10 client.0 _ll_register_callbacks cb 0x7f67ac004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67c114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x565165020f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67c114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x565165020f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37140/0 (socket says 172.21.15.78:37140) 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67c114d640 1 -- 172.21.15.78:0/383919636 learned_addr learned my addr 172.21.15.78:0/383919636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:22.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67c194e640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165020a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.524+0000 7f67c214f640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c721b0 0x565165021eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67c114d640 1 -- 172.21.15.78:0/383919636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 msgr2=0x565165020a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67c114d640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165020a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67c114d640 1 -- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c721b0 msgr2=0x565165021eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67c114d640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c721b0 0x565165021eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67c114d640 1 -- 172.21.15.78:0/383919636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565165022470 con 0x565164c514d0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67c194e640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165020a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67c114d640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x565165020f80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0ca0f0 tx=0x7f67ac08ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce48ad9dd1dd6dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67a57fa640 1 -- 172.21.15.78:0/383919636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0a7130 con 0x565164c514d0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67a57fa640 1 -- 172.21.15.78:0/383919636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f67ac0b0200 con 0x565164c514d0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67b4014250 con 0x565164c514d0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67a57fa640 1 -- 172.21.15.78:0/383919636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f67ac0a7d20 con 0x565164c514d0 2024-04-25T14:09:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 10 client.18408 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5651650927a0 con 0x565164c514d0 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67a57fa640 1 -- 172.21.15.78:0/383919636 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b4014250 con 0x565164c514d0 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67a57fa640 1 -- 172.21.15.78:0/383919636 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f67ac0a6380 con 0x565164c514d0 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 10 client.18408 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 10 client.18408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 20 client.18408 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c061c0 0x565165095020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 4 client.18408 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.525+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565164efb5f0 con 0x565164c061c0 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.526+0000 7f67c194e640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c061c0 0x565165095020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.526+0000 7f67a57fa640 10 client.18408 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.526+0000 7f67c194e640 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c061c0 0x565165095020 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44de08bb06a7844b server_cookie=7ad30140faa46605 in_seq=0 out_seq=0 2024-04-25T14:09:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67a57fa640 1 -- 172.21.15.78:0/383919636 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565164efb5f0 con 0x565164c061c0 2024-04-25T14:09:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67a57fa640 10 client.18408 handle_command_reply: tid=0 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 client.18408 shutdown 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 msgr2=0x565165020f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x565165020f80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0ca0f0 tx=0x7f67ac08ecf0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 shutdown_connections 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565164c721b0 0x565165021eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565164c514d0 0x565165020f80 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x565164c061c0 0x565165095020 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 --2- 172.21.15.78:0/383919636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565164c492f0 0x565165020a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67a57fa640 0 client.18408 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 >> 172.21.15.78:0/383919636 conn(0x565164c66300 msgr2=0x565165011100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 shutdown_connections 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 1 -- 172.21.15.78:0/383919636 wait complete. 2024-04-25T14:09:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.541+0000 7f67caa63740 20 client.18408 trim_cache size 0 max 16384 2024-04-25T14:09:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:09:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:09:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:09:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67a80afae0 msgr2=0x7f67a80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67a80afae0 0x7f67a80b1ff0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009270 tx=0x7f67b4016000 comp rx=0 tx=0).stop 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0769f0 msgr2=0x7f67bc1a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0769f0 0x7f67bc1a0da0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9f0 tx=0x7f67ac00edf0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 shutdown_connections 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67bc1102a0 0x7f67bc1a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f67a80afae0 0x7f67a80b1ff0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67bc0769f0 0x7f67bc1a0da0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 --2- 172.21.15.78:0/3943583831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f67bc0760a0 0x7f67bc1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 >> 172.21.15.78:0/3943583831 conn(0x7f67bc0fed10 msgr2=0x7f67bc103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 shutdown_connections 2024-04-25T14:09:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:22.542+0000 7f67c3bb2640 1 -- 172.21.15.78:0/3943583831 wait complete. 2024-04-25T14:09:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:22 smithi129 ceph-mon[32569]: pgmap v643: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:09:22.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:22 smithi078 ceph-mon[29163]: pgmap v643: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:09:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:22 smithi112 ceph-mon[32617]: pgmap v643: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:09:24.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:24 smithi112 ceph-mon[32617]: pgmap v644: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:24 smithi129 ceph-mon[32569]: pgmap v644: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:24 smithi078 ceph-mon[29163]: pgmap v644: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:26.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:26 smithi129 ceph-mon[32569]: pgmap v645: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T14:09:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:26 smithi078 ceph-mon[29163]: pgmap v645: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T14:09:26.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:26 smithi112 ceph-mon[32617]: pgmap v645: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-25T14:09:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:28 smithi129 ceph-mon[32569]: pgmap v646: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:09:28.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:28 smithi078 ceph-mon[29163]: pgmap v646: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:09:28.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:28 smithi112 ceph-mon[32617]: pgmap v646: 97 pgs: 97 active+clean; 3.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:09:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:30 smithi129 ceph-mon[32569]: pgmap v647: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:30 smithi078 ceph-mon[29163]: pgmap v647: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:30.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:30 smithi112 ceph-mon[32617]: pgmap v647: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:31.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:09:31.904 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:09:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 -- 172.21.15.78:0/2654275987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 msgr2=0x7fc46810ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 --2- 172.21.15.78:0/2654275987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc46810ad60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fc460009ca0 tx=0x7fc46002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 -- 172.21.15.78:0/2654275987 shutdown_connections 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 --2- 172.21.15.78:0/2654275987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc46810ad60 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 --2- 172.21.15.78:0/2654275987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc468103c40 0x7fc468104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 --2- 172.21.15.78:0/2654275987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc468102a00 0x7fc468103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 -- 172.21.15.78:0/2654275987 >> 172.21.15.78:0/2654275987 conn(0x7fc4680fdbf0 msgr2=0x7fc4681000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 -- 172.21.15.78:0/2654275987 shutdown_connections 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.024+0000 7fc46e17d640 1 -- 172.21.15.78:0/2654275987 wait complete. 2024-04-25T14:09:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 Processor -- start 2024-04-25T14:09:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 -- start start 2024-04-25T14:09:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc468102a00 0x7fc468075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc468103c40 0x7fc46819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc4681a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810e530 con 0x7fc468108940 2024-04-25T14:09:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810e3b0 con 0x7fc468103c40 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.025+0000 7fc46e17d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc46810e6b0 con 0x7fc468102a00 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc4681a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc4681a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37524/0 (socket says 172.21.15.78:37524) 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 -- 172.21.15.78:0/134083671 learned_addr learned my addr 172.21.15.78:0/134083671 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc466ffd640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc468103c40 0x7fc46819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc4677fe640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc468102a00 0x7fc468075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 -- 172.21.15.78:0/134083671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc468102a00 msgr2=0x7fc468075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc468102a00 0x7fc468075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 -- 172.21.15.78:0/134083671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc468103c40 msgr2=0x7fc46819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc468103c40 0x7fc46819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 -- 172.21.15.78:0/134083671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4681a1cc0 con 0x7fc468108940 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc467fff640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc4681a1530 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc460005e30 tx=0x7fc46002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc464ff9640 1 -- 172.21.15.78:0/134083671 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460003bf0 con 0x7fc468108940 2024-04-25T14:09:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc464ff9640 1 -- 172.21.15.78:0/134083671 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc460003d90 con 0x7fc468108940 2024-04-25T14:09:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4681a1f50 con 0x7fc468108940 2024-04-25T14:09:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc464ff9640 1 -- 172.21.15.78:0/134083671 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46002fb90 con 0x7fc468108940 2024-04-25T14:09:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.026+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a23b0 con 0x7fc468108940 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 Processor -- start 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc464ff9640 1 -- 172.21.15.78:0/134083671 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc46002fd30 con 0x7fc468108940 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 -- start start 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560840606d80 con 0x560840300390 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5608405c08b0 con 0x5608402df6b0 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc47502b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560840602d40 con 0x5608402d74d0 2024-04-25T14:09:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37532/0 (socket says 172.21.15.78:37532) 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc466ffd640 1 -- 172.21.15.78:0/173620389 learned_addr learned my addr 172.21.15.78:0/173620389 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc4677fe640 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.029+0000 7fc466ffd640 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc466ffd640 1 -- 172.21.15.78:0/173620389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 msgr2=0x5608406a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc464ff9640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4500afaf0 0x7fc4500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc466ffd640 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc466ffd640 1 -- 172.21.15.78:0/173620389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 msgr2=0x5608406a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc466ffd640 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc466ffd640 1 -- 172.21.15.78:0/173620389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560840591a40 con 0x560840300390 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc4677fe640 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc467fff640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4500afaf0 0x7fc4500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc466ffd640 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406a1a60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fc45c013630 tx=0x7fc45c00a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b225558099a6993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc464ff9640 1 -- 172.21.15.78:0/134083671 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc4601002c0 con 0x7fc468108940 2024-04-25T14:09:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc44dffb640 1 -- 172.21.15.78:0/173620389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c019030 con 0x560840300390 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc44dffb640 1 -- 172.21.15.78:0/173620389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc45c0110d0 con 0x560840300390 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.030+0000 7fc44dffb640 1 -- 172.21.15.78:0/173620389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00f800 con 0x560840300390 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 -- 172.21.15.78:0/173620389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 msgr2=0x5608406a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc467fff640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4500afaf0 0x7fc4500b2000 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc4600fe120 tx=0x7fc4600972e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406a1a60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fc45c013630 tx=0x7fc45c00a7e0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 -- 172.21.15.78:0/173620389 shutdown_connections 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 --2- 172.21.15.78:0/173620389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406a1a60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 -- 172.21.15.78:0/173620389 >> 172.21.15.78:0/173620389 conn(0x5608402f44e0 msgr2=0x56084069d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 -- 172.21.15.78:0/173620389 shutdown_connections 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 -- 172.21.15.78:0/173620389 wait complete. 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.031+0000 7fc47502b740 1 -- start start 2024-04-25T14:09:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 1 Processor -- start 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5608406a3780 con 0x560840300390 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56084058dd10 con 0x5608402df6b0 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5608406a3900 con 0x5608402d74d0 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc47502b740 10 client.0 _ll_register_callbacks cb 0x5608405ba210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37536/0 (socket says 172.21.15.78:37536) 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 -- 172.21.15.78:0/985182520 learned_addr learned my addr 172.21.15.78:0/985182520 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc4677fe640 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 -- 172.21.15.78:0/985182520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 msgr2=0x5608406ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 -- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 msgr2=0x5608406abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.032+0000 7fc466ffd640 1 -- 172.21.15.78:0/985182520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608403e27d0 con 0x560840300390 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc466ffd640 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406b2e00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fc45c011d60 tx=0x7fc45c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5610451b345140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc446ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc446ffd640 1 -- 172.21.15.78:0/985182520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c019030 con 0x560840300390 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc446ffd640 1 -- 172.21.15.78:0/985182520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc45c027070 con 0x560840300390 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45c0110d0 con 0x560840300390 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc446ffd640 1 -- 172.21.15.78:0/985182520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc45c032050 con 0x560840300390 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 10 client.18420 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5608403183f0 con 0x560840300390 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc446ffd640 1 -- 172.21.15.78:0/985182520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc45c0110d0 con 0x560840300390 2024-04-25T14:09:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc446ffd640 1 -- 172.21.15.78:0/985182520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc45c02a020 con 0x560840300390 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 10 client.18420 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 10 client.18420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 20 client.18420 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402bef60 0x5608406b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 4 client.18420 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.033+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560840614990 con 0x5608402bef60 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.034+0000 7fc4677fe640 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402bef60 0x5608406b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.034+0000 7fc4677fe640 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402bef60 0x5608406b90a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f6ae07a25c77b8 server_cookie=35c46cb10e66b89d in_seq=0 out_seq=0 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.034+0000 7fc446ffd640 10 client.18420 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc446ffd640 1 -- 172.21.15.78:0/985182520 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560840614990 con 0x5608402bef60 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc446ffd640 10 client.18420 handle_command_reply: tid=0 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 client.18420 shutdown 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 msgr2=0x5608406b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406b2e00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fc45c011d60 tx=0x7fc45c015040 comp rx=0 tx=0).stop 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 shutdown_connections 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560840300390 0x5608406b2e00 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402bef60 0x5608406b90a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 --2- 172.21.15.78:0/985182520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608402d74d0 0x5608406ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc446ffd640 0 client.18420 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 >> 172.21.15.78:0/985182520 conn(0x5608402f44e0 msgr2=0x5608406a0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 shutdown_connections 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 1 -- 172.21.15.78:0/985182520 wait complete. 2024-04-25T14:09:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.035+0000 7fc47502b740 20 client.18420 trim_cache size 0 max 16384 2024-04-25T14:09:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 Processor -- start 2024-04-25T14:09:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 -- start start 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402d74d0 0x5608406a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402df6b0 0x5608406aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 0x56084069cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681adba0 con 0x5608402df6b0 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681ab470 con 0x5608402d74d0 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc47502b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681a92c0 con 0x560840300390 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402df6b0 0x5608406aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402df6b0 0x5608406aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37548/0 (socket says 172.21.15.78:37548) 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 -- 172.21.15.78:0/1433419901 learned_addr learned my addr 172.21.15.78:0/1433419901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc466ffd640 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402d74d0 0x5608406a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 -- 172.21.15.78:0/1433419901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 msgr2=0x56084069cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 0x56084069cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 -- 172.21.15.78:0/1433419901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402d74d0 msgr2=0x5608406a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402d74d0 0x5608406a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 -- 172.21.15.78:0/1433419901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608402d80f0 con 0x5608402df6b0 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc4677fe640 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402df6b0 0x5608406aa4b0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x5608406a1780 tx=0x7fc458005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d1a4da53f8937e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc44d7fa640 1 -- 172.21.15.78:0/1433419901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458004250 con 0x5608402df6b0 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc44d7fa640 1 -- 172.21.15.78:0/1433419901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4580043f0 con 0x5608402df6b0 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.053+0000 7fc44d7fa640 1 -- 172.21.15.78:0/1433419901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580204d0 con 0x5608402df6b0 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 -- 172.21.15.78:0/1433419901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402df6b0 msgr2=0x5608406aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402df6b0 0x5608406aa4b0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x5608406a1780 tx=0x7fc458005f90 comp rx=0 tx=0).stop 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 -- 172.21.15.78:0/1433419901 shutdown_connections 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 0x56084069cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402df6b0 0x5608406aa4b0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 --2- 172.21.15.78:0/1433419901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402d74d0 0x5608406a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 -- 172.21.15.78:0/1433419901 >> 172.21.15.78:0/1433419901 conn(0x5608402f44e0 msgr2=0x5608406a66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 -- 172.21.15.78:0/1433419901 shutdown_connections 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.054+0000 7fc47502b740 1 -- 172.21.15.78:0/1433419901 wait complete. 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 -- start start 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 Processor -- start 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402d74d0 0x5608406af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 0x56084071f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681a9930 con 0x5608402d74d0 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681aada0 con 0x5608402df6b0 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681aa000 con 0x560840300390 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc47502b740 10 client.0 _ll_register_callbacks cb 0x7fc45c013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402d74d0 0x5608406af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402d74d0 0x5608406af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37556/0 (socket says 172.21.15.78:37556) 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 -- 172.21.15.78:0/1866295727 learned_addr learned my addr 172.21.15.78:0/1866295727 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc4677fe640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 0x56084071f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 -- 172.21.15.78:0/1866295727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 msgr2=0x56084071f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 0x56084071f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 -- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 msgr2=0x5608406af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc466ffd640 1 -- 172.21.15.78:0/1866295727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608405c74b0 con 0x5608402d74d0 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.055+0000 7fc4677fe640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc466ffd640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402d74d0 0x5608406af220 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00a350 tx=0x7fc45c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6c61f483772b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc446ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc446ffd640 1 -- 172.21.15.78:0/1866295727 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c019070 con 0x5608402d74d0 2024-04-25T14:09:32.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc446ffd640 1 -- 172.21.15.78:0/1866295727 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc45c011ac0 con 0x5608402d74d0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4580043f0 con 0x5608402d74d0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc446ffd640 1 -- 172.21.15.78:0/1866295727 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc45c00ae70 con 0x5608402d74d0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 10 client.18426 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560840720750 con 0x5608402d74d0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc446ffd640 1 -- 172.21.15.78:0/1866295727 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4580043f0 con 0x5608402d74d0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc446ffd640 1 -- 172.21.15.78:0/1866295727 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc45c03f050 con 0x5608402d74d0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 10 client.18426 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 10 client.18426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 20 client.18426 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402943a0 0x560840723150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 4 client.18426 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56084060e800 con 0x5608402943a0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.056+0000 7fc4677fe640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402943a0 0x560840723150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc446ffd640 10 client.18426 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc4677fe640 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402943a0 0x560840723150 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a0fb905547275a server_cookie=a86633d0fa11a757 in_seq=0 out_seq=0 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc446ffd640 1 -- 172.21.15.78:0/1866295727 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56084060e800 con 0x5608402943a0 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc446ffd640 10 client.18426 handle_command_reply: tid=0 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 client.18426 shutdown 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402d74d0 msgr2=0x5608406af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402d74d0 0x5608406af220 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00a350 tx=0x7fc45c000bc0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 shutdown_connections 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5608402943a0 0x560840723150 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560840300390 0x56084071f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc446ffd640 0 client.18426 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5608402df6b0 0x5608406af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 --2- 172.21.15.78:0/1866295727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608402d74d0 0x5608406af220 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 >> 172.21.15.78:0/1866295727 conn(0x5608402f44e0 msgr2=0x5608406a9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 shutdown_connections 2024-04-25T14:09:32.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 1 -- 172.21.15.78:0/1866295727 wait complete. 2024-04-25T14:09:32.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.057+0000 7fc47502b740 20 client.18426 trim_cache size 0 max 16384 2024-04-25T14:09:32.066 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:09:32.066 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:09:32.066 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:09:32.066 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:09:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4500afaf0 msgr2=0x7fc4500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4500afaf0 0x7fc4500b2000 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc4600fe120 tx=0x7fc4600972e0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 msgr2=0x7fc4681a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc4681a1530 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc460005e30 tx=0x7fc46002f380 comp rx=0 tx=0).stop 2024-04-25T14:09:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 shutdown_connections 2024-04-25T14:09:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc4500afaf0 0x7fc4500b2000 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc468108940 0x7fc4681a1530 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc468103c40 0x7fc46819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 --2- 172.21.15.78:0/134083671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc468102a00 0x7fc468075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 >> 172.21.15.78:0/134083671 conn(0x7fc4680fdbf0 msgr2=0x7fc468104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 shutdown_connections 2024-04-25T14:09:32.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.058+0000 7fc46e17d640 1 -- 172.21.15.78:0/134083671 wait complete. 2024-04-25T14:09:32.069 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:09:32.069 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 -- 172.21.15.78:0/1007978771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580109a10 msgr2=0x7f658010be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 --2- 172.21.15.78:0/1007978771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580109a10 0x7f658010be30 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ca0 tx=0x7f657802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 -- 172.21.15.78:0/1007978771 shutdown_connections 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 --2- 172.21.15.78:0/1007978771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580109a10 0x7f658010be30 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 --2- 172.21.15.78:0/1007978771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6580104d10 0x7f6580105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 --2- 172.21.15.78:0/1007978771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6580103ad0 0x7f65801047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 -- 172.21.15.78:0/1007978771 >> 172.21.15.78:0/1007978771 conn(0x7f65800fecc0 msgr2=0x7f6580101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 -- 172.21.15.78:0/1007978771 shutdown_connections 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.176+0000 7f65886d8640 1 -- 172.21.15.78:0/1007978771 wait complete. 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 Processor -- start 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 -- start start 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6580103ad0 0x7f658019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580104d10 0x7f658019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6580109a10 0x7f65801a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f658010f600 con 0x7f6580104d10 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f658010f480 con 0x7f6580103ad0 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65886d8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f658010f780 con 0x7f6580109a10 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580104d10 0x7f658019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580104d10 0x7f658019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37584/0 (socket says 172.21.15.78:37584) 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 -- 172.21.15.78:0/3207431514 learned_addr learned my addr 172.21.15.78:0/3207431514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6586474640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6580103ad0 0x7f658019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6586c75640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6580109a10 0x7f65801a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 -- 172.21.15.78:0/3207431514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6580109a10 msgr2=0x7f65801a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6580109a10 0x7f65801a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 -- 172.21.15.78:0/3207431514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6580103ad0 msgr2=0x7f658019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6580103ad0 0x7f658019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 -- 172.21.15.78:0/3207431514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65801a1cc0 con 0x7f6580104d10 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f6585c73640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580104d10 0x7f658019a6c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d770 tx=0x7f657c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.177+0000 7f65777fe640 1 -- 172.21.15.78:0/3207431514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c017070 con 0x7f6580104d10 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.178+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65801a1fb0 con 0x7f6580104d10 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.178+0000 7f65777fe640 1 -- 172.21.15.78:0/3207431514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c0054d0 con 0x7f6580104d10 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.178+0000 7f65777fe640 1 -- 172.21.15.78:0/3207431514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c004c30 con 0x7f6580104d10 2024-04-25T14:09:32.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.178+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65801a2420 con 0x7f6580104d10 2024-04-25T14:09:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.179+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6580108820 con 0x7f6580104d10 2024-04-25T14:09:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.180+0000 7f65777fe640 1 -- 172.21.15.78:0/3207431514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f657c004dd0 con 0x7f6580104d10 2024-04-25T14:09:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.181+0000 7f65777fe640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f656c0afaf0 0x7f656c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.181+0000 7f6586474640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f656c0afaf0 0x7f656c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.181+0000 7f65777fe640 1 -- 172.21.15.78:0/3207431514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f657c0deb00 con 0x7f6580104d10 2024-04-25T14:09:32.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.181+0000 7f6586474640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f656c0afaf0 0x7f656c0b2000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f65801046d0 tx=0x7f6570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.184+0000 7f65777fe640 1 -- 172.21.15.78:0/3207431514 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f657c0a76d0 con 0x7f6580104d10 2024-04-25T14:09:32.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.401+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f658002db50 con 0x7f6580104d10 2024-04-25T14:09:32.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.402+0000 7f65777fe640 1 -- 172.21.15.78:0/3207431514 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f657c0a6fd0 con 0x7f6580104d10 2024-04-25T14:09:32.402 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:09:32.402 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:09:32.403 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:09:32.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f656c0afaf0 msgr2=0x7f656c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f656c0afaf0 0x7f656c0b2000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f65801046d0 tx=0x7f6570005f90 comp rx=0 tx=0).stop 2024-04-25T14:09:32.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580104d10 msgr2=0x7f658019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580104d10 0x7f658019a6c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d770 tx=0x7f657c00dc30 comp rx=0 tx=0).stop 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 shutdown_connections 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6580109a10 0x7f65801a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f656c0afaf0 0x7f656c0b2000 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6580104d10 0x7f658019a6c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 --2- 172.21.15.78:0/3207431514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6580103ad0 0x7f658019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 >> 172.21.15.78:0/3207431514 conn(0x7f65800fecc0 msgr2=0x7f65801059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 shutdown_connections 2024-04-25T14:09:32.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.404+0000 7f65886d8640 1 -- 172.21.15.78:0/3207431514 wait complete. 2024-04-25T14:09:32.415 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 -- 172.21.15.78:0/1192124409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401098a0 msgr2=0x7fb84010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 --2- 172.21.15.78:0/1192124409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401098a0 0x7fb84010bcc0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb838009ca0 tx=0x7fb83802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 -- 172.21.15.78:0/1192124409 shutdown_connections 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 --2- 172.21.15.78:0/1192124409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401098a0 0x7fb84010bcc0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 --2- 172.21.15.78:0/1192124409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb840106f40 0x7fb840109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 --2- 172.21.15.78:0/1192124409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8401045e0 0x7fb840106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 -- 172.21.15.78:0/1192124409 >> 172.21.15.78:0/1192124409 conn(0x7fb8400fdbe0 msgr2=0x7fb840100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 -- 172.21.15.78:0/1192124409 shutdown_connections 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 -- 172.21.15.78:0/1192124409 wait complete. 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 Processor -- start 2024-04-25T14:09:32.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.525+0000 7fb84656c640 1 -- start start 2024-04-25T14:09:32.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb84656c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401045e0 0x7fb84019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb84656c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb840106f40 0x7fb84019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb84656c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8401098a0 0x7fb8401a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb84656c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb84010f600 con 0x7fb8401045e0 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb84656c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb84010f480 con 0x7fb8401098a0 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb84656c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb84010f780 con 0x7fb840106f40 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401045e0 0x7fb84019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401045e0 0x7fb84019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37608/0 (socket says 172.21.15.78:37608) 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 -- 172.21.15.78:0/4063452682 learned_addr learned my addr 172.21.15.78:0/4063452682 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb844b09640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8401098a0 0x7fb8401a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb840106f40 0x7fb84019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 -- 172.21.15.78:0/4063452682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb840106f40 msgr2=0x7fb84019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb840106f40 0x7fb84019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 -- 172.21.15.78:0/4063452682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8401098a0 msgr2=0x7fb8401a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8401098a0 0x7fb8401a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 -- 172.21.15.78:0/4063452682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8401a1cc0 con 0x7fb8401045e0 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83ffff640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401045e0 0x7fb84019a230 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb83000d770 tx=0x7fb83000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.526+0000 7fb83d7fa640 1 -- 172.21.15.78:0/4063452682 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830017070 con 0x7fb8401045e0 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.527+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8401a1fb0 con 0x7fb8401045e0 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.527+0000 7fb83d7fa640 1 -- 172.21.15.78:0/4063452682 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb830009c30 con 0x7fb8401045e0 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.527+0000 7fb83d7fa640 1 -- 172.21.15.78:0/4063452682 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83000f620 con 0x7fb8401045e0 2024-04-25T14:09:32.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.527+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8401a2410 con 0x7fb8401045e0 2024-04-25T14:09:32.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb83d7fa640 1 -- 172.21.15.78:0/4063452682 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb83001a070 con 0x7fb8401045e0 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 Processor -- start 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 -- start start 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56485984ed80 con 0x5648595276b0 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564859808a20 con 0x564859548390 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb84d416740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56485984ad40 con 0x56485951f4d0 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598eba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37624/0 (socket says 172.21.15.78:37624) 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1719835886 learned_addr learned my addr 172.21.15.78:0/1719835886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb844b09640 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.529+0000 7fb83ffff640 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83d7fa640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb8280afaf0 0x7fb8280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1719835886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 msgr2=0x5648598ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1719835886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 msgr2=0x5648598ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1719835886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564859813c30 con 0x5648595276b0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb844b09640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb8280afaf0 0x7fb8280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83d7fa640 1 -- 172.21.15.78:0/4063452682 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb8300de7e0 con 0x7fb8401045e0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598eba20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb83400bb20 tx=0x7fb83400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea41e92f717c93a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb8267fc640 1 -- 172.21.15.78:0/1719835886 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834005020 con 0x5648595276b0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb8267fc640 1 -- 172.21.15.78:0/1719835886 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8340047a0 con 0x5648595276b0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.530+0000 7fb8267fc640 1 -- 172.21.15.78:0/1719835886 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83400f710 con 0x5648595276b0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 -- 172.21.15.78:0/1719835886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 msgr2=0x5648598eba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb844b09640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb8280afaf0 0x7fb8280b2000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x5648598eb860 tx=0x7fb83802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598eba20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb83400bb20 tx=0x7fb83400d340 comp rx=0 tx=0).stop 2024-04-25T14:09:32.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 -- 172.21.15.78:0/1719835886 shutdown_connections 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598eba20 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 --2- 172.21.15.78:0/1719835886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 -- 172.21.15.78:0/1719835886 >> 172.21.15.78:0/1719835886 conn(0x56485953c4e0 msgr2=0x5648598e6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 -- 172.21.15.78:0/1719835886 shutdown_connections 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 -- 172.21.15.78:0/1719835886 wait complete. 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.531+0000 7fb84d416740 1 -- start start 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 1 Processor -- start 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485951f4d0 0x5648598f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648595276b0 0x5648598f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5648598ec530 con 0x56485951f4d0 2024-04-25T14:09:32.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5648597d5dc0 con 0x564859548390 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648598ec6b0 con 0x5648595276b0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb84d416740 10 client.0 _ll_register_callbacks cb 0x5648597f1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485951f4d0 0x5648598f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485951f4d0 0x5648598f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37628/0 (socket says 172.21.15.78:37628) 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 -- 172.21.15.78:0/3733023560 learned_addr learned my addr 172.21.15.78:0/3733023560 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648595276b0 0x5648598f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb844b09640 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 -- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648595276b0 msgr2=0x5648598f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648595276b0 0x5648598f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 -- 172.21.15.78:0/3733023560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 msgr2=0x5648598fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.532+0000 7fb83ffff640 1 -- 172.21.15.78:0/3733023560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648595603f0 con 0x56485951f4d0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb83ffff640 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485951f4d0 0x5648598f4500 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb830009610 tx=0x7fb830005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4ee727031a23ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb81f7fe640 1 -- 172.21.15.78:0/3733023560 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830021070 con 0x56485951f4d0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb81f7fe640 1 -- 172.21.15.78:0/3733023560 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb8300a2620 con 0x56485951f4d0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8340047a0 con 0x56485951f4d0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb81f7fe640 1 -- 172.21.15.78:0/3733023560 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb8300a6420 con 0x56485951f4d0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 10 client.18444 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56485962a7d0 con 0x56485951f4d0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb81f7fe640 1 -- 172.21.15.78:0/3733023560 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8340047a0 con 0x56485951f4d0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb81f7fe640 1 -- 172.21.15.78:0/3733023560 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb8300a2820 con 0x56485951f4d0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 10 client.18444 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 10 client.18444 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 20 client.18444 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564859506f60 0x5648599021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 4 client.18444 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56485985c990 con 0x564859506f60 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.533+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564859506f60 0x5648599021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.534+0000 7fb81f7fe640 10 client.18444 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.534+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564859506f60 0x5648599021e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bfe0749952957c2 server_cookie=7fbe0506298192a8 in_seq=0 out_seq=0 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb81f7fe640 1 -- 172.21.15.78:0/3733023560 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56485985c990 con 0x564859506f60 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb81f7fe640 10 client.18444 handle_command_reply: tid=0 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 client.18444 shutdown 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485951f4d0 msgr2=0x5648598f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485951f4d0 0x5648598f4500 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fb830009610 tx=0x7fb830005bf0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 shutdown_connections 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x564859506f60 0x5648599021e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648595276b0 0x5648598f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 --2- 172.21.15.78:0/3733023560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485951f4d0 0x5648598f4500 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb81f7fe640 0 client.18444 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 >> 172.21.15.78:0/3733023560 conn(0x56485953c4e0 msgr2=0x5648598e8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 shutdown_connections 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 1 -- 172.21.15.78:0/3733023560 wait complete. 2024-04-25T14:09:32.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.535+0000 7fb84d416740 20 client.18444 trim_cache size 0 max 16384 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.548+0000 7fb84d416740 1 Processor -- start 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.548+0000 7fb84d416740 1 -- start start 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb84d416740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598e1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb84d416740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb84d416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb84d416740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401adae0 con 0x5648595276b0 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb84d416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401ab3b0 con 0x564859548390 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb84d416740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401a9200 con 0x56485951f4d0 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598e59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37634/0 (socket says 172.21.15.78:37634) 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1033734466 learned_addr learned my addr 172.21.15.78:0/1033734466 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83ffff640 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1033734466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 msgr2=0x5648598e1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598e1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1033734466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 msgr2=0x5648598e5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 -- 172.21.15.78:0/1033734466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648598f2620 con 0x5648595276b0 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598e59e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb834000c00 tx=0x7fb834005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b348cb4216087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb825ffb640 1 -- 172.21.15.78:0/1033734466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83402c020 con 0x5648595276b0 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb825ffb640 1 -- 172.21.15.78:0/1033734466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83400dd80 con 0x5648595276b0 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.549+0000 7fb825ffb640 1 -- 172.21.15.78:0/1033734466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834022430 con 0x5648595276b0 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 -- 172.21.15.78:0/1033734466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 msgr2=0x5648598e59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598e59e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb834000c00 tx=0x7fb834005ba0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 -- 172.21.15.78:0/1033734466 shutdown_connections 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564859548390 0x5648598e5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598e59e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 --2- 172.21.15.78:0/1033734466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56485951f4d0 0x5648598e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 -- 172.21.15.78:0/1033734466 >> 172.21.15.78:0/1033734466 conn(0x56485953c4e0 msgr2=0x5648598ef290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 -- 172.21.15.78:0/1033734466 shutdown_connections 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 -- 172.21.15.78:0/1033734466 wait complete. 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.550+0000 7fb84d416740 1 -- start start 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 1 Processor -- start 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56485951f4d0 0x5648598f8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598f8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564859548390 0x5648598f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401a9870 con 0x5648595276b0 2024-04-25T14:09:32.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401aace0 con 0x56485951f4d0 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401a9f40 con 0x564859548390 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb84d416740 10 client.0 _ll_register_callbacks cb 0x7fb8300a24d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598f8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37636/0 (socket says 172.21.15.78:37636) 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 -- 172.21.15.78:0/4202203244 learned_addr learned my addr 172.21.15.78:0/4202203244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb844b09640 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564859548390 0x5648598f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83ffff640 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56485951f4d0 0x5648598f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 -- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564859548390 msgr2=0x5648598f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564859548390 0x5648598f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 -- 172.21.15.78:0/4202203244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56485951f4d0 msgr2=0x5648598f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56485951f4d0 0x5648598f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 -- 172.21.15.78:0/4202203244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648598f9d40 con 0x5648595276b0 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb83f7fe640 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598f8850 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb83400df50 tx=0x7fb83400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ae08c280ccc1bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.551+0000 7fb81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb81f7fe640 1 -- 172.21.15.78:0/4202203244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834027900 con 0x5648595276b0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb81f7fe640 1 -- 172.21.15.78:0/4202203244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb834027150 con 0x5648595276b0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83400dd80 con 0x5648595276b0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb81f7fe640 1 -- 172.21.15.78:0/4202203244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb834027aa0 con 0x5648595276b0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 10 client.18450 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5648598fafc0 con 0x5648595276b0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb81f7fe640 1 -- 172.21.15.78:0/4202203244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83400dd80 con 0x5648595276b0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb81f7fe640 1 -- 172.21.15.78:0/4202203244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb83403a440 con 0x5648595276b0 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 10 client.18450 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 10 client.18450 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 20 client.18450 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5648594dc3a0 0x56485996c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 4 client.18450 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564859856800 con 0x5648594dc3a0 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.552+0000 7fb83ffff640 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5648594dc3a0 0x56485996c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb81f7fe640 10 client.18450 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb83ffff640 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5648594dc3a0 0x56485996c740 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f1b635d41c55c27 server_cookie=8d69228f2d1586ee in_seq=0 out_seq=0 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb81f7fe640 1 -- 172.21.15.78:0/4202203244 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564859856800 con 0x5648594dc3a0 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb81f7fe640 10 client.18450 handle_command_reply: tid=0 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 client.18450 shutdown 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 msgr2=0x5648598f8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598f8850 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb83400df50 tx=0x7fb83400b630 comp rx=0 tx=0).stop 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 shutdown_connections 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x5648594dc3a0 0x56485996c740 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564859548390 0x5648598f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5648595276b0 0x5648598f8850 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 --2- 172.21.15.78:0/4202203244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56485951f4d0 0x5648598f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb81f7fe640 0 client.18450 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:09:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 >> 172.21.15.78:0/4202203244 conn(0x56485953c4e0 msgr2=0x5648598e8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 shutdown_connections 2024-04-25T14:09:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.553+0000 7fb84d416740 1 -- 172.21.15.78:0/4202203244 wait complete. 2024-04-25T14:09:32.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84d416740 20 client.18450 trim_cache size 0 max 16384 2024-04-25T14:09:32.560 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb8280afaf0 msgr2=0x7fb8280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb8280afaf0 0x7fb8280b2000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x5648598eb860 tx=0x7fb83802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401045e0 msgr2=0x7fb84019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401045e0 0x7fb84019a230 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb83000d770 tx=0x7fb83000dc30 comp rx=0 tx=0).stop 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 shutdown_connections 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8401098a0 0x7fb8401a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb840106f40 0x7fb84019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb8280afaf0 0x7fb8280b2000 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 --2- 172.21.15.78:0/4063452682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8401045e0 0x7fb84019a230 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 >> 172.21.15.78:0/4063452682 conn(0x7fb8400fdbe0 msgr2=0x7fb8401051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 shutdown_connections 2024-04-25T14:09:32.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.554+0000 7fb84656c640 1 -- 172.21.15.78:0/4063452682 wait complete. 2024-04-25T14:09:32.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:32 smithi078 ceph-mon[29163]: pgmap v648: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:09:32.565 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:09:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:32 smithi129 ceph-mon[32569]: pgmap v648: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:09:32.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.673+0000 7f736f51f640 1 -- 172.21.15.78:0/1688319211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73681043c0 msgr2=0x7f7368102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.673+0000 7f736f51f640 1 --2- 172.21.15.78:0/1688319211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73681043c0 0x7f7368102dc0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7358009900 tx=0x7f735802a410 comp rx=0 tx=0).stop 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 -- 172.21.15.78:0/1688319211 shutdown_connections 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 --2- 172.21.15.78:0/1688319211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108a10 0x7f736810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 --2- 172.21.15.78:0/1688319211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73681043c0 0x7f7368102dc0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 --2- 172.21.15.78:0/1688319211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7368103ee0 0x7f73681027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 -- 172.21.15.78:0/1688319211 >> 172.21.15.78:0/1688319211 conn(0x7f73680fd340 msgr2=0x7f73680ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 -- 172.21.15.78:0/1688319211 shutdown_connections 2024-04-25T14:09:32.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 -- 172.21.15.78:0/1688319211 wait complete. 2024-04-25T14:09:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 Processor -- start 2024-04-25T14:09:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 -- start start 2024-04-25T14:09:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7368103ee0 0x7f7368198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.674+0000 7f736f51f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108a10 0x7f7368199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736f51f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736819e190 0x7f73681a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736f51f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f736810e520 con 0x7f7368103ee0 2024-04-25T14:09:32.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736f51f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f736810e3a0 con 0x7f736819e190 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736f51f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f736810e6a0 con 0x7f7368108a10 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7368103ee0 0x7f7368198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7368103ee0 0x7f7368198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37652/0 (socket says 172.21.15.78:37652) 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 -- 172.21.15.78:0/3688861086 learned_addr learned my addr 172.21.15.78:0/3688861086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736dabc640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736819e190 0x7f73681a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736caba640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108a10 0x7f7368199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 -- 172.21.15.78:0/3688861086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108a10 msgr2=0x7f7368199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108a10 0x7f7368199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 -- 172.21.15.78:0/3688861086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736819e190 msgr2=0x7f73681a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736819e190 0x7f73681a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 -- 172.21.15.78:0/3688861086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73681a0ba0 con 0x7f7368103ee0 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736d2bb640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7368103ee0 0x7f7368198ce0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f736400ccb0 tx=0x7f736400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f735e7fc640 1 -- 172.21.15.78:0/3688861086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736400ce60 con 0x7f7368103ee0 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f735e7fc640 1 -- 172.21.15.78:0/3688861086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736400fb00 con 0x7f7368103ee0 2024-04-25T14:09:32.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73681a0e90 con 0x7f7368103ee0 2024-04-25T14:09:32.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f735e7fc640 1 -- 172.21.15.78:0/3688861086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364018bb0 con 0x7f7368103ee0 2024-04-25T14:09:32.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.675+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73681a1350 con 0x7f7368103ee0 2024-04-25T14:09:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 Processor -- start 2024-04-25T14:09:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f735e7fc640 1 -- 172.21.15.78:0/3688861086 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f736400f650 con 0x7f7368103ee0 2024-04-25T14:09:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 -- start start 2024-04-25T14:09:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4bb390 0x562ffa85fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa85ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa49a6b0 0x562ffa860c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562ffa7c1d80 con 0x562ffa4924d0 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ffa77ba20 con 0x562ffa49a6b0 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.678+0000 7f73763d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ffa7bdd40 con 0x562ffa4bb390 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa85ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa85ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37664/0 (socket says 172.21.15.78:37664) 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 -- 172.21.15.78:0/784019237 learned_addr learned my addr 172.21.15.78:0/784019237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736caba640 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa49a6b0 0x562ffa860c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736dabc640 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4bb390 0x562ffa85fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 -- 172.21.15.78:0/784019237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4bb390 msgr2=0x562ffa85fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4bb390 0x562ffa85fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 -- 172.21.15.78:0/784019237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa49a6b0 msgr2=0x562ffa860c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa49a6b0 0x562ffa860c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f735e7fc640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f73540afb70 0x7f73540b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 -- 172.21.15.78:0/784019237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffa786c30 con 0x562ffa4924d0 2024-04-25T14:09:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736dabc640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f73540afb70 0x7f73540b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f736d2bb640 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa85ffa0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f73640ae900 tx=0x7f73640a3bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44f688c61773652 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f735e7fc640 1 -- 172.21.15.78:0/3688861086 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f73640df010 con 0x7f7368103ee0 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f73537fe640 1 -- 172.21.15.78:0/784019237 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640a72b0 con 0x562ffa4924d0 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f73537fe640 1 -- 172.21.15.78:0/784019237 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73640aec10 con 0x562ffa4924d0 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.679+0000 7f73537fe640 1 -- 172.21.15.78:0/784019237 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640a6c50 con 0x562ffa4924d0 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 -- 172.21.15.78:0/784019237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 msgr2=0x562ffa85ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f736dabc640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f73540afb70 0x7f73540b2080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x562ffa860a70 tx=0x7f736000b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa85ffa0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f73640ae900 tx=0x7f73640a3bf0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 -- 172.21.15.78:0/784019237 shutdown_connections 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa49a6b0 0x562ffa860c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa85ffa0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 --2- 172.21.15.78:0/784019237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4bb390 0x562ffa85fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 -- 172.21.15.78:0/784019237 >> 172.21.15.78:0/784019237 conn(0x562ffa4af4e0 msgr2=0x562ffa859020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 -- 172.21.15.78:0/784019237 shutdown_connections 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.680+0000 7f73763d0740 1 -- 172.21.15.78:0/784019237 wait complete. 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 -- start start 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 Processor -- start 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa8697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa869ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa870c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562ffa861740 con 0x562ffa4924d0 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562ffa748dc0 con 0x562ffa4bb390 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ffa8618c0 con 0x562ffa49a6b0 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f73763d0740 10 client.0 _ll_register_callbacks cb 0x562ffa764600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa8697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa8697a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37668/0 (socket says 172.21.15.78:37668) 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.681+0000 7f736d2bb640 1 -- 172.21.15.78:0/385648616 learned_addr learned my addr 172.21.15.78:0/385648616 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736dabc640 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa870c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736caba640 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa869ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736d2bb640 1 -- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 msgr2=0x562ffa869ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736d2bb640 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa869ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736d2bb640 1 -- 172.21.15.78:0/385648616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 msgr2=0x562ffa870c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736d2bb640 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa870c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736d2bb640 1 -- 172.21.15.78:0/385648616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffa4d33f0 con 0x562ffa4924d0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f736d2bb640 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa8697a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f73640af440 tx=0x7f736400af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7770d6574479938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f7350ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f7350ff9640 1 -- 172.21.15.78:0/385648616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640a6e30 con 0x562ffa4924d0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f7350ff9640 1 -- 172.21.15.78:0/385648616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7364018d50 con 0x562ffa4924d0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ffa7cf990 con 0x562ffa4924d0 2024-04-25T14:09:32.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f7350ff9640 1 -- 172.21.15.78:0/385648616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f73640a6c50 con 0x562ffa4924d0 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 10 client.18462 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562ffa59d7d0 con 0x562ffa4924d0 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f7350ff9640 1 -- 172.21.15.78:0/385648616 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562ffa7cf990 con 0x562ffa4924d0 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f7350ff9640 1 -- 172.21.15.78:0/385648616 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f73640a2e50 con 0x562ffa4924d0 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 10 client.18462 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 10 client.18462 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 20 client.18462 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa479f60 0x562ffa877090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 4 client.18462 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.682+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ffa7c9800 con 0x562ffa479f60 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.683+0000 7f736caba640 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa479f60 0x562ffa877090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.683+0000 7f7350ff9640 10 client.18462 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.683+0000 7f736caba640 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa479f60 0x562ffa877090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f39912393e967d21 server_cookie=503a2e5b768b91d7 in_seq=0 out_seq=0 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f7350ff9640 1 -- 172.21.15.78:0/385648616 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ffa7c9800 con 0x562ffa479f60 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f7350ff9640 10 client.18462 handle_command_reply: tid=0 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 client.18462 shutdown 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 msgr2=0x562ffa8697a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa8697a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f73640af440 tx=0x7f736400af40 comp rx=0 tx=0).stop 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 shutdown_connections 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa870c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa479f60 0x562ffa877090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa869ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 --2- 172.21.15.78:0/385648616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa8697a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f7350ff9640 0 client.18462 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:32.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 >> 172.21.15.78:0/385648616 conn(0x562ffa4af4e0 msgr2=0x562ffa85bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 shutdown_connections 2024-04-25T14:09:32.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 1 -- 172.21.15.78:0/385648616 wait complete. 2024-04-25T14:09:32.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.684+0000 7f73763d0740 20 client.18462 trim_cache size 0 max 16384 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.697+0000 7f73763d0740 1 Processor -- start 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f73763d0740 1 -- start start 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f73763d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa869a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f73763d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa870bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f73763d0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa865590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f73763d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ab930 con 0x562ffa4924d0 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f73763d0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73681a9200 con 0x562ffa4bb390 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f73763d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73681a7050 con 0x562ffa49a6b0 2024-04-25T14:09:32.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa869a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa869a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37680/0 (socket says 172.21.15.78:37680) 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 -- 172.21.15.78:0/2653297656 learned_addr learned my addr 172.21.15.78:0/2653297656 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736caba640 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa870bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 -- 172.21.15.78:0/2653297656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 msgr2=0x562ffa870bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736dabc640 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa865590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa870bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 -- 172.21.15.78:0/2653297656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 msgr2=0x562ffa865590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa865590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 -- 172.21.15.78:0/2653297656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffa860310 con 0x562ffa4924d0 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736caba640 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa870bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736dabc640 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa865590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f736d2bb640 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa869a80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7364005fd0 tx=0x7f7364018610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a483ca7bfeb8b1c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f7352ffd640 1 -- 172.21.15.78:0/2653297656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640a72b0 con 0x562ffa4924d0 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f7352ffd640 1 -- 172.21.15.78:0/2653297656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736400f940 con 0x562ffa4924d0 2024-04-25T14:09:32.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.698+0000 7f7352ffd640 1 -- 172.21.15.78:0/2653297656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364018d50 con 0x562ffa4924d0 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 -- 172.21.15.78:0/2653297656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 msgr2=0x562ffa869a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa869a80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7364005fd0 tx=0x7f7364018610 comp rx=0 tx=0).stop 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 -- 172.21.15.78:0/2653297656 shutdown_connections 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa865590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa49a6b0 0x562ffa870bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 --2- 172.21.15.78:0/2653297656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa4924d0 0x562ffa869a80 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 -- 172.21.15.78:0/2653297656 >> 172.21.15.78:0/2653297656 conn(0x562ffa4af4e0 msgr2=0x562ffa869370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 -- 172.21.15.78:0/2653297656 shutdown_connections 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 -- 172.21.15.78:0/2653297656 wait complete. 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 -- start start 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.699+0000 7f73763d0740 1 Processor -- start 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f73763d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4924d0 0x562ffa8d5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f73763d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa49a6b0 0x562ffa8d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f73763d0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa8dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f73763d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73681a76c0 con 0x562ffa49a6b0 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f73763d0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73681a8b30 con 0x562ffa4bb390 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f73763d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73681a7d90 con 0x562ffa4924d0 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f73763d0740 10 client.0 _ll_register_callbacks cb 0x7f7358009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa49a6b0 0x562ffa8d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa49a6b0 0x562ffa8d6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37696/0 (socket says 172.21.15.78:37696) 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 -- 172.21.15.78:0/3715137680 learned_addr learned my addr 172.21.15.78:0/3715137680 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736dabc640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa8dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736d2bb640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4924d0 0x562ffa8d5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 -- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4924d0 msgr2=0x562ffa8d5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4924d0 0x562ffa8d5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 -- 172.21.15.78:0/3715137680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 msgr2=0x562ffa8dcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa8dcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 -- 172.21.15.78:0/3715137680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffa8dd550 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736d2bb640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4924d0 0x562ffa8d5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736dabc640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa8dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f736caba640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa49a6b0 0x562ffa8d6030 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f73580096c0 tx=0x7f7358004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff06ebae42c74b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.700+0000 7f7350ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f7350ff9640 1 -- 172.21.15.78:0/3715137680 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7358035040 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f7350ff9640 1 -- 172.21.15.78:0/3715137680 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7358004400 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ffa744610 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f7350ff9640 1 -- 172.21.15.78:0/3715137680 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7358005040 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 10 client.18468 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562ffa8de720 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f7350ff9640 1 -- 172.21.15.78:0/3715137680 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562ffa744610 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f7350ff9640 1 -- 172.21.15.78:0/3715137680 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7358043ce0 con 0x562ffa49a6b0 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 10 client.18468 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 10 client.18468 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 20 client.18468 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa44f3a0 0x562ffa8e1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 4 client.18468 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ffa792240 con 0x562ffa44f3a0 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.701+0000 7f736d2bb640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa44f3a0 0x562ffa8e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f736d2bb640 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa44f3a0 0x562ffa8e1190 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d9390f0cfd501b8 server_cookie=89ed618857892255 in_seq=0 out_seq=0 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f7350ff9640 10 client.18468 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f7350ff9640 1 -- 172.21.15.78:0/3715137680 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ffa792240 con 0x562ffa44f3a0 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f7350ff9640 10 client.18468 handle_command_reply: tid=0 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 client.18468 shutdown 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa49a6b0 msgr2=0x562ffa8d6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa49a6b0 0x562ffa8d6030 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f73580096c0 tx=0x7f7358004040 comp rx=0 tx=0).stop 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 shutdown_connections 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562ffa4bb390 0x562ffa8dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ffa49a6b0 0x562ffa8d6030 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x562ffa44f3a0 0x562ffa8e1190 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 --2- 172.21.15.78:0/3715137680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ffa4924d0 0x562ffa8d5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f7350ff9640 0 client.18468 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:09:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 >> 172.21.15.78:0/3715137680 conn(0x562ffa4af4e0 msgr2=0x562ffa85be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 shutdown_connections 2024-04-25T14:09:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 1 -- 172.21.15.78:0/3715137680 wait complete. 2024-04-25T14:09:32.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.702+0000 7f73763d0740 20 client.18468 trim_cache size 0 max 16384 2024-04-25T14:09:32.709 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f73540afb70 msgr2=0x7f73540b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f73540afb70 0x7f73540b2080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x562ffa860a70 tx=0x7f736000b2e0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7368103ee0 msgr2=0x7f7368198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7368103ee0 0x7f7368198ce0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f736400ccb0 tx=0x7f736400a620 comp rx=0 tx=0).stop 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 shutdown_connections 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f736819e190 0x7f73681a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108a10 0x7f7368199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f73540afb70 0x7f73540b2080 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 --2- 172.21.15.78:0/3688861086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7368103ee0 0x7f7368198ce0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 >> 172.21.15.78:0/3688861086 conn(0x7f73680fd340 msgr2=0x7f73680ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 shutdown_connections 2024-04-25T14:09:32.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.703+0000 7f736f51f640 1 -- 172.21.15.78:0/3688861086 wait complete. 2024-04-25T14:09:32.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:09:32.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:32 smithi112 ceph-mon[32617]: pgmap v648: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T14:09:32.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 -- 172.21.15.78:0/1798450996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 msgr2=0x7f086c10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 --2- 172.21.15.78:0/1798450996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c10ad50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0864009ca0 tx=0x7f086402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 -- 172.21.15.78:0/1798450996 shutdown_connections 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 --2- 172.21.15.78:0/1798450996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c10ad50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 --2- 172.21.15.78:0/1798450996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f086c103c30 0x7f086c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 --2- 172.21.15.78:0/1798450996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f086c1029f0 0x7f086c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 -- 172.21.15.78:0/1798450996 >> 172.21.15.78:0/1798450996 conn(0x7f086c0fdbc0 msgr2=0x7f086c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 -- 172.21.15.78:0/1798450996 shutdown_connections 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.823+0000 7f0871ed8640 1 -- 172.21.15.78:0/1798450996 wait complete. 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 Processor -- start 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 -- start start 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f086c1029f0 0x7f086c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f086c103c30 0x7f086c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10e520 con 0x7f086c108930 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10e3a0 con 0x7f086c1029f0 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f0871ed8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f086c10e6a0 con 0x7f086c103c30 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f086bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f086bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c1a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37720/0 (socket says 172.21.15.78:37720) 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f086bfff640 1 -- 172.21.15.78:0/3797987272 learned_addr learned my addr 172.21.15.78:0/3797987272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f086b7fe640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f086c1029f0 0x7f086c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.824+0000 7f086affd640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f086c103c30 0x7f086c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f086bfff640 1 -- 172.21.15.78:0/3797987272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f086c103c30 msgr2=0x7f086c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f086bfff640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f086c103c30 0x7f086c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f086bfff640 1 -- 172.21.15.78:0/3797987272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f086c1029f0 msgr2=0x7f086c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f086bfff640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f086c1029f0 0x7f086c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f086bfff640 1 -- 172.21.15.78:0/3797987272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c1a0c00 con 0x7f086c108930 2024-04-25T14:09:32.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f086bfff640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c1a0590 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f08640097b0 tx=0x7f0864003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f0868ff9640 1 -- 172.21.15.78:0/3797987272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864004240 con 0x7f086c108930 2024-04-25T14:09:32.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c1a0e90 con 0x7f086c108930 2024-04-25T14:09:32.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f0868ff9640 1 -- 172.21.15.78:0/3797987272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086402f380 con 0x7f086c108930 2024-04-25T14:09:32.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f0868ff9640 1 -- 172.21.15.78:0/3797987272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086402fc30 con 0x7f086c108930 2024-04-25T14:09:32.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.825+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c1a1300 con 0x7f086c108930 2024-04-25T14:09:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0868ff9640 1 -- 172.21.15.78:0/3797987272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f08640043e0 con 0x7f086c108930 2024-04-25T14:09:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 Processor -- start 2024-04-25T14:09:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 -- start start 2024-04-25T14:09:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41156a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b41156ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b41156ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b4114ced80 con 0x55b4111c8390 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b411488a20 con 0x55b4111a76b0 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0878d86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b4114cad40 con 0x55b41119f4d0 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41156a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41156a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37736/0 (socket says 172.21.15.78:37736) 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 -- 172.21.15.78:0/632681964 learned_addr learned my addr 172.21.15.78:0/632681964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086b7fe640 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b41156ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 -- 172.21.15.78:0/632681964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 msgr2=0x55b41156ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b41156ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 -- 172.21.15.78:0/632681964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 msgr2=0x55b41156ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b41156ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f0868ff9640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08540afaf0 0x7f08540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086affd640 1 -- 172.21.15.78:0/632681964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b411493c30 con 0x55b4111c8390 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.828+0000 7f086bfff640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08540afaf0 0x7f08540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f086affd640 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41156a810 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f086000b790 tx=0x7f086000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639e7f96e71c936e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f0851ffb640 1 -- 172.21.15.78:0/632681964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086000bea0 con 0x55b4111c8390 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f0868ff9640 1 -- 172.21.15.78:0/3797987272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0864100300 con 0x7f086c108930 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f0851ffb640 1 -- 172.21.15.78:0/632681964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0860016030 con 0x55b4111c8390 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f0851ffb640 1 -- 172.21.15.78:0/632681964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004d70 con 0x55b4111c8390 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f0878d86740 1 -- 172.21.15.78:0/632681964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 msgr2=0x55b41156a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f086bfff640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08540afaf0 0x7f08540b2000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f08640c84d0 tx=0x7f08640f6f20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.829+0000 7f0878d86740 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41156a810 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f086000b790 tx=0x7f086000bc50 comp rx=0 tx=0).stop 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- 172.21.15.78:0/632681964 shutdown_connections 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b41156ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b41156ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 --2- 172.21.15.78:0/632681964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41156a810 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- 172.21.15.78:0/632681964 >> 172.21.15.78:0/632681964 conn(0x55b4111bc4e0 msgr2=0x55b411566020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- 172.21.15.78:0/632681964 shutdown_connections 2024-04-25T14:09:32.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- 172.21.15.78:0/632681964 wait complete. 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- start start 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 Processor -- start 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b411574560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411574aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41157ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b41156c4d0 con 0x55b4111c8390 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b411455dc0 con 0x55b4111a76b0 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b41156c650 con 0x55b41119f4d0 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.830+0000 7f0878d86740 10 client.0 _ll_register_callbacks cb 0x55b411471600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41157ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41157ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37740/0 (socket says 172.21.15.78:37740) 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 -- 172.21.15.78:0/1453270709 learned_addr learned my addr 172.21.15.78:0/1453270709 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086b7fe640 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411574aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b411574560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 -- 172.21.15.78:0/1453270709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 msgr2=0x55b411574560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b411574560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 -- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 msgr2=0x55b411574aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411574aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 -- 172.21.15.78:0/1453270709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4111e03f0 con 0x55b4111c8390 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f086affd640 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41157ba00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0860004150 tx=0x7f086000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e838ddaa39e1d17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f084affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f084affd640 1 -- 172.21.15.78:0/1453270709 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004f30 con 0x55b4111c8390 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f084affd640 1 -- 172.21.15.78:0/1453270709 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f08600050d0 con 0x55b4111c8390 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0860016030 con 0x55b4111c8390 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f084affd640 1 -- 172.21.15.78:0/1453270709 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0860031050 con 0x55b4111c8390 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f0878d86740 10 client.18480 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.831+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b4112aa7d0 con 0x55b4111c8390 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f084affd640 1 -- 172.21.15.78:0/1453270709 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0860016030 con 0x55b4111c8390 2024-04-25T14:09:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f084affd640 1 -- 172.21.15.78:0/1453270709 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f086001ba40 con 0x55b4111c8390 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f0878d86740 10 client.18480 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f0878d86740 10 client.18480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f0878d86740 20 client.18480 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f0878d86740 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b411186f60 0x55b411582080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f0878d86740 4 client.18480 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4114dc990 con 0x55b411186f60 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f086b7fe640 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b411186f60 0x55b411582080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f084affd640 10 client.18480 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.832+0000 7f086b7fe640 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b411186f60 0x55b411582080 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833b111e28d79ed3 server_cookie=d5f657ebe8369a2e in_seq=0 out_seq=0 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.833+0000 7f084affd640 1 -- 172.21.15.78:0/1453270709 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4114dc990 con 0x55b411186f60 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.833+0000 7f084affd640 10 client.18480 handle_command_reply: tid=0 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.833+0000 7f0878d86740 1 client.18480 shutdown 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 msgr2=0x55b41157ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41157ba00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f0860004150 tx=0x7f086000b5b0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 shutdown_connections 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4111c8390 0x55b41157ba00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411574aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b411186f60 0x55b411582080 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f084affd640 0 client.18480 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 --2- 172.21.15.78:0/1453270709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41119f4d0 0x55b411574560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 >> 172.21.15.78:0/1453270709 conn(0x55b4111bc4e0 msgr2=0x55b411568f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 shutdown_connections 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 1 -- 172.21.15.78:0/1453270709 wait complete. 2024-04-25T14:09:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.834+0000 7f0878d86740 20 client.18480 trim_cache size 0 max 16384 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 Processor -- start 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 -- start start 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b411186f60 0x55b4115733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b41119f4d0 0x55b41157b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4111a76b0 0x55b411564610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1ac9f0 con 0x55b411186f60 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1aa2c0 con 0x55b41119f4d0 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f0878d86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a8110 con 0x55b4111a76b0 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.847+0000 7f086affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b411186f60 0x55b4115733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b411186f60 0x55b4115733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37742/0 (socket says 172.21.15.78:37742) 2024-04-25T14:09:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 -- 172.21.15.78:0/2096952387 learned_addr learned my addr 172.21.15.78:0/2096952387 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086b7fe640 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b41119f4d0 0x55b41157b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 -- 172.21.15.78:0/2096952387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4111a76b0 msgr2=0x55b411564610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4111a76b0 0x55b411564610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 -- 172.21.15.78:0/2096952387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b41119f4d0 msgr2=0x55b41157b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b41119f4d0 0x55b41157b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 -- 172.21.15.78:0/2096952387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41156c7a0 con 0x55b411186f60 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f086affd640 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b411186f60 0x55b4115733c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f086000b3f0 tx=0x7f086000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02a842badef38f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f08517fa640 1 -- 172.21.15.78:0/2096952387 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086000bea0 con 0x55b411186f60 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f08517fa640 1 -- 172.21.15.78:0/2096952387 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086003a070 con 0x55b411186f60 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.848+0000 7f08517fa640 1 -- 172.21.15.78:0/2096952387 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004a00 con 0x55b411186f60 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 -- 172.21.15.78:0/2096952387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b411186f60 msgr2=0x55b4115733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b411186f60 0x55b4115733c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f086000b3f0 tx=0x7f086000b4c0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 -- 172.21.15.78:0/2096952387 shutdown_connections 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4111a76b0 0x55b411564610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b41119f4d0 0x55b41157b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 --2- 172.21.15.78:0/2096952387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b411186f60 0x55b4115733c0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 -- 172.21.15.78:0/2096952387 >> 172.21.15.78:0/2096952387 conn(0x55b4111bc4e0 msgr2=0x55b411572cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 -- 172.21.15.78:0/2096952387 shutdown_connections 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 -- 172.21.15.78:0/2096952387 wait complete. 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 -- start start 2024-04-25T14:09:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.849+0000 7f0878d86740 1 Processor -- start 2024-04-25T14:09:32.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b411186f60 0x55b411578c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b41119f4d0 0x55b411579170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411579fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a8780 con 0x55b41119f4d0 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a9bf0 con 0x55b4111a76b0 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f086c1a8e50 con 0x55b411186f60 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 10 client.0 _ll_register_callbacks cb 0x7f086000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b41119f4d0 0x55b411579170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b41119f4d0 0x55b411579170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37756/0 (socket says 172.21.15.78:37756) 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 -- 172.21.15.78:0/1444610473 learned_addr learned my addr 172.21.15.78:0/1444610473 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086affd640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b411186f60 0x55b411578c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086affd640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411579fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 -- 172.21.15.78:0/1444610473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b411186f60 msgr2=0x55b411578c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b411186f60 0x55b411578c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 -- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 msgr2=0x55b411579fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411579fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 -- 172.21.15.78:0/1444610473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41156fce0 con 0x55b41119f4d0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086affd640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b411186f60 0x55b411578c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f086b7fe640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b41119f4d0 0x55b411579170 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f085c004700 tx=0x7f085c0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f2d705743d5fef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f084affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f084affd640 1 -- 172.21.15.78:0/1444610473 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c005220 con 0x55b41119f4d0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f084affd640 1 -- 172.21.15.78:0/1444610473 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f085c0053c0 con 0x55b41119f4d0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.850+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086003a070 con 0x55b41119f4d0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f084affd640 1 -- 172.21.15.78:0/1444610473 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f085c0055c0 con 0x55b41119f4d0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 10 client.18486 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b41157b750 con 0x55b41119f4d0 2024-04-25T14:09:32.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f084affd640 1 -- 172.21.15.78:0/1444610473 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086003a070 con 0x55b41119f4d0 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f084affd640 1 -- 172.21.15.78:0/1444610473 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f085c004b20 con 0x55b41119f4d0 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 10 client.18486 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 10 client.18486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 20 client.18486 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4111c8390 0x55b4115ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 4 client.18486 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4114d6800 con 0x55b4111c8390 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.851+0000 7f086affd640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4111c8390 0x55b4115ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f086affd640 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4111c8390 0x55b4115ec5f0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37eb24137d6f24cb server_cookie=ffd94056a0bcf6b9 in_seq=0 out_seq=0 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f084affd640 10 client.18486 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f084affd640 1 -- 172.21.15.78:0/1444610473 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4114d6800 con 0x55b4111c8390 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f084affd640 10 client.18486 handle_command_reply: tid=0 2024-04-25T14:09:32.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 client.18486 shutdown 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b41119f4d0 msgr2=0x55b411579170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b41119f4d0 0x55b411579170 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f085c004700 tx=0x7f085c0045b0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 shutdown_connections 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4111a76b0 0x55b411579fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b41119f4d0 0x55b411579170 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4111c8390 0x55b4115ec5f0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 --2- 172.21.15.78:0/1444610473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b411186f60 0x55b411578c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f084affd640 0 client.18486 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.852+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 >> 172.21.15.78:0/1444610473 conn(0x55b4111bc4e0 msgr2=0x55b411568e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 shutdown_connections 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0878d86740 1 -- 172.21.15.78:0/1444610473 wait complete. 2024-04-25T14:09:32.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0878d86740 20 client.18486 trim_cache size 0 max 16384 2024-04-25T14:09:32.860 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:09:32.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08540afaf0 msgr2=0x7f08540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08540afaf0 0x7f08540b2000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f08640c84d0 tx=0x7f08640f6f20 comp rx=0 tx=0).stop 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 msgr2=0x7f086c1a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c1a0590 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f08640097b0 tx=0x7f0864003c20 comp rx=0 tx=0).stop 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 shutdown_connections 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f08540afaf0 0x7f08540b2000 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f086c108930 0x7f086c1a0590 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f086c103c30 0x7f086c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 --2- 172.21.15.78:0/3797987272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f086c1029f0 0x7f086c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 >> 172.21.15.78:0/3797987272 conn(0x7f086c0fdbc0 msgr2=0x7f086c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 shutdown_connections 2024-04-25T14:09:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:32.853+0000 7f0871ed8640 1 -- 172.21.15.78:0/3797987272 wait complete. 2024-04-25T14:09:33.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:33 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3207431514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:09:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:33 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3207431514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:09:33.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:33 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3207431514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:09:33.865 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:09:33.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:09:33.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:09:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 -- 172.21.15.78:0/3131363562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128108920 msgr2=0x7f712810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 --2- 172.21.15.78:0/3131363562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128108920 0x7f712810ad40 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7120009ca0 tx=0x7f712002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 -- 172.21.15.78:0/3131363562 shutdown_connections 2024-04-25T14:09:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 --2- 172.21.15.78:0/3131363562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128108920 0x7f712810ad40 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 --2- 172.21.15.78:0/3131363562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128103c20 0x7f7128104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 --2- 172.21.15.78:0/3131363562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71281029e0 0x7f71281036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 -- 172.21.15.78:0/3131363562 >> 172.21.15.78:0/3131363562 conn(0x7f71280fdbd0 msgr2=0x7f7128100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 -- 172.21.15.78:0/3131363562 shutdown_connections 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.987+0000 7f712fc27640 1 -- 172.21.15.78:0/3131363562 wait complete. 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 Processor -- start 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 -- start start 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71281029e0 0x7f712806a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128103c20 0x7f712819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128108920 0x7f71281a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f712810e510 con 0x7f7128103c20 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f712810e390 con 0x7f7128108920 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712fc27640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f712810e690 con 0x7f71281029e0 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128103c20 0x7f712819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128103c20 0x7f712819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37778/0 (socket says 172.21.15.78:37778) 2024-04-25T14:09:33.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.988+0000 7f712d1c2640 1 -- 172.21.15.78:0/2426558443 learned_addr learned my addr 172.21.15.78:0/2426558443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712e1c4640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128108920 0x7f71281a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712d9c3640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71281029e0 0x7f712806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712d1c2640 1 -- 172.21.15.78:0/2426558443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71281029e0 msgr2=0x7f712806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712d1c2640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71281029e0 0x7f712806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712d1c2640 1 -- 172.21.15.78:0/2426558443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128108920 msgr2=0x7f71281a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712d1c2640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128108920 0x7f71281a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712d1c2640 1 -- 172.21.15.78:0/2426558443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71281a2030 con 0x7f7128103c20 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712d1c2640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128103c20 0x7f712819f260 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f711800d950 tx=0x7f711800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f711effd640 1 -- 172.21.15.78:0/2426558443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118017070 con 0x7f7128103c20 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f711effd640 1 -- 172.21.15.78:0/2426558443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118009c30 con 0x7f7128103c20 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71281a2320 con 0x7f7128103c20 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f711effd640 1 -- 172.21.15.78:0/2426558443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711800f620 con 0x7f7128103c20 2024-04-25T14:09:33.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.989+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71281a2870 con 0x7f7128103c20 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.992+0000 7f7136bb8740 1 Processor -- start 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f7136bb8740 1 -- start start 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f711effd640 1 -- 172.21.15.78:0/2426558443 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7118010460 con 0x7f7128103c20 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d18920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f7136bb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e754c76b00 con 0x55e75494f730 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f7136bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e754c12920 con 0x55e754947550 2024-04-25T14:09:33.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f7136bb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e754c72ac0 con 0x55e754970410 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d18920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d18920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37790/0 (socket says 172.21.15.78:37790) 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 -- 172.21.15.78:0/2820493998 learned_addr learned my addr 172.21.15.78:0/2820493998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712e1c4640 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d9c3640 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 -- 172.21.15.78:0/2820493998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 msgr2=0x55e754d114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 -- 172.21.15.78:0/2820493998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 msgr2=0x55e754d11a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d11a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d1c2640 1 -- 172.21.15.78:0/2820493998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e754a507e0 con 0x55e75494f730 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.993+0000 7f712d9c3640 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.994+0000 7f712d1c2640 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d18920 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f71180e4c10 tx=0x7f71180e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9ab9ee5bdf7c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.994+0000 7f711effd640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f71140afaf0 0x7f71140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:33.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.994+0000 7f7113fff640 1 -- 172.21.15.78:0/2820493998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180ed070 con 0x55e75494f730 2024-04-25T14:09:33.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.994+0000 7f712e1c4640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f71140afaf0 0x7f71140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:33.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.994+0000 7f711effd640 1 -- 172.21.15.78:0/2426558443 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f71180de8a0 con 0x7f7128103c20 2024-04-25T14:09:33.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.994+0000 7f7113fff640 1 -- 172.21.15.78:0/2820493998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71180a6880 con 0x55e75494f730 2024-04-25T14:09:33.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.994+0000 7f7113fff640 1 -- 172.21.15.78:0/2820493998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180a66c0 con 0x55e75494f730 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 -- 172.21.15.78:0/2820493998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 msgr2=0x55e754d18920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d18920 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f71180e4c10 tx=0x7f71180e6690 comp rx=0 tx=0).stop 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f712e1c4640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f71140afaf0 0x7f71140b2000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55e754d12670 tx=0x7f7120009450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 -- 172.21.15.78:0/2820493998 shutdown_connections 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d18920 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 --2- 172.21.15.78:0/2820493998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 -- 172.21.15.78:0/2820493998 >> 172.21.15.78:0/2820493998 conn(0x55e754964560 msgr2=0x55e754d0ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 -- 172.21.15.78:0/2820493998 shutdown_connections 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 -- 172.21.15.78:0/2820493998 wait complete. 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 -- start start 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.995+0000 7f7136bb8740 1 Processor -- start 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d1b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d1b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f7136bb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e754d19370 con 0x55e75494f730 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f7136bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e754c105d0 con 0x55e754947550 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f7136bb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e754d194f0 con 0x55e754970410 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f7136bb8740 10 client.0 _ll_register_callbacks cb 0x55e754bfe080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d1b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d1b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37798/0 (socket says 172.21.15.78:37798) 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 -- 172.21.15.78:0/1710017305 learned_addr learned my addr 172.21.15.78:0/1710017305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712e1c4640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d9c3640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d1b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 -- 172.21.15.78:0/1710017305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 msgr2=0x55e754d22760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d22760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 -- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 msgr2=0x55e754d1b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d1b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 -- 172.21.15.78:0/1710017305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e754ab0f70 con 0x55e75494f730 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d9c3640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d1b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f712d1c2640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d1b870 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f71180e41b0 tx=0x7f711800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad07ff831c2ed8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.996+0000 7f71117fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f71117fa640 1 -- 172.21.15.78:0/1710017305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118020070 con 0x55e75494f730 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f71117fa640 1 -- 172.21.15.78:0/1710017305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f71180a3360 con 0x55e75494f730 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71180a6880 con 0x55e75494f730 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f71117fa640 1 -- 172.21.15.78:0/1710017305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f71180ed070 con 0x55e75494f730 2024-04-25T14:09:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 10 client.18498 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e754948170 con 0x55e75494f730 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f71117fa640 1 -- 172.21.15.78:0/1710017305 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71180a6880 con 0x55e75494f730 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f71117fa640 1 -- 172.21.15.78:0/1710017305 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f71180a3000 con 0x55e75494f730 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 10 client.18498 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 10 client.18498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 20 client.18498 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e75492f1f0 0x55e754d28820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 4 client.18498 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e754c84710 con 0x55e75492f1f0 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.997+0000 7f712d9c3640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e75492f1f0 0x55e754d28820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.998+0000 7f712d9c3640 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e75492f1f0 0x55e754d28820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6e55a5edbc890d server_cookie=9d746c6817254a0 in_seq=0 out_seq=0 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.998+0000 7f71117fa640 10 client.18498 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f71117fa640 1 -- 172.21.15.78:0/1710017305 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e754c84710 con 0x55e75492f1f0 2024-04-25T14:09:34.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f71117fa640 10 client.18498 handle_command_reply: tid=0 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 client.18498 shutdown 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 msgr2=0x55e754d1b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d1b870 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f71180e41b0 tx=0x7f711800bf60 comp rx=0 tx=0).stop 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 shutdown_connections 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e75492f1f0 0x55e754d28820 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754970410 0x55e754d22760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f71117fa640 0 client.18498 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d1b870 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1710017305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e754947550 0x55e754d1b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 >> 172.21.15.78:0/1710017305 conn(0x55e754964560 msgr2=0x55e754d0bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 shutdown_connections 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 1 -- 172.21.15.78:0/1710017305 wait complete. 2024-04-25T14:09:34.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:33.999+0000 7f7136bb8740 20 client.18498 trim_cache size 0 max 16384 2024-04-25T14:09:34.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 Processor -- start 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 -- start start 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 0x55e754d1a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e754947550 0x55e754d2cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e75494f730 0x55e754d2d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71281adb80 con 0x55e754947550 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71281ab450 con 0x55e75492f1f0 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f7136bb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71281a92a0 con 0x55e75494f730 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e754947550 0x55e754d2cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e754947550 0x55e754d2cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37802/0 (socket says 172.21.15.78:37802) 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 -- 172.21.15.78:0/4109319639 learned_addr learned my addr 172.21.15.78:0/4109319639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 -- 172.21.15.78:0/4109319639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e75494f730 msgr2=0x55e754d2d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 --2- 172.21.15.78:0/4109319639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e75494f730 0x55e754d2d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 -- 172.21.15.78:0/4109319639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 msgr2=0x55e754d1a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 --2- 172.21.15.78:0/4109319639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 0x55e754d1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.017+0000 7f712d1c2640 1 -- 172.21.15.78:0/4109319639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e754d16180 con 0x55e754947550 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f712d1c2640 1 --2- 172.21.15.78:0/4109319639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e754947550 0x55e754d2cf00 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f71180a6d00 tx=0x7f71180a6b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea5e9c5a296fadc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f71137fe640 1 -- 172.21.15.78:0/4109319639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118020070 con 0x55e754947550 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f71137fe640 1 -- 172.21.15.78:0/4109319639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71180e4a90 con 0x55e754947550 2024-04-25T14:09:34.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f71137fe640 1 -- 172.21.15.78:0/4109319639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180a64e0 con 0x55e754947550 2024-04-25T14:09:34.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f7136bb8740 1 -- 172.21.15.78:0/4109319639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e754947550 msgr2=0x55e754d2cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f7136bb8740 1 --2- 172.21.15.78:0/4109319639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e754947550 0x55e754d2cf00 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f71180a6d00 tx=0x7f71180a6b40 comp rx=0 tx=0).stop 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f7136bb8740 1 -- 172.21.15.78:0/4109319639 shutdown_connections 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f7136bb8740 1 --2- 172.21.15.78:0/4109319639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e75494f730 0x55e754d2d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f7136bb8740 1 --2- 172.21.15.78:0/4109319639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e754947550 0x55e754d2cf00 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.018+0000 7f7136bb8740 1 --2- 172.21.15.78:0/4109319639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 0x55e754d1a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 -- 172.21.15.78:0/4109319639 >> 172.21.15.78:0/4109319639 conn(0x55e754964560 msgr2=0x55e754d198f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 -- 172.21.15.78:0/4109319639 shutdown_connections 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 -- 172.21.15.78:0/4109319639 wait complete. 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 -- start start 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 Processor -- start 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 0x55e754d1f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754947550 0x55e754d1fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d20be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71281a9910 con 0x55e75494f730 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71281aad80 con 0x55e75492f1f0 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71281a9fe0 con 0x55e754947550 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f7136bb8740 10 client.0 _ll_register_callbacks cb 0x7f71180deba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f712e1c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d20be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f712e1c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d20be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37808/0 (socket says 172.21.15.78:37808) 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f712e1c4640 1 -- 172.21.15.78:0/1207222249 learned_addr learned my addr 172.21.15.78:0/1207222249 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.019+0000 7f712d9c3640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 0x55e754d1f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712d1c2640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754947550 0x55e754d1fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712e1c4640 1 -- 172.21.15.78:0/1207222249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754947550 msgr2=0x55e754d1fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712e1c4640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754947550 0x55e754d1fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712e1c4640 1 -- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 msgr2=0x55e754d1f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712e1c4640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 0x55e754d1f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712e1c4640 1 -- 172.21.15.78:0/1207222249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e754d211a0 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712d1c2640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754947550 0x55e754d1fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f712e1c4640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d20be0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7120009ca0 tx=0x7f71200039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0706f3529853f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f71117fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f71117fa640 1 -- 172.21.15.78:0/1207222249 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120004250 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f71117fa640 1 -- 172.21.15.78:0/1207222249 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7120042030 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71180e4a90 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f71117fa640 1 -- 172.21.15.78:0/1207222249 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f71200043f0 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f7136bb8740 10 client.18504 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e754d90cd0 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.020+0000 7f71117fa640 1 -- 172.21.15.78:0/1207222249 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71180e4a90 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f71117fa640 1 -- 172.21.15.78:0/1207222249 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f71200305c0 con 0x55e75494f730 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f7136bb8740 10 client.18504 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:34.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f7136bb8740 10 client.18504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f7136bb8740 20 client.18504 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e754970410 0x55e754d93720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f7136bb8740 4 client.18504 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e754c7e580 con 0x55e754970410 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f712d9c3640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e754970410 0x55e754d93720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f71117fa640 10 client.18504 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.021+0000 7f712d9c3640 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e754970410 0x55e754d93720 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2671a05b41e79609 server_cookie=bab946cfc8cdb4d9 in_seq=0 out_seq=0 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.022+0000 7f71117fa640 1 -- 172.21.15.78:0/1207222249 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e754c7e580 con 0x55e754970410 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.022+0000 7f71117fa640 10 client.18504 handle_command_reply: tid=0 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.022+0000 7f7136bb8740 1 client.18504 shutdown 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.022+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 msgr2=0x55e754d20be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.022+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d20be0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7120009ca0 tx=0x7f71200039c0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 shutdown_connections 2024-04-25T14:09:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e75494f730 0x55e754d20be0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55e754970410 0x55e754d93720 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e754947550 0x55e754d1fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 --2- 172.21.15.78:0/1207222249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e75492f1f0 0x55e754d1f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f71117fa640 0 client.18504 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 >> 172.21.15.78:0/1207222249 conn(0x55e754964560 msgr2=0x55e754d0fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 shutdown_connections 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 1 -- 172.21.15.78:0/1207222249 wait complete. 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f7136bb8740 20 client.18504 trim_cache size 0 max 16384 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:09:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:09:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "25fb51bc-d928-43ba-8ea7-0df25fa0f0f2", 2024-04-25T14:09:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:09:34.029 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:09:34.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f71140afaf0 msgr2=0x7f71140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f71140afaf0 0x7f71140b2000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55e754d12670 tx=0x7f7120009450 comp rx=0 tx=0).stop 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128103c20 msgr2=0x7f712819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128103c20 0x7f712819f260 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f711800d950 tx=0x7f711800de10 comp rx=0 tx=0).stop 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 shutdown_connections 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7128108920 0x7f71281a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f71140afaf0 0x7f71140b2000 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7128103c20 0x7f712819f260 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 --2- 172.21.15.78:0/2426558443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71281029e0 0x7f712806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 >> 172.21.15.78:0/2426558443 conn(0x7f71280fdbd0 msgr2=0x7f7128104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 shutdown_connections 2024-04-25T14:09:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.023+0000 7f712fc27640 1 -- 172.21.15.78:0/2426558443 wait complete. 2024-04-25T14:09:34.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:09:34.034 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 -- 172.21.15.78:0/3616550976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 msgr2=0x7f7d28102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 --2- 172.21.15.78:0/3616550976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d28102e90 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009900 tx=0x7f7d2402a410 comp rx=0 tx=0).stop 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 -- 172.21.15.78:0/3616550976 shutdown_connections 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 --2- 172.21.15.78:0/3616550976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d281080d0 0x7f7d2810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 --2- 172.21.15.78:0/3616550976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d281033d0 0x7f7d281037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 --2- 172.21.15.78:0/3616550976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d28102e90 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 -- 172.21.15.78:0/3616550976 >> 172.21.15.78:0/3616550976 conn(0x7f7d280fd380 msgr2=0x7f7d280ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 -- 172.21.15.78:0/3616550976 shutdown_connections 2024-04-25T14:09:34.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 -- 172.21.15.78:0/3616550976 wait complete. 2024-04-25T14:09:34.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.144+0000 7f7d30067640 1 Processor -- start 2024-04-25T14:09:34.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d30067640 1 -- start start 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d30067640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d2819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d30067640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d281033d0 0x7f7d2819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d30067640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d281080d0 0x7f7d281a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d30067640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2810f690 con 0x7f7d28102190 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d30067640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2810f510 con 0x7f7d281080d0 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d30067640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2810f810 con 0x7f7d281033d0 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d2819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d2819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37828/0 (socket says 172.21.15.78:37828) 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 -- 172.21.15.78:0/308342879 learned_addr learned my addr 172.21.15.78:0/308342879 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2d602640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d281033d0 0x7f7d2819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2e604640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d281080d0 0x7f7d281a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 -- 172.21.15.78:0/308342879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d281033d0 msgr2=0x7f7d2819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d281033d0 0x7f7d2819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 -- 172.21.15.78:0/308342879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d281080d0 msgr2=0x7f7d281a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d281080d0 0x7f7d281a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 -- 172.21.15.78:0/308342879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d281a1da0 con 0x7f7d28102190 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d2de03640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d2819a260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7d24031440 tx=0x7f7d24003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.145+0000 7f7d1affd640 1 -- 172.21.15.78:0/308342879 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2403f070 con 0x7f7d28102190 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.146+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d281a2030 con 0x7f7d28102190 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.146+0000 7f7d1affd640 1 -- 172.21.15.78:0/308342879 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2403b460 con 0x7f7d28102190 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.146+0000 7f7d1affd640 1 -- 172.21.15.78:0/308342879 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24035030 con 0x7f7d28102190 2024-04-25T14:09:34.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.146+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d281a2570 con 0x7f7d28102190 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 Processor -- start 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d1affd640 1 -- 172.21.15.78:0/308342879 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7d24042070 con 0x7f7d28102190 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 -- start start 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9278a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9278fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9279c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdd91ddd80 con 0x55cdd8ed7390 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdd91978b0 con 0x55cdd8eae4d0 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.148+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cdd91d9d40 con 0x55cdd8eb66b0 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9278a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2de03640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9278fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9278a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37832/0 (socket says 172.21.15.78:37832) 2024-04-25T14:09:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2de03640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9278fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.78:33194/0 (socket says 172.21.15.78:33194) 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 -- 172.21.15.78:0/247976730 learned_addr learned my addr 172.21.15.78:0/247976730 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2d602640 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9279c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d1affd640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d140afaf0 0x7f7d140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 -- 172.21.15.78:0/247976730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 msgr2=0x55cdd9279c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9279c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 -- 172.21.15.78:0/247976730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 msgr2=0x55cdd9278fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9278fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 -- 172.21.15.78:0/247976730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd9168a40 con 0x55cdd8ed7390 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d1affd640 1 -- 172.21.15.78:0/308342879 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7d240ff5c0 con 0x7f7d28102190 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d140afaf0 0x7f7d140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d2e604640 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9278a60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009440 tx=0x7f7d2000e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21c5deee047ec82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d13fff640 1 -- 172.21.15.78:0/247976730 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20020070 con 0x55cdd8ed7390 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d13fff640 1 -- 172.21.15.78:0/247976730 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2001adb0 con 0x55cdd8ed7390 2024-04-25T14:09:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.149+0000 7f7d13fff640 1 -- 172.21.15.78:0/247976730 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20019b10 con 0x55cdd8ed7390 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 -- 172.21.15.78:0/247976730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 msgr2=0x55cdd9278a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d2e604640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d140afaf0 0x7f7d140b2000 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7d2001d000 tx=0x7f7d2001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9278a60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009440 tx=0x7f7d2000e780 comp rx=0 tx=0).stop 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 -- 172.21.15.78:0/247976730 shutdown_connections 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9279c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9278fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/247976730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9278a60 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 -- 172.21.15.78:0/247976730 >> 172.21.15.78:0/247976730 conn(0x55cdd8ecb4e0 msgr2=0x55cdd9274270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 -- 172.21.15.78:0/247976730 shutdown_connections 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.150+0000 7f7d36ff8740 1 -- 172.21.15.78:0/247976730 wait complete. 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 -- start start 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 Processor -- start 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9282930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9282e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9289dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdd927a780 con 0x55cdd8ed7390 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cdd9164d10 con 0x55cdd8eae4d0 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cdd927a900 con 0x55cdd8eb66b0 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d36ff8740 10 client.0 _ll_register_callbacks cb 0x55cdd9191210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d2e604640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9289dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d2e604640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9289dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37844/0 (socket says 172.21.15.78:37844) 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d2e604640 1 -- 172.21.15.78:0/2269742070 learned_addr learned my addr 172.21.15.78:0/2269742070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.151+0000 7f7d2de03640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9282930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2d602640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9282e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2e604640 1 -- 172.21.15.78:0/2269742070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 msgr2=0x55cdd9282e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2e604640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9282e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2e604640 1 -- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 msgr2=0x55cdd9282930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2e604640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9282930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2e604640 1 -- 172.21.15.78:0/2269742070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd8fb97d0 con 0x55cdd8ed7390 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2de03640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9282930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d2e604640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9289dd0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7d20012920 tx=0x7f7d20019c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3657ef6ab518b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d117fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d117fa640 1 -- 172.21.15.78:0/2269742070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20020040 con 0x55cdd8ed7390 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d117fa640 1 -- 172.21.15.78:0/2269742070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7d20019dc0 con 0x55cdd8ed7390 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2001adb0 con 0x55cdd8ed7390 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d117fa640 1 -- 172.21.15.78:0/2269742070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7d20019390 con 0x55cdd8ed7390 2024-04-25T14:09:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d36ff8740 10 client.18516 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cdd8eef3f0 con 0x55cdd8ed7390 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.152+0000 7f7d117fa640 1 -- 172.21.15.78:0/2269742070 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2001adb0 con 0x55cdd8ed7390 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d117fa640 1 -- 172.21.15.78:0/2269742070 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7d2000f040 con 0x55cdd8ed7390 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d36ff8740 10 client.18516 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d36ff8740 10 client.18516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d36ff8740 20 client.18516 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e95f60 0x55cdd9290050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d36ff8740 4 client.18516 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdd91eb990 con 0x55cdd8e95f60 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d2de03640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e95f60 0x55cdd9290050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d117fa640 10 client.18516 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.153+0000 7f7d2de03640 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e95f60 0x55cdd9290050 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0e3c6cbb0bb983 server_cookie=71465fd94165bdcf in_seq=0 out_seq=0 2024-04-25T14:09:34.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d117fa640 1 -- 172.21.15.78:0/2269742070 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cdd91eb990 con 0x55cdd8e95f60 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d117fa640 10 client.18516 handle_command_reply: tid=0 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 client.18516 shutdown 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 msgr2=0x55cdd9289dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9289dd0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7d20012920 tx=0x7f7d20019c30 comp rx=0 tx=0).stop 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 shutdown_connections 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9289dd0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e95f60 0x55cdd9290050 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9282e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/2269742070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9282930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d117fa640 0 client.18516 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 >> 172.21.15.78:0/2269742070 conn(0x55cdd8ecb4e0 msgr2=0x55cdd92733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 shutdown_connections 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.154+0000 7f7d36ff8740 1 -- 172.21.15.78:0/2269742070 wait complete. 2024-04-25T14:09:34.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.155+0000 7f7d36ff8740 20 client.18516 trim_cache size 0 max 16384 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 Processor -- start 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 -- start start 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9280f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9281440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9294800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281adb60 con 0x55cdd8ed7390 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281ab430 con 0x55cdd8eae4d0 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281a9280 con 0x55cdd8eb66b0 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d2e604640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9294800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d2e604640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9294800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37854/0 (socket says 172.21.15.78:37854) 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.172+0000 7f7d2e604640 1 -- 172.21.15.78:0/3892817508 learned_addr learned my addr 172.21.15.78:0/3892817508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2d602640 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9281440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2de03640 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9280f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2e604640 1 -- 172.21.15.78:0/3892817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 msgr2=0x55cdd9281440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2e604640 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9281440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2e604640 1 -- 172.21.15.78:0/3892817508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 msgr2=0x55cdd9280f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2e604640 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9280f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2e604640 1 -- 172.21.15.78:0/3892817508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd8eaf0f0 con 0x55cdd8ed7390 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2de03640 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9280f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2d602640 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9281440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d2e604640 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9294800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009f80 tx=0x7f7d20019820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2bdcb1969d4001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d137fe640 1 -- 172.21.15.78:0/3892817508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20020050 con 0x55cdd8ed7390 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d137fe640 1 -- 172.21.15.78:0/3892817508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2003e020 con 0x55cdd8ed7390 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.173+0000 7f7d137fe640 1 -- 172.21.15.78:0/3892817508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2001ab90 con 0x55cdd8ed7390 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 -- 172.21.15.78:0/3892817508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 msgr2=0x55cdd9294800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9294800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009f80 tx=0x7f7d20019820 comp rx=0 tx=0).stop 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 -- 172.21.15.78:0/3892817508 shutdown_connections 2024-04-25T14:09:34.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8ed7390 0x55cdd9294800 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8eb66b0 0x55cdd9281440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/3892817508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eae4d0 0x55cdd9280f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 -- 172.21.15.78:0/3892817508 >> 172.21.15.78:0/3892817508 conn(0x55cdd8ecb4e0 msgr2=0x55cdd928a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 -- 172.21.15.78:0/3892817508 shutdown_connections 2024-04-25T14:09:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 -- 172.21.15.78:0/3892817508 wait complete. 2024-04-25T14:09:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 -- start start 2024-04-25T14:09:34.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.174+0000 7f7d36ff8740 1 Processor -- start 2024-04-25T14:09:34.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8eae4d0 0x55cdd92f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eb66b0 0x55cdd92f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8ed7390 0x55cdd92fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281a98f0 con 0x55cdd8eae4d0 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281aad60 con 0x55cdd8eb66b0 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281a9fc0 con 0x55cdd8ed7390 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 10 client.0 _ll_register_callbacks cb 0x7f7d2001e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8eae4d0 0x55cdd92f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8eae4d0 0x55cdd92f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37858/0 (socket says 172.21.15.78:37858) 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 -- 172.21.15.78:0/620883013 learned_addr learned my addr 172.21.15.78:0/620883013 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2d602640 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eb66b0 0x55cdd92f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 -- 172.21.15.78:0/620883013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8ed7390 msgr2=0x55cdd92fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8ed7390 0x55cdd92fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 -- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eb66b0 msgr2=0x55cdd92f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eb66b0 0x55cdd92f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 -- 172.21.15.78:0/620883013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd919e4b0 con 0x55cdd8eae4d0 2024-04-25T14:09:34.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2d602640 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eb66b0 0x55cdd92f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d2de03640 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8eae4d0 0x55cdd92f2ef0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7d240fd0b0 tx=0x7f7d240ea040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ae4b2422d6cfd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d117fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d117fa640 1 -- 172.21.15.78:0/620883013 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24049040 con 0x55cdd8eae4d0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d117fa640 1 -- 172.21.15.78:0/620883013 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7d240c3d20 con 0x55cdd8eae4d0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2003e020 con 0x55cdd8eae4d0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d117fa640 1 -- 172.21.15.78:0/620883013 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7d240c73e0 con 0x55cdd8eae4d0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 10 client.18522 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.175+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55cdd92fb960 con 0x55cdd8eae4d0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d117fa640 1 -- 172.21.15.78:0/620883013 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2003e020 con 0x55cdd8eae4d0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d117fa640 1 -- 172.21.15.78:0/620883013 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7d2403b8d0 con 0x55cdd8eae4d0 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d36ff8740 10 client.18522 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d36ff8740 10 client.18522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d36ff8740 20 client.18522 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:34.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e6b3a0 0x55cdd92fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d36ff8740 4 client.18522 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cdd91e5800 con 0x55cdd8e6b3a0 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d2d602640 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e6b3a0 0x55cdd92fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d117fa640 10 client.18522 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.176+0000 7f7d2d602640 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e6b3a0 0x55cdd92fe2d0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6679125893644870 server_cookie=28ea64f4456ab7cc in_seq=0 out_seq=0 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d117fa640 1 -- 172.21.15.78:0/620883013 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cdd91e5800 con 0x55cdd8e6b3a0 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d117fa640 10 client.18522 handle_command_reply: tid=0 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 client.18522 shutdown 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8eae4d0 msgr2=0x55cdd92f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8eae4d0 0x55cdd92f2ef0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f7d240fd0b0 tx=0x7f7d240ea040 comp rx=0 tx=0).stop 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 shutdown_connections 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55cdd8e6b3a0 0x55cdd92fe2d0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cdd8ed7390 0x55cdd92fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cdd8eb66b0 0x55cdd92f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 --2- 172.21.15.78:0/620883013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdd8eae4d0 0x55cdd92f2ef0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d117fa640 0 client.18522 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 >> 172.21.15.78:0/620883013 conn(0x55cdd8ecb4e0 msgr2=0x55cdd9277070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 shutdown_connections 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 1 -- 172.21.15.78:0/620883013 wait complete. 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d36ff8740 20 client.18522 trim_cache size 0 max 16384 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:09:34.183 INFO:teuthology.orchestra.run.smithi078.stdout: "25fb51bc-d928-43ba-8ea7-0df25fa0f0f2": { 2024-04-25T14:09:34.184 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:09:34.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "25fb51bc-d928-43ba-8ea7-0df25fa0f0f2", 2024-04-25T14:09:34.184 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:09:34.184 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:09:34.184 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:09:34.184 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d140afaf0 msgr2=0x7f7d140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d140afaf0 0x7f7d140b2000 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7d2001d000 tx=0x7f7d2001e000 comp rx=0 tx=0).stop 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 msgr2=0x7f7d2819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d2819a260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7d24031440 tx=0x7f7d24003d50 comp rx=0 tx=0).stop 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 shutdown_connections 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d281080d0 0x7f7d281a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d281033d0 0x7f7d2819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d140afaf0 0x7f7d140b2000 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.177+0000 7f7d30067640 1 --2- 172.21.15.78:0/308342879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28102190 0x7f7d2819a260 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.178+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 >> 172.21.15.78:0/308342879 conn(0x7f7d280fd380 msgr2=0x7f7d281040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.178+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 shutdown_connections 2024-04-25T14:09:34.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:09:34.178+0000 7f7d30067640 1 -- 172.21.15.78:0/308342879 wait complete. 2024-04-25T14:09:34.188 INFO:tasks.cephfs.filesystem:scrub status for tag:25fb51bc-d928-43ba-8ea7-0df25fa0f0f2 - {'path': '/', 'tag': '25fb51bc-d928-43ba-8ea7-0df25fa0f0f2', 'options': 'recursive,force'} 2024-04-25T14:09:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:34 smithi078 ceph-mon[29163]: pgmap v649: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:34 smithi129 ceph-mon[32569]: pgmap v649: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:34.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:34 smithi112 ceph-mon[32617]: pgmap v649: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 -- 172.21.15.78:0/2211516015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668102a20 msgr2=0x7f9668103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 --2- 172.21.15.78:0/2211516015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668102a20 0x7f9668103720 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802a410 comp rx=0 tx=0).stop 2024-04-25T14:09:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 -- 172.21.15.78:0/2211516015 shutdown_connections 2024-04-25T14:09:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 --2- 172.21.15.78:0/2211516015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9668108960 0x7f966810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 --2- 172.21.15.78:0/2211516015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9668103c60 0x7f9668104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 --2- 172.21.15.78:0/2211516015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668102a20 0x7f9668103720 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 -- 172.21.15.78:0/2211516015 >> 172.21.15.78:0/2211516015 conn(0x7f96680fdc10 msgr2=0x7f96681000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 -- 172.21.15.78:0/2211516015 shutdown_connections 2024-04-25T14:09:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.649+0000 7f966ccf2640 1 -- 172.21.15.78:0/2211516015 wait complete. 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 Processor -- start 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 -- start start 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9668102a20 0x7f966806ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668103c60 0x7f966819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9668108960 0x7f96681a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f966810e500 con 0x7f9668103c60 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f966810e380 con 0x7f9668108960 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966ccf2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f966810e680 con 0x7f9668102a20 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f965ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668103c60 0x7f966819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f965ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668103c60 0x7f966819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37880/0 (socket says 172.21.15.78:37880) 2024-04-25T14:09:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f965ffff640 1 -- 172.21.15.78:0/3312150565 learned_addr learned my addr 172.21.15.78:0/3312150565 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f9666d9d640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9668108960 0x7f96681a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.650+0000 7f966659c640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9668102a20 0x7f966806ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965ffff640 1 -- 172.21.15.78:0/3312150565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9668102a20 msgr2=0x7f966806ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965ffff640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9668102a20 0x7f966806ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965ffff640 1 -- 172.21.15.78:0/3312150565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9668108960 msgr2=0x7f96681a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965ffff640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9668108960 0x7f96681a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965ffff640 1 -- 172.21.15.78:0/3312150565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96681a1d70 con 0x7f9668103c60 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965ffff640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668103c60 0x7f966819efa0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f965000d950 tx=0x7f965000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965f7fe640 1 -- 172.21.15.78:0/3312150565 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650017070 con 0x7f9668103c60 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965f7fe640 1 -- 172.21.15.78:0/3312150565 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9650009c30 con 0x7f9668103c60 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96681a2060 con 0x7f9668103c60 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f965f7fe640 1 -- 172.21.15.78:0/3312150565 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000f620 con 0x7f9668103c60 2024-04-25T14:09:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.651+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96681a25b0 con 0x7f9668103c60 2024-04-25T14:09:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 Processor -- start 2024-04-25T14:09:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f965f7fe640 1 -- 172.21.15.78:0/3312150565 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9650010460 con 0x7f9668103c60 2024-04-25T14:09:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 -- start start 2024-04-25T14:09:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0742d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07434df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc07393460 con 0x55dc070642f0 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc0734d590 con 0x55dc0708d1b0 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.654+0000 7f9673ba2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc0738f420 con 0x55dc0706c4d0 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37892/0 (socket says 172.21.15.78:37892) 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 -- 172.21.15.78:0/3092530724 learned_addr learned my addr 172.21.15.78:0/3092530724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f965ffff640 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07434df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f9666d9d640 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0742d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f965f7fe640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96540afae0 0x7f96540b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 -- 172.21.15.78:0/3092530724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 msgr2=0x55dc07434df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07434df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 -- 172.21.15.78:0/3092530724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 msgr2=0x55dc0742d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0742d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 -- 172.21.15.78:0/3092530724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0735a0d0 con 0x55dc070642f0 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f9666d9d640 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0742d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f9666d9d640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96540afae0 0x7f96540b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f965f7fe640 1 -- 172.21.15.78:0/3312150565 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f96500de7f0 con 0x7f9668103c60 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f966659c640 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742def0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f9658009230 tx=0x7f9658031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a01632dba299ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.655+0000 7f965cff9640 1 -- 172.21.15.78:0/3092530724 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658035030 con 0x55dc070642f0 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9666d9d640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96540afae0 0x7f96540b1ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9660009370 tx=0x7f9660004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f965cff9640 1 -- 172.21.15.78:0/3092530724 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658040070 con 0x55dc070642f0 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f965cff9640 1 -- 172.21.15.78:0/3092530724 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658003a20 con 0x55dc070642f0 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9673ba2740 1 -- 172.21.15.78:0/3092530724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 msgr2=0x55dc0742def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9673ba2740 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742def0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f9658009230 tx=0x7f9658031bc0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9673ba2740 1 -- 172.21.15.78:0/3092530724 shutdown_connections 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9673ba2740 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07434df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9673ba2740 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742def0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9673ba2740 1 --2- 172.21.15.78:0/3092530724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0742d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.656+0000 7f9673ba2740 1 -- 172.21.15.78:0/3092530724 >> 172.21.15.78:0/3092530724 conn(0x55dc07081300 msgr2=0x55dc07429300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 -- 172.21.15.78:0/3092530724 shutdown_connections 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 -- 172.21.15.78:0/3092530724 wait complete. 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 -- start start 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 Processor -- start 2024-04-25T14:09:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc070642f0 0x55dc074375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07437af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0708d1b0 0x55dc0743ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dc074357e0 con 0x55dc0708d1b0 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc0731ab60 con 0x55dc070642f0 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc07435960 con 0x55dc0706c4d0 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9673ba2740 10 client.0 _ll_register_callbacks cb 0x55dc0733f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0708d1b0 0x55dc0743ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0708d1b0 0x55dc0743ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37898/0 (socket says 172.21.15.78:37898) 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f9666d9d640 1 -- 172.21.15.78:0/2278945734 learned_addr learned my addr 172.21.15.78:0/2278945734 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.657+0000 7f966659c640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc070642f0 0x55dc074375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f965ffff640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07437af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9666d9d640 1 -- 172.21.15.78:0/2278945734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 msgr2=0x55dc07437af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9666d9d640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07437af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9666d9d640 1 -- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc070642f0 msgr2=0x55dc074375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9666d9d640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc070642f0 0x55dc074375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9666d9d640 1 -- 172.21.15.78:0/2278945734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc070a5210 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f965ffff640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07437af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9666d9d640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0708d1b0 0x55dc0743ea50 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f966001e700 tx=0x7f9660004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b20364bd34c3d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f964dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f964dffb640 1 -- 172.21.15.78:0/2278945734 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660012a80 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f964dffb640 1 -- 172.21.15.78:0/2278945734 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9660012c20 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9658040070 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f964dffb640 1 -- 172.21.15.78:0/2278945734 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9660012e20 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9673ba2740 10 client.18534 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55dc07171620 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.658+0000 7f964dffb640 1 -- 172.21.15.78:0/2278945734 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658040070 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f964dffb640 1 -- 172.21.15.78:0/2278945734 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9660030050 con 0x55dc0708d1b0 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f9673ba2740 10 client.18534 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f9673ba2740 10 client.18534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f9673ba2740 20 client.18534 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc0704bd80 0x55dc074453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f9673ba2740 4 client.18534 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc073a1070 con 0x55dc0704bd80 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f966659c640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc0704bd80 0x55dc074453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f964dffb640 10 client.18534 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.659+0000 7f966659c640 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc0704bd80 0x55dc074453d0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6f32a3b312040f server_cookie=9491b1bd00137b92 in_seq=0 out_seq=0 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f964dffb640 1 -- 172.21.15.78:0/2278945734 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc073a1070 con 0x55dc0704bd80 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f964dffb640 10 client.18534 handle_command_reply: tid=0 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 client.18534 shutdown 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0708d1b0 msgr2=0x55dc0743ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0708d1b0 0x55dc0743ea50 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f966001e700 tx=0x7f9660004ab0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 shutdown_connections 2024-04-25T14:09:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc0708d1b0 0x55dc0743ea50 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc0704bd80 0x55dc074453d0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc07437af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2278945734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc070642f0 0x55dc074375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f964dffb640 0 client.18534 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.660+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 >> 172.21.15.78:0/2278945734 conn(0x55dc07081300 msgr2=0x55dc0742c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.661+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 shutdown_connections 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.661+0000 7f9673ba2740 1 -- 172.21.15.78:0/2278945734 wait complete. 2024-04-25T14:09:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.661+0000 7f9673ba2740 20 client.18534 trim_cache size 0 max 16384 2024-04-25T14:09:35.667 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:35 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:09:35.667 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:35 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:35.667 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:35 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:09:35.668 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:35 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 Processor -- start 2024-04-25T14:09:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 -- start start 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc0743e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc07428bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96681adb20 con 0x55dc070642f0 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96681ab3f0 con 0x55dc0708d1b0 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f9673ba2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96681a9240 con 0x55dc0706c4d0 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37904/0 (socket says 172.21.15.78:37904) 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 -- 172.21.15.78:0/2727275584 learned_addr learned my addr 172.21.15.78:0/2727275584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 -- 172.21.15.78:0/2727275584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 msgr2=0x55dc0743e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 --2- 172.21.15.78:0/2727275584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc0743e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 -- 172.21.15.78:0/2727275584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 msgr2=0x55dc07428bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 --2- 172.21.15.78:0/2727275584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc07428bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 -- 172.21.15.78:0/2727275584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc074324c0 con 0x55dc070642f0 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.676+0000 7f966659c640 1 --2- 172.21.15.78:0/2727275584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742c810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8949ef0204af74f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f964ffff640 1 -- 172.21.15.78:0/2727275584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658057070 con 0x55dc070642f0 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f964ffff640 1 -- 172.21.15.78:0/2727275584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658040050 con 0x55dc070642f0 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f964ffff640 1 -- 172.21.15.78:0/2727275584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658035040 con 0x55dc070642f0 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 -- 172.21.15.78:0/2727275584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 msgr2=0x55dc0742c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2727275584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742c810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802a410 comp rx=0 tx=0).stop 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 -- 172.21.15.78:0/2727275584 shutdown_connections 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2727275584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc07428bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2727275584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc0743e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 --2- 172.21.15.78:0/2727275584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0742c810 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 -- 172.21.15.78:0/2727275584 >> 172.21.15.78:0/2727275584 conn(0x55dc07081300 msgr2=0x55dc0742c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 -- 172.21.15.78:0/2727275584 shutdown_connections 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.677+0000 7f9673ba2740 1 -- 172.21.15.78:0/2727275584 wait complete. 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 -- start start 2024-04-25T14:09:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 Processor -- start 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0743c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc0743c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0743d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96681a98b0 con 0x55dc070642f0 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96681aad20 con 0x55dc0708d1b0 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96681a9f80 con 0x55dc0706c4d0 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9673ba2740 10 client.0 _ll_register_callbacks cb 0x7f9660009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0743c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0743c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37908/0 (socket says 172.21.15.78:37908) 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 -- 172.21.15.78:0/683167869 learned_addr learned my addr 172.21.15.78:0/683167869 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f965ffff640 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc0743c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 -- 172.21.15.78:0/683167869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 msgr2=0x55dc0743c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc0743c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 -- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 msgr2=0x55dc0743d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f9666d9d640 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0743d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0743d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 -- 172.21.15.78:0/683167869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0743db80 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f966659c640 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0743c230 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9658005f70 tx=0x7f9658009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a151a231e79ae978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.678+0000 7f964dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f964dffb640 1 -- 172.21.15.78:0/683167869 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658057070 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f964dffb640 1 -- 172.21.15.78:0/683167869 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9658004040 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9658040050 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f964dffb640 1 -- 172.21.15.78:0/683167869 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9658035040 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 10 client.18540 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55dc074ad780 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f964dffb640 1 -- 172.21.15.78:0/683167869 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658040050 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f964dffb640 1 -- 172.21.15.78:0/683167869 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f965804e070 con 0x55dc070642f0 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 10 client.18540 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 10 client.18540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 20 client.18540 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc070211c0 0x55dc074afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 4 client.18540 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.679+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc0739aee0 con 0x55dc070211c0 2024-04-25T14:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.680+0000 7f965ffff640 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc070211c0 0x55dc074afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.680+0000 7f964dffb640 10 client.18540 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.680+0000 7f965ffff640 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc070211c0 0x55dc074afc20 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3b409e2533e415 server_cookie=43caf866f5c9cee9 in_seq=0 out_seq=0 2024-04-25T14:09:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f964dffb640 1 -- 172.21.15.78:0/683167869 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc0739aee0 con 0x55dc070211c0 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f964dffb640 10 client.18540 handle_command_reply: tid=0 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 client.18540 shutdown 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 msgr2=0x55dc0743c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0743c230 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9658005f70 tx=0x7f9658009260 comp rx=0 tx=0).stop 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 shutdown_connections 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc0708d1b0 0x55dc0743d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55dc070211c0 0x55dc074afc20 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc0706c4d0 0x55dc0743c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 --2- 172.21.15.78:0/683167869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dc070642f0 0x55dc0743c230 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f964dffb640 0 client.18540 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 >> 172.21.15.78:0/683167869 conn(0x55dc07081300 msgr2=0x55dc07449bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 shutdown_connections 2024-04-25T14:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.695+0000 7f9673ba2740 1 -- 172.21.15.78:0/683167869 wait complete. 2024-04-25T14:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f9673ba2740 20 client.18540 trim_cache size 0 max 16384 2024-04-25T14:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:09:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96540afae0 msgr2=0x7f96540b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96540afae0 0x7f96540b1ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9660009370 tx=0x7f9660004a40 comp rx=0 tx=0).stop 2024-04-25T14:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668103c60 msgr2=0x7f966819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668103c60 0x7f966819efa0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f965000d950 tx=0x7f965000de10 comp rx=0 tx=0).stop 2024-04-25T14:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 shutdown_connections 2024-04-25T14:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9668108960 0x7f96681a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f96540afae0 0x7f96540b1ff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9668103c60 0x7f966819efa0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 --2- 172.21.15.78:0/3312150565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9668102a20 0x7f966806ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 >> 172.21.15.78:0/3312150565 conn(0x7f96680fdc10 msgr2=0x7f9668104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 shutdown_connections 2024-04-25T14:09:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:35.696+0000 7f966ccf2640 1 -- 172.21.15.78:0/3312150565 wait complete. 2024-04-25T14:09:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:35 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:09:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:35 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:35 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:09:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:35 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:35 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:09:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:35 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:35 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:09:35.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:35 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:09:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:36 smithi129 ceph-mon[32569]: pgmap v650: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:36 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:09:36.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:36 smithi078 ceph-mon[29163]: pgmap v650: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:36.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:36 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:09:36.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:36 smithi112 ceph-mon[32617]: pgmap v650: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:36.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:36 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:09:38.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:38 smithi129 ceph-mon[32569]: pgmap v651: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:38.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:38 smithi078 ceph-mon[29163]: pgmap v651: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:38.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:38 smithi112 ceph-mon[32617]: pgmap v651: 97 pgs: 97 active+clean; 3.1 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:40 smithi129 ceph-mon[32569]: pgmap v652: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:09:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:40 smithi078 ceph-mon[29163]: pgmap v652: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:09:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:40 smithi112 ceph-mon[32617]: pgmap v652: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 17 op/s 2024-04-25T14:09:41.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:41 smithi129 ceph-mon[32569]: pgmap v653: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:41 smithi078 ceph-mon[29163]: pgmap v653: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:41.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:41 smithi112 ceph-mon[32617]: pgmap v653: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:42.666 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:42 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:42 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:42.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:42 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:09:43.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:43 smithi129 ceph-mon[32569]: pgmap v654: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:43.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:43 smithi078 ceph-mon[29163]: pgmap v654: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:43.765 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:43 smithi112 ceph-mon[32617]: pgmap v654: 97 pgs: 97 active+clean; 3.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:45.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:45.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:45.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:44 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:44 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:09:45.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:44 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:09:46.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:45 smithi078 ceph-mon[29163]: pgmap v655: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:09:46.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:45 smithi112 ceph-mon[32617]: pgmap v655: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:09:46.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:45 smithi129 ceph-mon[32569]: pgmap v655: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-04-25T14:09:48.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:47 smithi078 ceph-mon[29163]: pgmap v656: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:48.265 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:47 smithi112 ceph-mon[32617]: pgmap v656: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:48.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:47 smithi129 ceph-mon[32569]: pgmap v656: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:09:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 -- 172.21.15.78:0/3760883518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 msgr2=0x7fb52c10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 --2- 172.21.15.78:0/3760883518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c10bea0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb524009ca0 tx=0x7fb52402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 -- 172.21.15.78:0/3760883518 shutdown_connections 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 --2- 172.21.15.78:0/3760883518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c10bea0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 --2- 172.21.15.78:0/3760883518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52c104d80 0x7fb52c105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 --2- 172.21.15.78:0/3760883518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c103b40 0x7fb52c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 -- 172.21.15.78:0/3760883518 >> 172.21.15.78:0/3760883518 conn(0x7fb52c0fed30 msgr2=0x7fb52c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 -- 172.21.15.78:0/3760883518 shutdown_connections 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.800+0000 7fb531f1f640 1 -- 172.21.15.78:0/3760883518 wait complete. 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 Processor -- start 2024-04-25T14:09:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 -- start start 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52c103b40 0x7fb52c077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c104d80 0x7fb52c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10f650 con 0x7fb52c109a80 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10f4d0 con 0x7fb52c104d80 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb531f1f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c10f7d0 con 0x7fb52c103b40 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56050/0 (socket says 172.21.15.78:56050) 2024-04-25T14:09:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb52bfff640 1 -- 172.21.15.78:0/96038694 learned_addr learned my addr 172.21.15.78:0/96038694 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb522ffd640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c104d80 0x7fb52c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.801+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52c103b40 0x7fb52c077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb52bfff640 1 -- 172.21.15.78:0/96038694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52c103b40 msgr2=0x7fb52c077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb52bfff640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52c103b40 0x7fb52c077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb52bfff640 1 -- 172.21.15.78:0/96038694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c104d80 msgr2=0x7fb52c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb52bfff640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c104d80 0x7fb52c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb52bfff640 1 -- 172.21.15.78:0/96038694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c076be0 con 0x7fb52c109a80 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb52bfff640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c076540 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb5240097b0 tx=0x7fb524003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb5297fa640 1 -- 172.21.15.78:0/96038694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524004240 con 0x7fb52c109a80 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c1a4280 con 0x7fb52c109a80 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb5297fa640 1 -- 172.21.15.78:0/96038694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52402f380 con 0x7fb52c109a80 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb5297fa640 1 -- 172.21.15.78:0/96038694 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402fc20 con 0x7fb52c109a80 2024-04-25T14:09:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.802+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a4680 con 0x7fb52c109a80 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.804+0000 7fb5297fa640 1 -- 172.21.15.78:0/96038694 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb5240043e0 con 0x7fb52c109a80 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.804+0000 7fb538dd3740 1 Processor -- start 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.804+0000 7fb538dd3740 1 -- start start 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c72140f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c72140fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721416df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb538dd3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c721375460 con 0x55c72104e4d0 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb538dd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c72132f590 con 0x55c72106f1b0 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb538dd3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c721371420 con 0x55c7210462f0 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721416df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721416df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56058/0 (socket says 172.21.15.78:56058) 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 -- 172.21.15.78:0/1909585970 learned_addr learned my addr 172.21.15.78:0/1909585970 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c72140fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c72140f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb5297fa640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5180afae0 0x7fb5180b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 -- 172.21.15.78:0/1909585970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 msgr2=0x55c72140fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c72140fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 -- 172.21.15.78:0/1909585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 msgr2=0x55c72140f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c72140f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb522ffd640 1 -- 172.21.15.78:0/1909585970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c72133c0d0 con 0x55c72104e4d0 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.805+0000 7fb52bfff640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5180afae0 0x7fb5180b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.806+0000 7fb522ffd640 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721416df0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb514011720 tx=0x7fb51400a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefd07482fe20c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.806+0000 7fb5297fa640 1 -- 172.21.15.78:0/96038694 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb524100620 con 0x7fb52c109a80 2024-04-25T14:09:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.806+0000 7fb521ffb640 1 -- 172.21.15.78:0/1909585970 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514005020 con 0x55c72104e4d0 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.806+0000 7fb521ffb640 1 -- 172.21.15.78:0/1909585970 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5140047a0 con 0x55c72104e4d0 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.806+0000 7fb521ffb640 1 -- 172.21.15.78:0/1909585970 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51400fa20 con 0x55c72104e4d0 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- 172.21.15.78:0/1909585970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 msgr2=0x55c721416df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb52bfff640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5180afae0 0x7fb5180b1ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb524005d80 tx=0x7fb524097550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721416df0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb514011720 tx=0x7fb51400a6e0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- 172.21.15.78:0/1909585970 shutdown_connections 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721416df0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c72140fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 --2- 172.21.15.78:0/1909585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c72140f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- 172.21.15.78:0/1909585970 >> 172.21.15.78:0/1909585970 conn(0x55c721063300 msgr2=0x55c72140b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- 172.21.15.78:0/1909585970 shutdown_connections 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- 172.21.15.78:0/1909585970 wait complete. 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- start start 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 Processor -- start 2024-04-25T14:09:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c721419670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721419bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c721420b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c721417840 con 0x55c72104e4d0 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c7212fcb60 con 0x55c72106f1b0 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.807+0000 7fb538dd3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c7214179c0 con 0x55c7210462f0 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb538dd3740 10 client.0 _ll_register_callbacks cb 0x55c7213212b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721419bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721419bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56066/0 (socket says 172.21.15.78:56066) 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 -- 172.21.15.78:0/3360622585 learned_addr learned my addr 172.21.15.78:0/3360622585 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb522ffd640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c721419670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb522ffd640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c721420b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 -- 172.21.15.78:0/3360622585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 msgr2=0x55c721419670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c721419670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 -- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 msgr2=0x55c721420b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c721420b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 -- 172.21.15.78:0/3360622585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c721087210 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb522ffd640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c721420b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721419bb0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00df40 tx=0x7fb51c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02e95b564163af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb50affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb50affd640 1 -- 172.21.15.78:0/3360622585 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c005220 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb50affd640 1 -- 172.21.15.78:0/3360622585 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb51c0053c0 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5140047a0 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb50affd640 1 -- 172.21.15.78:0/3360622585 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb51c0055c0 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb538dd3740 10 client.18552 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.808+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c721153620 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb50affd640 1 -- 172.21.15.78:0/3360622585 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5140047a0 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb50affd640 1 -- 172.21.15.78:0/3360622585 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb51c019030 con 0x55c72104e4d0 2024-04-25T14:09:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb538dd3740 10 client.18552 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb538dd3740 10 client.18552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb538dd3740 20 client.18552 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb538dd3740 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c72102dd80 0x55c721427460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb538dd3740 4 client.18552 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c721383070 con 0x55c72102dd80 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.809+0000 7fb522ffd640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c72102dd80 0x55c721427460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.810+0000 7fb50affd640 10 client.18552 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.810+0000 7fb522ffd640 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c72102dd80 0x55c721427460 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=486640494077d18 server_cookie=7c3b9f076b9e9658 in_seq=0 out_seq=0 2024-04-25T14:09:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.810+0000 7fb50affd640 1 -- 172.21.15.78:0/3360622585 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c721383070 con 0x55c72102dd80 2024-04-25T14:09:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.810+0000 7fb50affd640 10 client.18552 handle_command_reply: tid=0 2024-04-25T14:09:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 client.18552 shutdown 2024-04-25T14:09:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 msgr2=0x55c721419bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721419bb0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00df40 tx=0x7fb51c0128e0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 shutdown_connections 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72106f1b0 0x55c721420b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72104e4d0 0x55c721419bb0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c72102dd80 0x55c721427460 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 --2- 172.21.15.78:0/3360622585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7210462f0 0x55c721419670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb50affd640 0 client.18552 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 >> 172.21.15.78:0/3360622585 conn(0x55c721063300 msgr2=0x55c72140e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 shutdown_connections 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 1 -- 172.21.15.78:0/3360622585 wait complete. 2024-04-25T14:09:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.811+0000 7fb538dd3740 20 client.18552 trim_cache size 0 max 16384 2024-04-25T14:09:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 Processor -- start 2024-04-25T14:09:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 -- start start 2024-04-25T14:09:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7210462f0 0x55c72140e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72104e4d0 0x55c721420a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72106f1b0 0x55c72140e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1afc60 con 0x55c72106f1b0 2024-04-25T14:09:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ad530 con 0x55c7210462f0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb538dd3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ab380 con 0x55c72104e4d0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72106f1b0 0x55c72140e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72106f1b0 0x55c72140e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56078/0 (socket says 172.21.15.78:56078) 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 -- 172.21.15.78:0/2201482745 learned_addr learned my addr 172.21.15.78:0/2201482745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7210462f0 0x55c72140e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72104e4d0 0x55c721420a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 -- 172.21.15.78:0/2201482745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72104e4d0 msgr2=0x55c721420a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72104e4d0 0x55c721420a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 -- 172.21.15.78:0/2201482745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7210462f0 msgr2=0x55c72140e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7210462f0 0x55c72140e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 -- 172.21.15.78:0/2201482745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7214144e0 con 0x55c72106f1b0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72104e4d0 0x55c721420a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.826+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72106f1b0 0x55c72140e640 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb514001eb0 tx=0x7fb51401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4609cee1452b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb5217fa640 1 -- 172.21.15.78:0/2201482745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514032070 con 0x55c72106f1b0 2024-04-25T14:09:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb5217fa640 1 -- 172.21.15.78:0/2201482745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51400ae80 con 0x55c72106f1b0 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb5217fa640 1 -- 172.21.15.78:0/2201482745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514028040 con 0x55c72106f1b0 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 -- 172.21.15.78:0/2201482745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72106f1b0 msgr2=0x55c72140e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72106f1b0 0x55c72140e640 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb514001eb0 tx=0x7fb51401a040 comp rx=0 tx=0).stop 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 -- 172.21.15.78:0/2201482745 shutdown_connections 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c72106f1b0 0x55c72140e640 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72104e4d0 0x55c721420a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2201482745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c7210462f0 0x55c72140e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 -- 172.21.15.78:0/2201482745 >> 172.21.15.78:0/2201482745 conn(0x55c721063300 msgr2=0x55c72140f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.827+0000 7fb538dd3740 1 -- 172.21.15.78:0/2201482745 shutdown_connections 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 -- 172.21.15.78:0/2201482745 wait complete. 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 -- start start 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 Processor -- start 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7210462f0 0x55c72141e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72104e4d0 0x55c72141ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72106f1b0 0x55c72141fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ab9f0 con 0x55c7210462f0 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ace60 con 0x55c72104e4d0 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ac0c0 con 0x55c72106f1b0 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb538dd3740 10 client.0 _ll_register_callbacks cb 0x7fb51c00d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb522ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7210462f0 0x55c72141e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb522ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7210462f0 0x55c72141e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56094/0 (socket says 172.21.15.78:56094) 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb522ffd640 1 -- 172.21.15.78:0/2714541016 learned_addr learned my addr 172.21.15.78:0/2714541016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.828+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72104e4d0 0x55c72141ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72106f1b0 0x55c72141fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb522ffd640 1 -- 172.21.15.78:0/2714541016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72106f1b0 msgr2=0x55c72141fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72106f1b0 0x55c72141fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb522ffd640 1 -- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72104e4d0 msgr2=0x55c72141ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:09:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72104e4d0 0x55c72141ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb522ffd640 1 -- 172.21.15.78:0/2714541016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c721420f70 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72104e4d0 0x55c72141ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb522ffd640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7210462f0 0x55c72141e740 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb51400aa90 tx=0x7fb51400a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed6118cb4dcfd83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb50affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb50affd640 1 -- 172.21.15.78:0/2714541016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514032040 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb50affd640 1 -- 172.21.15.78:0/2714541016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fb514011230 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c72137cee0 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb50affd640 1 -- 172.21.15.78:0/2714541016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb51402a7c0 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb538dd3740 10 client.18558 fetch_fsmap learned FSMap version 22 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c72148fa50 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb50affd640 1 -- 172.21.15.78:0/2714541016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c72137cee0 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.829+0000 7fb50affd640 1 -- 172.21.15.78:0/2714541016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb514035070 con 0x55c7210462f0 2024-04-25T14:09:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb538dd3740 10 client.18558 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb538dd3740 10 client.18558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb538dd3740 20 client.18558 populate_metadata read hostname 'smithi078' 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c7210031c0 0x55c721492240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb538dd3740 4 client.18558 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7212f85f0 con 0x55c7210031c0 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c7210031c0 0x55c721492240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb52b7fe640 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c7210031c0 0x55c721492240 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9455d8b6ceb8ad99 server_cookie=6b6983bf0aec88a2 in_seq=0 out_seq=0 2024-04-25T14:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.830+0000 7fb50affd640 10 client.18558 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.845+0000 7fb50affd640 1 -- 172.21.15.78:0/2714541016 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7212f85f0 con 0x55c7210031c0 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.845+0000 7fb50affd640 10 client.18558 handle_command_reply: tid=0 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.845+0000 7fb538dd3740 1 client.18558 shutdown 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7210462f0 msgr2=0x55c72141e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7210462f0 0x55c72141e740 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb51400aa90 tx=0x7fb51400a6c0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 shutdown_connections 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55c7210031c0 0x55c721492240 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c72106f1b0 0x55c72141fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c72104e4d0 0x55c72141ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb50affd640 0 client.18558 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 --2- 172.21.15.78:0/2714541016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c7210462f0 0x55c72141e740 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 >> 172.21.15.78:0/2714541016 conn(0x55c721063300 msgr2=0x55c72140f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 shutdown_connections 2024-04-25T14:09:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 1 -- 172.21.15.78:0/2714541016 wait complete. 2024-04-25T14:09:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb538dd3740 20 client.18558 trim_cache size 0 max 16384 2024-04-25T14:09:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:09:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:09:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:09:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5180afae0 msgr2=0x7fb5180b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5180afae0 0x7fb5180b1ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb524005d80 tx=0x7fb524097550 comp rx=0 tx=0).stop 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 msgr2=0x7fb52c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c076540 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb5240097b0 tx=0x7fb524003c20 comp rx=0 tx=0).stop 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 shutdown_connections 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fb5180afae0 0x7fb5180b1ff0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb52c109a80 0x7fb52c076540 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c104d80 0x7fb52c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 --2- 172.21.15.78:0/96038694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52c103b40 0x7fb52c077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:09:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 >> 172.21.15.78:0/96038694 conn(0x7fb52c0fed30 msgr2=0x7fb52c105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:09:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 shutdown_connections 2024-04-25T14:09:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:09:48.846+0000 7fb531f1f640 1 -- 172.21.15.78:0/96038694 wait complete. 2024-04-25T14:09:50.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:49 smithi078 ceph-mon[29163]: pgmap v657: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:09:50.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:49 smithi112 ceph-mon[32617]: pgmap v657: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:09:50.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:49 smithi129 ceph-mon[32569]: pgmap v657: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:09:52.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:51 smithi112 ceph-mon[32617]: pgmap v658: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:09:52.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:51 smithi129 ceph-mon[32569]: pgmap v658: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:09:52.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:51 smithi078 ceph-mon[29163]: pgmap v658: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-25T14:09:54.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:53 smithi112 ceph-mon[32617]: pgmap v659: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:09:54.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:53 smithi129 ceph-mon[32569]: pgmap v659: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:09:54.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:53 smithi078 ceph-mon[29163]: pgmap v659: 97 pgs: 97 active+clean; 3.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T14:09:56.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:55 smithi129 ceph-mon[32569]: pgmap v660: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:56.261 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:55 smithi112 ceph-mon[32617]: pgmap v660: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:55 smithi078 ceph-mon[29163]: pgmap v660: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:09:58.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:57 smithi112 ceph-mon[32617]: pgmap v661: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:58.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:57 smithi129 ceph-mon[32569]: pgmap v661: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:09:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:57 smithi078 ceph-mon[29163]: pgmap v661: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:00.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:09:59 smithi112 ceph-mon[32617]: pgmap v662: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:00.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:09:59 smithi129 ceph-mon[32569]: pgmap v662: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:09:59 smithi078 ceph-mon[29163]: pgmap v662: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:01.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:00 smithi112 ceph-mon[32617]: overall HEALTH_OK 2024-04-25T14:10:01.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:00 smithi129 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T14:10:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:00 smithi078 ceph-mon[29163]: overall HEALTH_OK 2024-04-25T14:10:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 -- 172.21.15.78:0/232044164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4108970 msgr2=0x7fbdb410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/232044164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4108970 0x7fbdb410ad90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009ca0 tx=0x7fbdac02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 -- 172.21.15.78:0/232044164 shutdown_connections 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/232044164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4108970 0x7fbdb410ad90 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/232044164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb4103c70 0x7fbdb4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/232044164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdb4102a30 0x7fbdb4103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 -- 172.21.15.78:0/232044164 >> 172.21.15.78:0/232044164 conn(0x7fbdb40fdc20 msgr2=0x7fbdb41000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 -- 172.21.15.78:0/232044164 shutdown_connections 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.950+0000 7fbdb9f00640 1 -- 172.21.15.78:0/232044164 wait complete. 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 Processor -- start 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 -- start start 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdb4102a30 0x7fbdb4075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4103c70 0x7fbdb419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb4108970 0x7fbdb41a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410e560 con 0x7fbdb4103c70 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410e3e0 con 0x7fbdb4108970 2024-04-25T14:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb9f00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410e6e0 con 0x7fbdb4102a30 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4103c70 0x7fbdb419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4103c70 0x7fbdb419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55546/0 (socket says 172.21.15.78:55546) 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 -- 172.21.15.78:0/3799038587 learned_addr learned my addr 172.21.15.78:0/3799038587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdb4102a30 0x7fbdb4075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb4108970 0x7fbdb41a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 -- 172.21.15.78:0/3799038587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdb4102a30 msgr2=0x7fbdb4075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdb4102a30 0x7fbdb4075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 -- 172.21.15.78:0/3799038587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb4108970 msgr2=0x7fbdb41a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb4108970 0x7fbdb41a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdabfff640 1 -- 172.21.15.78:0/3799038587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb41a1cc0 con 0x7fbdb4103c70 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdb4102a30 0x7fbdb4075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.951+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb4108970 0x7fbdb41a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.952+0000 7fbdabfff640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4103c70 0x7fbdb419ef30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00d950 tx=0x7fbd9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.952+0000 7fbdb17fa640 1 -- 172.21.15.78:0/3799038587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c017070 con 0x7fbdb4103c70 2024-04-25T14:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.952+0000 7fbdb17fa640 1 -- 172.21.15.78:0/3799038587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd9c009c30 con 0x7fbdb4103c70 2024-04-25T14:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.952+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb41a1fb0 con 0x7fbdb4103c70 2024-04-25T14:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.952+0000 7fbdb17fa640 1 -- 172.21.15.78:0/3799038587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00f620 con 0x7fbdb4103c70 2024-04-25T14:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.952+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb41a2420 con 0x7fbdb4103c70 2024-04-25T14:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.954+0000 7fbdb17fa640 1 -- 172.21.15.78:0/3799038587 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbd9c010460 con 0x7fbdb4103c70 2024-04-25T14:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.954+0000 7fbdc0db5740 1 Processor -- start 2024-04-25T14:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.954+0000 7fbdc0db5740 1 -- start start 2024-04-25T14:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570900f9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090100df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55709005f460 con 0x55708fd384d0 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557090019590 con 0x55708fd302f0 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55709005b420 con 0x55708fd591b0 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090100df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55552/0 (socket says 172.21.15.78:55552) 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 -- 172.21.15.78:0/2966671920 learned_addr learned my addr 172.21.15.78:0/2966671920 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570900f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdb17fa640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbda00afae0 0x7fbda00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 -- 172.21.15.78:0/2966671920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 msgr2=0x5570900f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570900f9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 -- 172.21.15.78:0/2966671920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 msgr2=0x5570900f9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900f9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 -- 172.21.15.78:0/2966671920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570900260d0 con 0x55708fd384d0 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570900f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbda00afae0 0x7fbda00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdb17fa640 1 -- 172.21.15.78:0/3799038587 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbd9c0de950 con 0x7fbdb4103c70 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.955+0000 7fbdabfff640 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090100df0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c085360 tx=0x7fbd9c06d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6386c43dbb61e09c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbda9ffb640 1 -- 172.21.15.78:0/2966671920 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0e0e60 con 0x55708fd384d0 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbda00afae0 0x7fbda00b1ff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbdac031a90 tx=0x7fbdac0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbda9ffb640 1 -- 172.21.15.78:0/2966671920 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd9c082d60 con 0x55708fd384d0 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbda9ffb640 1 -- 172.21.15.78:0/2966671920 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c082460 con 0x55708fd384d0 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2966671920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 msgr2=0x557090100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090100df0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c085360 tx=0x7fbd9c06d290 comp rx=0 tx=0).stop 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2966671920 shutdown_connections 2024-04-25T14:10:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090100df0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900f9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2966671920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570900f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2966671920 >> 172.21.15.78:0/2966671920 conn(0x55708fd4d300 msgr2=0x5570900f5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2966671920 shutdown_connections 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.956+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2966671920 wait complete. 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 -- start start 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 Processor -- start 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd302f0 0x557090103670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd591b0 0x55709010ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5570901017e0 con 0x55708fd384d0 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55708ffe6b60 con 0x55708fd591b0 2024-04-25T14:10:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557090101960 con 0x55708fd302f0 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdc0db5740 10 client.0 _ll_register_callbacks cb 0x55709000b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090103bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55568/0 (socket says 172.21.15.78:55568) 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.957+0000 7fbdabfff640 1 -- 172.21.15.78:0/3000647195 learned_addr learned my addr 172.21.15.78:0/3000647195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd591b0 0x55709010ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd302f0 0x557090103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdabfff640 1 -- 172.21.15.78:0/3000647195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd302f0 msgr2=0x557090103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdabfff640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd302f0 0x557090103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdabfff640 1 -- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd591b0 msgr2=0x55709010ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdabfff640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd591b0 0x55709010ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdabfff640 1 -- 172.21.15.78:0/3000647195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55708fd71210 con 0x55708fd384d0 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd591b0 0x55709010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdabfff640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090103bb0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c082f20 tx=0x7fbd9c0dec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b2a156da34edaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbd92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbd92ffd640 1 -- 172.21.15.78:0/3000647195 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c020030 con 0x55708fd384d0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbd92ffd640 1 -- 172.21.15.78:0/3000647195 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbd9c0af7f0 con 0x55708fd384d0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55709006d070 con 0x55708fd384d0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbd92ffd640 1 -- 172.21.15.78:0/3000647195 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbd9c0a6bf0 con 0x55708fd384d0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdc0db5740 10 client.18570 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55708fe3d620 con 0x55708fd384d0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.958+0000 7fbd92ffd640 1 -- 172.21.15.78:0/3000647195 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55709006d070 con 0x55708fd384d0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbd92ffd640 1 -- 172.21.15.78:0/3000647195 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbd9c0710c0 con 0x55708fd384d0 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdc0db5740 10 client.18570 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdc0db5740 10 client.18570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdc0db5740 20 client.18570 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fd17d80 0x557090110f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdc0db5740 4 client.18570 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557090066ee0 con 0x55708fd17d80 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fd17d80 0x557090110f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbd92ffd640 10 client.18570 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.959+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fd17d80 0x557090110f60 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d023404a3419d960 server_cookie=417275bafd5602c9 in_seq=0 out_seq=0 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbd92ffd640 1 -- 172.21.15.78:0/3000647195 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557090066ee0 con 0x55708fd17d80 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbd92ffd640 10 client.18570 handle_command_reply: tid=0 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 client.18570 shutdown 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 msgr2=0x557090103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090103bb0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c082f20 tx=0x7fbd9c0dec50 comp rx=0 tx=0).stop 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 shutdown_connections 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd591b0 0x55709010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x557090103bb0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fd17d80 0x557090110f60 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/3000647195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd302f0 0x557090103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbd92ffd640 0 client.18570 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 >> 172.21.15.78:0/3000647195 conn(0x55708fd4d300 msgr2=0x5570900f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 shutdown_connections 2024-04-25T14:10:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 1 -- 172.21.15.78:0/3000647195 wait complete. 2024-04-25T14:10:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.960+0000 7fbdc0db5740 20 client.18570 trim_cache size 0 max 16384 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.975+0000 7fbdc0db5740 1 Processor -- start 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.975+0000 7fbdc0db5740 1 -- start start 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x5570900fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x557090101d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41adb60 con 0x55708fd384d0 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41ab430 con 0x55708fd302f0 2024-04-25T14:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41a9280 con 0x55708fd591b0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x5570900fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x5570900fe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55582/0 (socket says 172.21.15.78:55582) 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 -- 172.21.15.78:0/2705698731 learned_addr learned my addr 172.21.15.78:0/2705698731 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x557090101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 -- 172.21.15.78:0/2705698731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 msgr2=0x557090101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x557090101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 -- 172.21.15.78:0/2705698731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 msgr2=0x5570900fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 -- 172.21.15.78:0/2705698731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570900f5e90 con 0x55708fd384d0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbdabfff640 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x5570900fe0e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0dedd0 tx=0x7fbd9c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4703fe379269820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbda97fa640 1 -- 172.21.15.78:0/2705698731 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c020040 con 0x55708fd384d0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbda97fa640 1 -- 172.21.15.78:0/2705698731 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd9c0a6930 con 0x55708fd384d0 2024-04-25T14:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.976+0000 7fbda97fa640 1 -- 172.21.15.78:0/2705698731 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0ab1d0 con 0x55708fd384d0 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2705698731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 msgr2=0x5570900fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x5570900fe0e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0dedd0 tx=0x7fbd9c005f90 comp rx=0 tx=0).stop 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2705698731 shutdown_connections 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x557090101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd384d0 0x5570900fe0e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/2705698731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd302f0 0x5570900fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2705698731 >> 172.21.15.78:0/2705698731 conn(0x55708fd4d300 msgr2=0x5570900f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2705698731 shutdown_connections 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 -- 172.21.15.78:0/2705698731 wait complete. 2024-04-25T14:10:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.977+0000 7fbdc0db5740 1 -- start start 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 1 Processor -- start 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd302f0 0x5570901091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd384d0 0x557090109730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570901771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41a98f0 con 0x55708fd302f0 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41aad60 con 0x55708fd384d0 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb41a9fc0 con 0x55708fd591b0 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdc0db5740 10 client.0 _ll_register_callbacks cb 0x7fbd9c082f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd302f0 0x5570901091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd302f0 0x5570901091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55596/0 (socket says 172.21.15.78:55596) 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 -- 172.21.15.78:0/1860398607 learned_addr learned my addr 172.21.15.78:0/1860398607 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdabfff640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd384d0 0x557090109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570901771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 -- 172.21.15.78:0/1860398607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 msgr2=0x5570901771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570901771e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 -- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd384d0 msgr2=0x557090109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd384d0 0x557090109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 -- 172.21.15.78:0/1860398607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557090177720 con 0x55708fd302f0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdabfff640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd384d0 0x557090109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb3fff640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570901771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbdb37fe640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd302f0 0x5570901091f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009780 tx=0x7fbda401b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e33b3e5b00219b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.978+0000 7fbd92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbd92ffd640 1 -- 172.21.15.78:0/1860398607 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4012920 con 0x55708fd302f0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbd92ffd640 1 -- 172.21.15.78:0/1860398607 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbda4012ac0 con 0x55708fd302f0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9c0a6930 con 0x55708fd302f0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbd92ffd640 1 -- 172.21.15.78:0/1860398607 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbda4012cc0 con 0x55708fd302f0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 10 client.18576 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x557090178a50 con 0x55708fd302f0 2024-04-25T14:10:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbd92ffd640 1 -- 172.21.15.78:0/1860398607 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd9c0a6930 con 0x55708fd302f0 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbd92ffd640 1 -- 172.21.15.78:0/1860398607 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbda400f040 con 0x55708fd302f0 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 10 client.18576 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 10 client.18576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 20 client.18576 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fced1c0 0x55709017b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 4 client.18576 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55708ffe25f0 con 0x55708fced1c0 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.979+0000 7fbdabfff640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fced1c0 0x55709017b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.980+0000 7fbd92ffd640 10 client.18576 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.980+0000 7fbdabfff640 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fced1c0 0x55709017b420 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf99f0b3c8af1578 server_cookie=5caf8b851f250ac3 in_seq=0 out_seq=0 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbd92ffd640 1 -- 172.21.15.78:0/1860398607 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55708ffe25f0 con 0x55708fced1c0 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbd92ffd640 10 client.18576 handle_command_reply: tid=0 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 client.18576 shutdown 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd302f0 msgr2=0x5570901091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd302f0 0x5570901091f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009780 tx=0x7fbda401b600 comp rx=0 tx=0).stop 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 shutdown_connections 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55708fced1c0 0x55709017b420 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55708fd591b0 0x5570901771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbd92ffd640 0 client.18576 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55708fd384d0 0x557090109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 --2- 172.21.15.78:0/1860398607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55708fd302f0 0x5570901091f0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 >> 172.21.15.78:0/1860398607 conn(0x55708fd4d300 msgr2=0x5570900f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 shutdown_connections 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 1 -- 172.21.15.78:0/1860398607 wait complete. 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.996+0000 7fbdc0db5740 20 client.18576 trim_cache size 0 max 16384 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:10:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:10:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbda00afae0 msgr2=0x7fbda00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbda00afae0 0x7fbda00b1ff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbdac031a90 tx=0x7fbdac0095c0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4103c70 msgr2=0x7fbdb419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4103c70 0x7fbdb419ef30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00d950 tx=0x7fbd9c00de10 comp rx=0 tx=0).stop 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 shutdown_connections 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdb4108970 0x7fbdb41a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbda00afae0 0x7fbda00b1ff0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb4103c70 0x7fbdb419ef30 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 --2- 172.21.15.78:0/3799038587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbdb4102a30 0x7fbdb4075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 >> 172.21.15.78:0/3799038587 conn(0x7fbdb40fdc20 msgr2=0x7fbdb4104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 shutdown_connections 2024-04-25T14:10:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:01.997+0000 7fbdb9f00640 1 -- 172.21.15.78:0/3799038587 wait complete. 2024-04-25T14:10:02.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:01 smithi112 ceph-mon[32617]: pgmap v663: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:02.370 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:01 smithi078 ceph-mon[29163]: pgmap v663: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:02.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:01 smithi129 ceph-mon[32569]: pgmap v663: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:04.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:10:04.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:10:04.212 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:03 smithi078 ceph-mon[29163]: pgmap v664: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:04.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:03 smithi112 ceph-mon[32617]: pgmap v664: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 -- 172.21.15.78:0/1587973127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac1070c0 msgr2=0x7f8cac1094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/1587973127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac1070c0 0x7f8cac1094e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009900 tx=0x7f8c9c02a410 comp rx=0 tx=0).stop 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 -- 172.21.15.78:0/1587973127 shutdown_connections 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/1587973127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cac109a20 0x7f8cac10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/1587973127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac1070c0 0x7f8cac1094e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/1587973127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac101f10 0x7f8cac106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 -- 172.21.15.78:0/1587973127 >> 172.21.15.78:0/1587973127 conn(0x7f8cac0fd350 msgr2=0x7f8cac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 -- 172.21.15.78:0/1587973127 shutdown_connections 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.304+0000 7f8cb36a7640 1 -- 172.21.15.78:0/1587973127 wait complete. 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 Processor -- start 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 -- start start 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac101f10 0x7f8cac19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cac1070c0 0x7f8cac19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac109a20 0x7f8cac1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10f720 con 0x7f8cac109a20 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10f5a0 con 0x7f8cac101f10 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb36a7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10f8a0 con 0x7f8cac1070c0 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac109a20 0x7f8cac1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac109a20 0x7f8cac1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55614/0 (socket says 172.21.15.78:55614) 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 -- 172.21.15.78:0/4288682582 learned_addr learned my addr 172.21.15.78:0/4288682582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cac1070c0 0x7f8cac19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1443640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac101f10 0x7f8cac19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 -- 172.21.15.78:0/4288682582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cac1070c0 msgr2=0x7f8cac19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cac1070c0 0x7f8cac19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 -- 172.21.15.78:0/4288682582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac101f10 msgr2=0x7f8cac19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac101f10 0x7f8cac19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 -- 172.21.15.78:0/4288682582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac1a1de0 con 0x7f8cac109a20 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cac1070c0 0x7f8cac19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.305+0000 7f8cb1c44640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac109a20 0x7f8cac1a1820 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400b970 tx=0x7f8ca400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.306+0000 7f8ca27fc640 1 -- 172.21.15.78:0/4288682582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4019070 con 0x7f8cac109a20 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.306+0000 7f8ca27fc640 1 -- 172.21.15.78:0/4288682582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca4014440 con 0x7f8cac109a20 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.306+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac1a20d0 con 0x7f8cac109a20 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.306+0000 7f8ca27fc640 1 -- 172.21.15.78:0/4288682582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40075d0 con 0x7f8cac109a20 2024-04-25T14:10:04.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.306+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac1a26a0 con 0x7f8cac109a20 2024-04-25T14:10:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.308+0000 7f8cba630740 1 Processor -- start 2024-04-25T14:10:04.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.308+0000 7f8ca27fc640 1 -- 172.21.15.78:0/4288682582 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8ca401c070 con 0x7f8cac109a20 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.308+0000 7f8cba630740 1 -- start start 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cba630740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799617a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cba630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799617fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cba630740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799618c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cba630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d79957cd80 con 0x55d79924d4d0 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cba630740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d7995368b0 con 0x55d7992556b0 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cba630740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d799578d40 con 0x55d799276390 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799617fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799617fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55630/0 (socket says 172.21.15.78:55630) 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 -- 172.21.15.78:0/1588625932 learned_addr learned my addr 172.21.15.78:0/1588625932 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799617a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799618c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 -- 172.21.15.78:0/1588625932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 msgr2=0x55d799617a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799617a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8ca27fc640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c980afaf0 0x7f8c980b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 -- 172.21.15.78:0/1588625932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 msgr2=0x55d799618c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799618c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 -- 172.21.15.78:0/1588625932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d799507a40 con 0x55d79924d4d0 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799617a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1c44640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c980afaf0 0x7f8c980b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8cb1443640 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799617fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800cc40 tx=0x7f8ca800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3712ed969f17a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.309+0000 7f8ca27fc640 1 -- 172.21.15.78:0/4288682582 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8ca40de220 con 0x7f8cac109a20 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.310+0000 7f8c977fe640 1 -- 172.21.15.78:0/1588625932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8014070 con 0x55d79924d4d0 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.310+0000 7f8c977fe640 1 -- 172.21.15.78:0/1588625932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca80047a0 con 0x55d79924d4d0 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.310+0000 7f8c977fe640 1 -- 172.21.15.78:0/1588625932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8010630 con 0x55d79924d4d0 2024-04-25T14:10:04.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.310+0000 7f8cba630740 1 -- 172.21.15.78:0/1588625932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 msgr2=0x55d799617fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.310+0000 7f8cb1c44640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c980afaf0 0x7f8c980b2000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40dc080 tx=0x7f8ca408f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.310+0000 7f8cba630740 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799617fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800cc40 tx=0x7f8ca800e720 comp rx=0 tx=0).stop 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- 172.21.15.78:0/1588625932 shutdown_connections 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799618c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799617fe0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 --2- 172.21.15.78:0/1588625932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799617a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- 172.21.15.78:0/1588625932 >> 172.21.15.78:0/1588625932 conn(0x55d79926a4e0 msgr2=0x55d799613270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- 172.21.15.78:0/1588625932 shutdown_connections 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- 172.21.15.78:0/1588625932 wait complete. 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- start start 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 Processor -- start 2024-04-25T14:10:04.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799621900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799621e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799628da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d799619780 con 0x55d79924d4d0 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d799503d10 con 0x55d7992556b0 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.311+0000 7f8cba630740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d799619900 con 0x55d799276390 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cba630740 10 client.0 _ll_register_callbacks cb 0x55d799530210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799621900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799621900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55632/0 (socket says 172.21.15.78:55632) 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 -- 172.21.15.78:0/4143417197 learned_addr learned my addr 172.21.15.78:0/4143417197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb1443640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799621e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799628da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 -- 172.21.15.78:0/4143417197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 msgr2=0x55d799628da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799628da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 -- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 msgr2=0x55d799621e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799621e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 -- 172.21.15.78:0/4143417197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7993587d0 con 0x55d79924d4d0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb1443640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799621e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799621900 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005f70 tx=0x7f8c9c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54024c9875dd8a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8c94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8c94ff9640 1 -- 172.21.15.78:0/4143417197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c035070 con 0x55d79924d4d0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8c94ff9640 1 -- 172.21.15.78:0/4143417197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8c9c004450 con 0x55d79924d4d0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca80047a0 con 0x55d79924d4d0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8c94ff9640 1 -- 172.21.15.78:0/4143417197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c0035f0 con 0x55d79924d4d0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cba630740 10 client.18588 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.312+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d79928e3f0 con 0x55d79924d4d0 2024-04-25T14:10:04.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8c94ff9640 1 -- 172.21.15.78:0/4143417197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca80047a0 con 0x55d79924d4d0 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8c94ff9640 1 -- 172.21.15.78:0/4143417197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8c9c032470 con 0x55d79924d4d0 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8cba630740 10 client.18588 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8cba630740 10 client.18588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8cba630740 20 client.18588 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8cba630740 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d799234f60 0x55d79962f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8cba630740 4 client.18588 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d79958a990 con 0x55d799234f60 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.313+0000 7f8cb1443640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d799234f60 0x55d79962f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8c94ff9640 10 client.18588 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cb1443640 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d799234f60 0x55d79962f240 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60e63ed5d4de107 server_cookie=d671789d8d88e27b in_seq=0 out_seq=0 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8c94ff9640 1 -- 172.21.15.78:0/4143417197 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d79958a990 con 0x55d799234f60 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8c94ff9640 10 client.18588 handle_command_reply: tid=0 2024-04-25T14:10:04.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 client.18588 shutdown 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 msgr2=0x55d799621900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799621900 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005f70 tx=0x7f8c9c009260 comp rx=0 tx=0).stop 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 shutdown_connections 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d799234f60 0x55d79962f240 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799628da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799621e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8c94ff9640 0 client.18588 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 --2- 172.21.15.78:0/4143417197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799621900 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.314+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 >> 172.21.15.78:0/4143417197 conn(0x55d79926a4e0 msgr2=0x55d799616180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.315+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 shutdown_connections 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.315+0000 7f8cba630740 1 -- 172.21.15.78:0/4143417197 wait complete. 2024-04-25T14:10:04.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.315+0000 7f8cba630740 20 client.18588 trim_cache size 0 max 16384 2024-04-25T14:10:04.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.332+0000 7f8cba630740 1 Processor -- start 2024-04-25T14:10:04.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.332+0000 7f8cba630740 1 -- start start 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.332+0000 7f8cba630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799620610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.332+0000 7f8cba630740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799612c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cba630740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799613150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cba630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1adc80 con 0x55d79924d4d0 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cba630740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1ab550 con 0x55d7992556b0 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cba630740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1a93a0 con 0x55d799276390 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799620610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799620610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55646/0 (socket says 172.21.15.78:55646) 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 -- 172.21.15.78:0/1704128414 learned_addr learned my addr 172.21.15.78:0/1704128414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb1443640 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799612c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 -- 172.21.15.78:0/1704128414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 msgr2=0x55d799613150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799613150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 -- 172.21.15.78:0/1704128414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 msgr2=0x55d799612c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799612c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 -- 172.21.15.78:0/1704128414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79924e0f0 con 0x55d79924d4d0 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb1443640 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799612c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799620610 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0328e0 tx=0x7f8c9c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0b6a21a89f8501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8c96ffd640 1 -- 172.21.15.78:0/1704128414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c05b070 con 0x55d79924d4d0 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8c96ffd640 1 -- 172.21.15.78:0/1704128414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0324b0 con 0x55d79924d4d0 2024-04-25T14:10:04.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.333+0000 7f8c96ffd640 1 -- 172.21.15.78:0/1704128414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c035040 con 0x55d79924d4d0 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 -- 172.21.15.78:0/1704128414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 msgr2=0x55d799620610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799620610 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0328e0 tx=0x7f8c9c02a410 comp rx=0 tx=0).stop 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 -- 172.21.15.78:0/1704128414 shutdown_connections 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d799276390 0x55d799613150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d7992556b0 0x55d799612c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 --2- 172.21.15.78:0/1704128414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d79924d4d0 0x55d799620610 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 -- 172.21.15.78:0/1704128414 >> 172.21.15.78:0/1704128414 conn(0x55d79926a4e0 msgr2=0x55d79961ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 -- 172.21.15.78:0/1704128414 shutdown_connections 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 -- 172.21.15.78:0/1704128414 wait complete. 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 -- start start 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.334+0000 7f8cba630740 1 Processor -- start 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cba630740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d79924d4d0 0x55d799692380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cba630740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7992556b0 0x55d7996928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cba630740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d799276390 0x55d799699820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cba630740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1a9a10 con 0x55d799276390 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cba630740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1aae80 con 0x55d79924d4d0 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cba630740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1aa0e0 con 0x55d7992556b0 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cba630740 10 client.0 _ll_register_callbacks cb 0x7f8c9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d799276390 0x55d799699820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d799276390 0x55d799699820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55658/0 (socket says 172.21.15.78:55658) 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 -- 172.21.15.78:0/3840879829 learned_addr learned my addr 172.21.15.78:0/3840879829 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb1443640 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7992556b0 0x55d7996928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d79924d4d0 0x55d799692380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 -- 172.21.15.78:0/3840879829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7992556b0 msgr2=0x55d7996928c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7992556b0 0x55d7996928c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 -- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d79924d4d0 msgr2=0x55d799692380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d79924d4d0 0x55d799692380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 -- 172.21.15.78:0/3840879829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79953d4b0 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8cb0c42640 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d799276390 0x55d799699820 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009900 tx=0x7f8c9c031aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a305e9f32670310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.335+0000 7f8c94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8c94ff9640 1 -- 172.21.15.78:0/3840879829 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c035070 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8c94ff9640 1 -- 172.21.15.78:0/3840879829 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f8c9c003db0 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c0324b0 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8c94ff9640 1 -- 172.21.15.78:0/3840879829 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c05b030 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 10 client.18594 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d79969ae10 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8c94ff9640 1 -- 172.21.15.78:0/3840879829 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0324b0 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8c94ff9640 1 -- 172.21.15.78:0/3840879829 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8c9c061070 con 0x55d799276390 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 10 client.18594 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 10 client.18594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 20 client.18594 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d79920a3a0 0x55d79969d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 4 client.18594 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.336+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d799584800 con 0x55d79920a3a0 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cb1443640 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d79920a3a0 0x55d79969d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8c94ff9640 10 client.18594 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cb1443640 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d79920a3a0 0x55d79969d7a0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=945b58a120c46542 server_cookie=d16ec201b74e403f in_seq=0 out_seq=0 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8c94ff9640 1 -- 172.21.15.78:0/3840879829 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d799584800 con 0x55d79920a3a0 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8c94ff9640 10 client.18594 handle_command_reply: tid=0 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 client.18594 shutdown 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d799276390 msgr2=0x55d799699820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d799276390 0x55d799699820 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009900 tx=0x7f8c9c031aa0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 shutdown_connections 2024-04-25T14:10:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d799276390 0x55d799699820 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d79920a3a0 0x55d79969d7a0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d7992556b0 0x55d7996928c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 --2- 172.21.15.78:0/3840879829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d79924d4d0 0x55d799692380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8c94ff9640 0 client.18594 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.337+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 >> 172.21.15.78:0/3840879829 conn(0x55d79926a4e0 msgr2=0x55d799612c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 shutdown_connections 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cba630740 1 -- 172.21.15.78:0/3840879829 wait complete. 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cba630740 20 client.18594 trim_cache size 0 max 16384 2024-04-25T14:10:04.345 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:10:04.346 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:10:04.346 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:10:04.346 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c980afaf0 msgr2=0x7f8c980b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c980afaf0 0x7f8c980b2000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40dc080 tx=0x7f8ca408f040 comp rx=0 tx=0).stop 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac109a20 msgr2=0x7f8cac1a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac109a20 0x7f8cac1a1820 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400b970 tx=0x7f8ca400be30 comp rx=0 tx=0).stop 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 shutdown_connections 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f8c980afaf0 0x7f8c980b2000 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cac109a20 0x7f8cac1a1820 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cac1070c0 0x7f8cac19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 --2- 172.21.15.78:0/4288682582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8cac101f10 0x7f8cac19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 >> 172.21.15.78:0/4288682582 conn(0x7f8cac0fd350 msgr2=0x7f8cac1056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 shutdown_connections 2024-04-25T14:10:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.338+0000 7f8cb36a7640 1 -- 172.21.15.78:0/4288682582 wait complete. 2024-04-25T14:10:04.349 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:10:04.349 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:04.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:03 smithi129 ceph-mon[32569]: pgmap v664: 97 pgs: 97 active+clean; 3.4 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:04.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 -- 172.21.15.78:0/3204883107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 msgr2=0x7fee2410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 --2- 172.21.15.78:0/3204883107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee2410ad40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009ca0 tx=0x7fee1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 -- 172.21.15.78:0/3204883107 shutdown_connections 2024-04-25T14:10:04.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 --2- 172.21.15.78:0/3204883107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee2410ad40 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 --2- 172.21.15.78:0/3204883107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee24103c20 0x7fee24104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 --2- 172.21.15.78:0/3204883107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee241029e0 0x7fee241036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 -- 172.21.15.78:0/3204883107 >> 172.21.15.78:0/3204883107 conn(0x7fee240fdbd0 msgr2=0x7fee24100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 -- 172.21.15.78:0/3204883107 shutdown_connections 2024-04-25T14:10:04.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.498+0000 7fee296a7640 1 -- 172.21.15.78:0/3204883107 wait complete. 2024-04-25T14:10:04.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 Processor -- start 2024-04-25T14:10:04.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 -- start start 2024-04-25T14:10:04.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee241029e0 0x7fee2419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee24103c20 0x7fee2419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee241a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410f610 con 0x7fee24108920 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410f490 con 0x7fee241029e0 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee296a7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410f790 con 0x7fee24103c20 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee241a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee241a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55676/0 (socket says 172.21.15.78:55676) 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee237fe640 1 -- 172.21.15.78:0/2948316589 learned_addr learned my addr 172.21.15.78:0/2948316589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.499+0000 7fee22ffd640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee241029e0 0x7fee2419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee227fc640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee24103c20 0x7fee2419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee237fe640 1 -- 172.21.15.78:0/2948316589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee24103c20 msgr2=0x7fee2419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee237fe640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee24103c20 0x7fee2419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee237fe640 1 -- 172.21.15.78:0/2948316589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee241029e0 msgr2=0x7fee2419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee237fe640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee241029e0 0x7fee2419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee237fe640 1 -- 172.21.15.78:0/2948316589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee241a1fe0 con 0x7fee24108920 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee237fe640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee241a1970 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0097b0 tx=0x7fee1c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee0bfff640 1 -- 172.21.15.78:0/2948316589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c003bf0 con 0x7fee24108920 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee241a2270 con 0x7fee24108920 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee0bfff640 1 -- 172.21.15.78:0/2948316589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1c003d90 con 0x7fee24108920 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee0bfff640 1 -- 172.21.15.78:0/2948316589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c02fc30 con 0x7fee24108920 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.500+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241a27c0 con 0x7fee24108920 2024-04-25T14:10:04.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.501+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24107730 con 0x7fee24108920 2024-04-25T14:10:04.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.503+0000 7fee0bfff640 1 -- 172.21.15.78:0/2948316589 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fee1c004040 con 0x7fee24108920 2024-04-25T14:10:04.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.503+0000 7fee0bfff640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fee0c0afaf0 0x7fee0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.504+0000 7fee22ffd640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fee0c0afaf0 0x7fee0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.504+0000 7fee0bfff640 1 -- 172.21.15.78:0/2948316589 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fee1c1004a0 con 0x7fee24108920 2024-04-25T14:10:04.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.504+0000 7fee22ffd640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fee0c0afaf0 0x7fee0c0b2000 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee241035e0 tx=0x7fee14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.508+0000 7fee0bfff640 1 -- 172.21.15.78:0/2948316589 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee1c0c9070 con 0x7fee24108920 2024-04-25T14:10:04.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.720+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee24069b70 con 0x7fee24108920 2024-04-25T14:10:04.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.720+0000 7fee0bfff640 1 -- 172.21.15.78:0/2948316589 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7fee1c0c8970 con 0x7fee24108920 2024-04-25T14:10:04.720 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:10:04.721 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:10:04.721 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fee0c0afaf0 msgr2=0x7fee0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fee0c0afaf0 0x7fee0c0b2000 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee241035e0 tx=0x7fee14005f90 comp rx=0 tx=0).stop 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 msgr2=0x7fee241a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee241a1970 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0097b0 tx=0x7fee1c031840 comp rx=0 tx=0).stop 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 shutdown_connections 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fee0c0afaf0 0x7fee0c0b2000 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee24108920 0x7fee241a1970 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee24103c20 0x7fee2419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 --2- 172.21.15.78:0/2948316589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee241029e0 0x7fee2419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 >> 172.21.15.78:0/2948316589 conn(0x7fee240fdbd0 msgr2=0x7fee24104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 shutdown_connections 2024-04-25T14:10:04.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.723+0000 7fee296a7640 1 -- 172.21.15.78:0/2948316589 wait complete. 2024-04-25T14:10:04.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:10:04.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 -- 172.21.15.78:0/3601534626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508109a30 msgr2=0x7f050810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 --2- 172.21.15.78:0/3601534626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508109a30 0x7f050810be50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0500009ca0 tx=0x7f050002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 -- 172.21.15.78:0/3601534626 shutdown_connections 2024-04-25T14:10:04.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 --2- 172.21.15.78:0/3601534626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508109a30 0x7f050810be50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 --2- 172.21.15.78:0/3601534626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508104d30 0x7f0508105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 --2- 172.21.15.78:0/3601534626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0508103af0 0x7f05081047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 -- 172.21.15.78:0/3601534626 >> 172.21.15.78:0/3601534626 conn(0x7f05080fece0 msgr2=0x7f0508101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 -- 172.21.15.78:0/3601534626 shutdown_connections 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.843+0000 7f050e7b2640 1 -- 172.21.15.78:0/3601534626 wait complete. 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 Processor -- start 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 -- start start 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0508103af0 0x7f050819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508104d30 0x7f050819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508109a30 0x7f05081a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f050810f600 con 0x7f0508104d30 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f050810f480 con 0x7f0508109a30 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050e7b2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f050810f780 con 0x7f0508103af0 2024-04-25T14:10:04.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508104d30 0x7f050819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508104d30 0x7f050819c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55694/0 (socket says 172.21.15.78:55694) 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 -- 172.21.15.78:0/828098339 learned_addr learned my addr 172.21.15.78:0/828098339 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f0507fff640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0508103af0 0x7f050819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050cd4f640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508109a30 0x7f05081a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 -- 172.21.15.78:0/828098339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0508103af0 msgr2=0x7f050819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0508103af0 0x7f050819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 -- 172.21.15.78:0/828098339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508109a30 msgr2=0x7f05081a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508109a30 0x7f05081a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f05077fe640 1 -- 172.21.15.78:0/828098339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05081a3ef0 con 0x7f0508104d30 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.844+0000 7f050cd4f640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508109a30 0x7f05081a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.845+0000 7f05077fe640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508104d30 0x7f050819c8f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c890 tx=0x7f04fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.845+0000 7f05057fa640 1 -- 172.21.15.78:0/828098339 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc004220 con 0x7f0508104d30 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.845+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05081a41e0 con 0x7f0508104d30 2024-04-25T14:10:04.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.845+0000 7f05057fa640 1 -- 172.21.15.78:0/828098339 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc0043c0 con 0x7f0508104d30 2024-04-25T14:10:04.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.845+0000 7f05057fa640 1 -- 172.21.15.78:0/828098339 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc004a00 con 0x7f0508104d30 2024-04-25T14:10:04.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.845+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a4780 con 0x7f0508104d30 2024-04-25T14:10:04.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 Processor -- start 2024-04-25T14:10:04.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 -- start start 2024-04-25T14:10:04.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f05057fa640 1 -- 172.21.15.78:0/828098339 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f04fc0079b0 con 0x7f0508104d30 2024-04-25T14:10:04.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x561559fce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fcfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561559f32d80 con 0x561559c0b6b0 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561559eeca20 con 0x561559c2c390 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0515666740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561559f2ed40 con 0x561559c034d0 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f05077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fcfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f05077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fcfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55706/0 (socket says 172.21.15.78:55706) 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f05077fe640 1 -- 172.21.15.78:0/604750417 learned_addr learned my addr 172.21.15.78:0/604750417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f0507fff640 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.848+0000 7f050cd4f640 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x561559fce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05077fe640 1 -- 172.21.15.78:0/604750417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 msgr2=0x561559fced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05077fe640 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05077fe640 1 -- 172.21.15.78:0/604750417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 msgr2=0x561559fce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05077fe640 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x561559fce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05077fe640 1 -- 172.21.15.78:0/604750417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561559ef7c30 con 0x561559c0b6b0 2024-04-25T14:10:04.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05057fa640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04f00afaf0 0x7f04f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05077fe640 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fcfa20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0e4ba0 tx=0x7f04fc0e6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb3a2698e14a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f050cd4f640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04f00afaf0 0x7f04f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f05057fa640 1 -- 172.21.15.78:0/828098339 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f04fc0de9a0 con 0x7f0508104d30 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f04ee7fc640 1 -- 172.21.15.78:0/604750417 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0a2d30 con 0x561559c0b6b0 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f04ee7fc640 1 -- 172.21.15.78:0/604750417 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc0dee50 con 0x561559c0b6b0 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.849+0000 7f04ee7fc640 1 -- 172.21.15.78:0/604750417 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0a65e0 con 0x561559c0b6b0 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 -- 172.21.15.78:0/604750417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 msgr2=0x561559fcfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f050cd4f640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04f00afaf0 0x7f04f00b2000 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x561559fcf860 tx=0x7f0500031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fcfa20 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0e4ba0 tx=0x7f04fc0e6690 comp rx=0 tx=0).stop 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 -- 172.21.15.78:0/604750417 shutdown_connections 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fcfa20 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 --2- 172.21.15.78:0/604750417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x561559fce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 -- 172.21.15.78:0/604750417 >> 172.21.15.78:0/604750417 conn(0x561559c204e0 msgr2=0x561559fca020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 -- 172.21.15.78:0/604750417 shutdown_connections 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 -- 172.21.15.78:0/604750417 wait complete. 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.850+0000 7f0515666740 1 -- start start 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 1 Processor -- start 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c0b6b0 0x561559fd8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c2c390 0x561559fdfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561559fd0530 con 0x561559c2c390 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561559eb9dc0 con 0x561559c0b6b0 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561559fd06b0 con 0x561559c034d0 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0515666740 10 client.0 _ll_register_callbacks cb 0x561559ed5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:04.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c2c390 0x561559fdfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c2c390 0x561559fdfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55714/0 (socket says 172.21.15.78:55714) 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 -- 172.21.15.78:0/4046756412 learned_addr learned my addr 172.21.15.78:0/4046756412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0507fff640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f05077fe640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c0b6b0 0x561559fd8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 -- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 msgr2=0x561559fd8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fd8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 -- 172.21.15.78:0/4046756412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c0b6b0 msgr2=0x561559fd8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c0b6b0 0x561559fd8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f050cd4f640 1 -- 172.21.15.78:0/4046756412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561559c443f0 con 0x561559c2c390 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f05077fe640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c0b6b0 0x561559fd8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.851+0000 7f0507fff640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fd8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f050cd4f640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c2c390 0x561559fdfa20 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0500009ca0 tx=0x7f050002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8852452f013163f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f04e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:04.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f04e77fe640 1 -- 172.21.15.78:0/4046756412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500004310 con 0x561559c2c390 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f04e77fe640 1 -- 172.21.15.78:0/4046756412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f05000044b0 con 0x561559c2c390 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04fc0dee50 con 0x561559c2c390 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f04e77fe640 1 -- 172.21.15.78:0/4046756412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f05000046b0 con 0x561559c2c390 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 10 client.18612 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561559d0e7d0 con 0x561559c2c390 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f04e77fe640 1 -- 172.21.15.78:0/4046756412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc0dee50 con 0x561559c2c390 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f04e77fe640 1 -- 172.21.15.78:0/4046756412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f050005c050 con 0x561559c2c390 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 10 client.18612 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 10 client.18612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 20 client.18612 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559beaf60 0x561559fe62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 4 client.18612 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.852+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561559f40990 con 0x561559beaf60 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.853+0000 7f0507fff640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559beaf60 0x561559fe62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.853+0000 7f04e77fe640 10 client.18612 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.853+0000 7f0507fff640 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559beaf60 0x561559fe62a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee18ce6084cacfc2 server_cookie=52ec015ebced5e1e in_seq=0 out_seq=0 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f04e77fe640 1 -- 172.21.15.78:0/4046756412 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561559f40990 con 0x561559beaf60 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f04e77fe640 10 client.18612 handle_command_reply: tid=0 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 client.18612 shutdown 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c2c390 msgr2=0x561559fdfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c2c390 0x561559fdfa20 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0500009ca0 tx=0x7f050002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 shutdown_connections 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559beaf60 0x561559fe62a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c2c390 0x561559fdfa20 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c0b6b0 0x561559fd8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f04e77fe640 0 client.18612 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 --2- 172.21.15.78:0/4046756412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c034d0 0x561559fd8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 >> 172.21.15.78:0/4046756412 conn(0x561559c204e0 msgr2=0x561559fccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 shutdown_connections 2024-04-25T14:10:04.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.854+0000 7f0515666740 1 -- 172.21.15.78:0/4046756412 wait complete. 2024-04-25T14:10:04.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.855+0000 7f0515666740 20 client.18612 trim_cache size 0 max 16384 2024-04-25T14:10:04.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 Processor -- start 2024-04-25T14:10:04.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 -- start start 2024-04-25T14:10:04.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c034d0 0x561559fd38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fdf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c2c390 0x561559fcd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05081afd50 con 0x561559c0b6b0 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05081ad620 con 0x561559c034d0 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f0515666740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05081ab470 con 0x561559c2c390 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fdf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fdf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55720/0 (socket says 172.21.15.78:55720) 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 -- 172.21.15.78:0/2721421559 learned_addr learned my addr 172.21.15.78:0/2721421559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 -- 172.21.15.78:0/2721421559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c2c390 msgr2=0x561559fcd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 --2- 172.21.15.78:0/2721421559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c2c390 0x561559fcd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 -- 172.21.15.78:0/2721421559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c034d0 msgr2=0x561559fd38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 --2- 172.21.15.78:0/2721421559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c034d0 0x561559fd38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 -- 172.21.15.78:0/2721421559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561559fdfe80 con 0x561559c0b6b0 2024-04-25T14:10:04.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.868+0000 7f05077fe640 1 --2- 172.21.15.78:0/2721421559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fdf940 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0dc800 tx=0x7f04fc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f54992138f5e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f04edffb640 1 -- 172.21.15.78:0/2721421559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc01d040 con 0x561559c0b6b0 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f04edffb640 1 -- 172.21.15.78:0/2721421559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc0dee50 con 0x561559c0b6b0 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f04edffb640 1 -- 172.21.15.78:0/2721421559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0a67c0 con 0x561559c0b6b0 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 -- 172.21.15.78:0/2721421559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 msgr2=0x561559fdf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 --2- 172.21.15.78:0/2721421559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fdf940 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0dc800 tx=0x7f04fc005760 comp rx=0 tx=0).stop 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 -- 172.21.15.78:0/2721421559 shutdown_connections 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 --2- 172.21.15.78:0/2721421559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c2c390 0x561559fcd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 --2- 172.21.15.78:0/2721421559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c0b6b0 0x561559fdf940 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 --2- 172.21.15.78:0/2721421559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c034d0 0x561559fd38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 -- 172.21.15.78:0/2721421559 >> 172.21.15.78:0/2721421559 conn(0x561559c204e0 msgr2=0x561559fcce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 -- 172.21.15.78:0/2721421559 shutdown_connections 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.869+0000 7f0515666740 1 -- 172.21.15.78:0/2721421559 wait complete. 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 -- start start 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 Processor -- start 2024-04-25T14:10:04.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c034d0 0x561559fdca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c0b6b0 0x561559fdcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x56155a04c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05081abae0 con 0x561559c034d0 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05081acf50 con 0x561559c2c390 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05081ac1b0 con 0x561559c0b6b0 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0515666740 10 client.0 _ll_register_callbacks cb 0x7f050002f450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c034d0 0x561559fdca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c034d0 0x561559fdca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55734/0 (socket says 172.21.15.78:55734) 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0507fff640 1 -- 172.21.15.78:0/3398660876 learned_addr learned my addr 172.21.15.78:0/3398660876 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f05077fe640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c0b6b0 0x561559fdcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f050cd4f640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x56155a04c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.870+0000 7f0507fff640 1 -- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c0b6b0 msgr2=0x561559fdcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0507fff640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c0b6b0 0x561559fdcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0507fff640 1 -- 172.21.15.78:0/3398660876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 msgr2=0x56155a04c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0507fff640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x56155a04c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0507fff640 1 -- 172.21.15.78:0/3398660876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56155a04cbb0 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f05077fe640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c0b6b0 0x561559fdcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0507fff640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c034d0 0x561559fdca60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f04f8004650 tx=0x7f04f800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bf14d452e73568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f04e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f04e77fe640 1 -- 172.21.15.78:0/3398660876 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8005220 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f04e77fe640 1 -- 172.21.15.78:0/3398660876 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f04f80053c0 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04fc0dee50 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f04e77fe640 1 -- 172.21.15.78:0/3398660876 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f04f802e050 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 10 client.18618 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56155a04de80 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f04e77fe640 1 -- 172.21.15.78:0/3398660876 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc0dee50 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f04e77fe640 1 -- 172.21.15.78:0/3398660876 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f04f801f2b0 con 0x561559c034d0 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 10 client.18618 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:04.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 10 client.18618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 20 client.18618 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559bc03a0 0x56155a0508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 4 client.18618 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.871+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561559f3a800 con 0x561559bc03a0 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.872+0000 7f05077fe640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559bc03a0 0x56155a0508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.872+0000 7f05077fe640 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559bc03a0 0x56155a0508a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9722e6f28c47402b server_cookie=35f955a430e6a9b7 in_seq=0 out_seq=0 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.872+0000 7f04e77fe640 10 client.18618 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.872+0000 7f04e77fe640 1 -- 172.21.15.78:0/3398660876 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561559f3a800 con 0x561559bc03a0 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.872+0000 7f04e77fe640 10 client.18618 handle_command_reply: tid=0 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.872+0000 7f0515666740 1 client.18618 shutdown 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c034d0 msgr2=0x561559fdca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c034d0 0x561559fdca60 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f04f8004650 tx=0x7f04f800bea0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 shutdown_connections 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x561559bc03a0 0x56155a0508a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561559c2c390 0x56155a04c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561559c0b6b0 0x561559fdcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f04e77fe640 0 client.18618 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 --2- 172.21.15.78:0/3398660876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561559c034d0 0x561559fdca60 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 >> 172.21.15.78:0/3398660876 conn(0x561559c204e0 msgr2=0x561559fd7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 shutdown_connections 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 1 -- 172.21.15.78:0/3398660876 wait complete. 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f0515666740 20 client.18618 trim_cache size 0 max 16384 2024-04-25T14:10:04.879 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04f00afaf0 msgr2=0x7f04f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04f00afaf0 0x7f04f00b2000 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x561559fcf860 tx=0x7f0500031af0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508104d30 msgr2=0x7f050819c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508104d30 0x7f050819c8f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c890 tx=0x7f04fc00cd50 comp rx=0 tx=0).stop 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 shutdown_connections 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0508109a30 0x7f05081a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f04f00afaf0 0x7f04f00b2000 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0508104d30 0x7f050819c8f0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 --2- 172.21.15.78:0/828098339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0508103af0 0x7f050819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 >> 172.21.15.78:0/828098339 conn(0x7f05080fece0 msgr2=0x7f05080ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 shutdown_connections 2024-04-25T14:10:04.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.873+0000 7f050e7b2640 1 -- 172.21.15.78:0/828098339 wait complete. 2024-04-25T14:10:04.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 -- 172.21.15.78:0/2429717237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 msgr2=0x7f8044104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 --2- 172.21.15.78:0/2429717237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f8044104050 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f8038009900 tx=0x7f803802a410 comp rx=0 tx=0).stop 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 -- 172.21.15.78:0/2429717237 shutdown_connections 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 --2- 172.21.15.78:0/2429717237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8044108940 0x7f804410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 --2- 172.21.15.78:0/2429717237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f8044104050 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 --2- 172.21.15.78:0/2429717237 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8044102a00 0x7f8044103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 -- 172.21.15.78:0/2429717237 >> 172.21.15.78:0/2429717237 conn(0x7f80440fdbf0 msgr2=0x7f80441000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:04.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 -- 172.21.15.78:0/2429717237 shutdown_connections 2024-04-25T14:10:04.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.993+0000 7f804a497640 1 -- 172.21.15.78:0/2429717237 wait complete. 2024-04-25T14:10:04.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.994+0000 7f804a497640 1 Processor -- start 2024-04-25T14:10:04.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.994+0000 7f804a497640 1 -- start start 2024-04-25T14:10:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f804a497640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8044102a00 0x7f8044075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f804a497640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f804419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f804a497640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8044108940 0x7f80441a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f804a497640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f804410e530 con 0x7f8044103c40 2024-04-25T14:10:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f804a497640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f804410e3b0 con 0x7f8044108940 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f804a497640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f804410e6b0 con 0x7f8044102a00 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f804419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f804419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55744/0 (socket says 172.21.15.78:55744) 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 -- 172.21.15.78:0/418759890 learned_addr learned my addr 172.21.15.78:0/418759890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f8043fff640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8044102a00 0x7f8044075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f8048a34640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8044108940 0x7f80441a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 -- 172.21.15.78:0/418759890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8044102a00 msgr2=0x7f8044075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8044102a00 0x7f8044075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 -- 172.21.15.78:0/418759890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8044108940 msgr2=0x7f80441a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8044108940 0x7f80441a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 -- 172.21.15.78:0/418759890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80441a1db0 con 0x7f8044103c40 2024-04-25T14:10:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.995+0000 7f80437fe640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f804419f040 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f8038006010 tx=0x7f80380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:04.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.998+0000 7f80417fa640 1 -- 172.21.15.78:0/418759890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803803f030 con 0x7f8044103c40 2024-04-25T14:10:04.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.998+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80441a2040 con 0x7f8044103c40 2024-04-25T14:10:04.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.998+0000 7f80417fa640 1 -- 172.21.15.78:0/418759890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8038031910 con 0x7f8044103c40 2024-04-25T14:10:04.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.998+0000 7f80417fa640 1 -- 172.21.15.78:0/418759890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038035070 con 0x7f8044103c40 2024-04-25T14:10:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:04.998+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80441a2530 con 0x7f8044103c40 2024-04-25T14:10:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 Processor -- start 2024-04-25T14:10:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 -- start start 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f80417fa640 1 -- 172.21.15.78:0/418759890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f8038042030 con 0x7f8044103c40 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b738b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc04d0 0x55f2b738bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc86b0 0x55f2b738ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2b72efd80 con 0x55f2b6fe9390 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2b72a9a20 con 0x55f2b6fc86b0 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.001+0000 7f8051422740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f2b72ebd40 con 0x55f2b6fc04d0 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b738b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b738b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55750/0 (socket says 172.21.15.78:55750) 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 -- 172.21.15.78:0/2983660327 learned_addr learned my addr 172.21.15.78:0/2983660327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8043fff640 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc04d0 0x55f2b738bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f80437fe640 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc86b0 0x55f2b738ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 -- 172.21.15.78:0/2983660327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc04d0 msgr2=0x55f2b738bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc04d0 0x55f2b738bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 -- 172.21.15.78:0/2983660327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc86b0 msgr2=0x55f2b738ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc86b0 0x55f2b738ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 -- 172.21.15.78:0/2983660327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b72b4c30 con 0x55f2b6fe9390 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f80417fa640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f802c0afaf0 0x7f802c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8048a34640 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b738b810 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f803c009fd0 tx=0x7f803c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efb50ecaf0f1d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f8043fff640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f802c0afaf0 0x7f802c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.002+0000 7f80417fa640 1 -- 172.21.15.78:0/418759890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f80380ff0b0 con 0x7f8044103c40 2024-04-25T14:10:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.003+0000 7f8043fff640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f802c0afaf0 0x7f802c0b2000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55f2b738b530 tx=0x7f8034005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f802a7fc640 1 -- 172.21.15.78:0/2983660327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c004d20 con 0x55f2b6fe9390 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f802a7fc640 1 -- 172.21.15.78:0/2983660327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803c004050 con 0x55f2b6fe9390 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f802a7fc640 1 -- 172.21.15.78:0/2983660327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c005280 con 0x55f2b6fe9390 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f8051422740 1 -- 172.21.15.78:0/2983660327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 msgr2=0x55f2b738b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f8051422740 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b738b810 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f803c009fd0 tx=0x7f803c0118d0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f8051422740 1 -- 172.21.15.78:0/2983660327 shutdown_connections 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f8051422740 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc86b0 0x55f2b738ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f8051422740 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc04d0 0x55f2b738bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f8051422740 1 --2- 172.21.15.78:0/2983660327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b738b810 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.004+0000 7f8051422740 1 -- 172.21.15.78:0/2983660327 >> 172.21.15.78:0/2983660327 conn(0x55f2b6fdd4e0 msgr2=0x55f2b7387020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 -- 172.21.15.78:0/2983660327 shutdown_connections 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 -- 172.21.15.78:0/2983660327 wait complete. 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 -- start start 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 Processor -- start 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7395600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fc86b0 0x55f2b7395b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fe9390 0x55f2b739caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f2b738d530 con 0x55f2b6fc86b0 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f2b7276dc0 con 0x55f2b6fc04d0 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f2b738d6b0 con 0x55f2b6fe9390 2024-04-25T14:10:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.005+0000 7f8051422740 10 client.0 _ll_register_callbacks cb 0x55f2b7292600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fc86b0 0x55f2b7395b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fc86b0 0x55f2b7395b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55766/0 (socket says 172.21.15.78:55766) 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 -- 172.21.15.78:0/268217886 learned_addr learned my addr 172.21.15.78:0/268217886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f8043fff640 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7395600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f8048a34640 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fe9390 0x55f2b739caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 -- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fe9390 msgr2=0x55f2b739caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fe9390 0x55f2b739caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 -- 172.21.15.78:0/268217886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 msgr2=0x55f2b7395600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7395600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 -- 172.21.15.78:0/268217886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b70013f0 con 0x55f2b6fc86b0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80437fe640 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fc86b0 0x55f2b7395b40 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f80380c48a0 tx=0x7f80380c3570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff258ba37752df9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80237fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80237fe640 1 -- 172.21.15.78:0/268217886 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038049050 con 0x55f2b6fc86b0 2024-04-25T14:10:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80237fe640 1 -- 172.21.15.78:0/268217886 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f80380b00e0 con 0x55f2b6fc86b0 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803c004050 con 0x55f2b6fc86b0 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80237fe640 1 -- 172.21.15.78:0/268217886 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f80380044e0 con 0x55f2b6fc86b0 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f8051422740 10 client.18630 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f2b70cb7d0 con 0x55f2b6fc86b0 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.006+0000 7f80237fe640 1 -- 172.21.15.78:0/268217886 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803c004050 con 0x55f2b6fc86b0 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f80237fe640 1 -- 172.21.15.78:0/268217886 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f80380c38c0 con 0x55f2b6fc86b0 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f8051422740 10 client.18630 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f8051422740 10 client.18630 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f8051422740 20 client.18630 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f8051422740 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6fa7f60 0x55f2b73a3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f8051422740 4 client.18630 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b72fd990 con 0x55f2b6fa7f60 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.007+0000 7f8043fff640 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6fa7f60 0x55f2b73a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f80237fe640 10 client.18630 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f8043fff640 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6fa7f60 0x55f2b73a3160 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a2343fb59fe256 server_cookie=7e756fc2a624e154 in_seq=0 out_seq=0 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f80237fe640 1 -- 172.21.15.78:0/268217886 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f2b72fd990 con 0x55f2b6fa7f60 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f80237fe640 10 client.18630 handle_command_reply: tid=0 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f8051422740 1 client.18630 shutdown 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fc86b0 msgr2=0x55f2b7395b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f8051422740 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fc86b0 0x55f2b7395b40 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f80380c48a0 tx=0x7f80380c3570 comp rx=0 tx=0).stop 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 shutdown_connections 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.008+0000 7f8051422740 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6fa7f60 0x55f2b73a3160 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f8051422740 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fe9390 0x55f2b739caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f80237fe640 0 client.18630 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f8051422740 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fc86b0 0x55f2b7395b40 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f8051422740 1 --2- 172.21.15.78:0/268217886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7395600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 >> 172.21.15.78:0/268217886 conn(0x55f2b6fdd4e0 msgr2=0x55f2b7389f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 shutdown_connections 2024-04-25T14:10:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f8051422740 1 -- 172.21.15.78:0/268217886 wait complete. 2024-04-25T14:10:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.009+0000 7f8051422740 20 client.18630 trim_cache size 0 max 16384 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 Processor -- start 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 -- start start 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b73869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7386f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b7385610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80441adb20 con 0x55f2b6fe9390 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80441ab3f0 con 0x55f2b6fc04d0 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8051422740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80441a9240 con 0x55f2b6fc86b0 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.022+0000 7f8048a34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b7385610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b7385610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55778/0 (socket says 172.21.15.78:55778) 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 -- 172.21.15.78:0/3036317200 learned_addr learned my addr 172.21.15.78:0/3036317200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8043fff640 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b73869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f80437fe640 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7386f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 -- 172.21.15.78:0/3036317200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 msgr2=0x55f2b7386f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7386f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 -- 172.21.15.78:0/3036317200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 msgr2=0x55f2b73869e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b73869e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 -- 172.21.15.78:0/3036317200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b739cdb0 con 0x55f2b6fe9390 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8048a34640 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b7385610 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x55f2b738c860 tx=0x7f803c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11bf64e6122d596 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8029ffb640 1 -- 172.21.15.78:0/3036317200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c003db0 con 0x55f2b6fe9390 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8029ffb640 1 -- 172.21.15.78:0/3036317200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803c005570 con 0x55f2b6fe9390 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.023+0000 7f8029ffb640 1 -- 172.21.15.78:0/3036317200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c00fa20 con 0x55f2b6fe9390 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 -- 172.21.15.78:0/3036317200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 msgr2=0x55f2b7385610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b7385610 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x55f2b738c860 tx=0x7f803c009d30 comp rx=0 tx=0).stop 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 -- 172.21.15.78:0/3036317200 shutdown_connections 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b7385610 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7386f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 --2- 172.21.15.78:0/3036317200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b73869e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 -- 172.21.15.78:0/3036317200 >> 172.21.15.78:0/3036317200 conn(0x55f2b6fdd4e0 msgr2=0x55f2b7390330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 -- 172.21.15.78:0/3036317200 shutdown_connections 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 -- 172.21.15.78:0/3036317200 wait complete. 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 -- start start 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.024+0000 7f8051422740 1 Processor -- start 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7406a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7406f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b740ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80441a98b0 con 0x55f2b6fe9390 2024-04-25T14:10:05.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80441aad20 con 0x55f2b6fc04d0 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80441a9f80 con 0x55f2b6fc86b0 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 10 client.0 _ll_register_callbacks cb 0x7f8038005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b740ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b740ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55784/0 (socket says 172.21.15.78:55784) 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 -- 172.21.15.78:0/1843059180 learned_addr learned my addr 172.21.15.78:0/1843059180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8043fff640 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7406a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f80437fe640 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7406f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 -- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 msgr2=0x55f2b7406f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7406f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 -- 172.21.15.78:0/1843059180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 msgr2=0x55f2b7406a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7406a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 -- 172.21.15.78:0/1843059180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b740e490 con 0x55f2b6fe9390 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8048a34640 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b740ded0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f803c011910 tx=0x7f803c00f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dab82ee77a99dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f80237fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f80237fe640 1 -- 172.21.15.78:0/1843059180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c032070 con 0x55f2b6fe9390 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f80237fe640 1 -- 172.21.15.78:0/1843059180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f803c005270 con 0x55f2b6fe9390 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.025+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f803c005570 con 0x55f2b6fe9390 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f80237fe640 1 -- 172.21.15.78:0/1843059180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f803c03e050 con 0x55f2b6fe9390 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 10 client.18636 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f2b740f710 con 0x55f2b6fe9390 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f80237fe640 1 -- 172.21.15.78:0/1843059180 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803c005570 con 0x55f2b6fe9390 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f80237fe640 1 -- 172.21.15.78:0/1843059180 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f803c025850 con 0x55f2b6fe9390 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 10 client.18636 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 10 client.18636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 20 client.18636 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6f7d3a0 0x55f2b7411f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 4 client.18636 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2b72f7800 con 0x55f2b6f7d3a0 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.026+0000 7f8043fff640 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6f7d3a0 0x55f2b7411f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f80237fe640 10 client.18636 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8043fff640 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6f7d3a0 0x55f2b7411f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24fae89a958ef564 server_cookie=8b5f26d1fa62518 in_seq=0 out_seq=0 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f80237fe640 1 -- 172.21.15.78:0/1843059180 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2b72f7800 con 0x55f2b6f7d3a0 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f80237fe640 10 client.18636 handle_command_reply: tid=0 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 client.18636 shutdown 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 msgr2=0x55f2b740ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b740ded0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f803c011910 tx=0x7f803c00f760 comp rx=0 tx=0).stop 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 shutdown_connections 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f2b6fe9390 0x55f2b740ded0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55f2b6f7d3a0 0x55f2b7411f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f2b6fc86b0 0x55f2b7406f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f80237fe640 0 client.18636 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 --2- 172.21.15.78:0/1843059180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f2b6fc04d0 0x55f2b7406a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 >> 172.21.15.78:0/1843059180 conn(0x55f2b6fdd4e0 msgr2=0x55f2b7389e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 shutdown_connections 2024-04-25T14:10:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 1 -- 172.21.15.78:0/1843059180 wait complete. 2024-04-25T14:10:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.027+0000 7f8051422740 20 client.18636 trim_cache size 0 max 16384 2024-04-25T14:10:05.034 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:10:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f802c0afaf0 msgr2=0x7f802c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f802c0afaf0 0x7f802c0b2000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55f2b738b530 tx=0x7f8034005f90 comp rx=0 tx=0).stop 2024-04-25T14:10:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 msgr2=0x7f804419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f804419f040 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f8038006010 tx=0x7f80380042f0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 shutdown_connections 2024-04-25T14:10:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8044108940 0x7f80441a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f802c0afaf0 0x7f802c0b2000 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8044103c40 0x7f804419f040 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 --2- 172.21.15.78:0/418759890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8044102a00 0x7f8044075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 >> 172.21.15.78:0/418759890 conn(0x7f80440fdbf0 msgr2=0x7f8044104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 shutdown_connections 2024-04-25T14:10:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.028+0000 7f804a497640 1 -- 172.21.15.78:0/418759890 wait complete. 2024-04-25T14:10:05.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:10:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 -- 172.21.15.78:0/1808149746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98109a40 msgr2=0x7f0a9810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1808149746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98109a40 0x7f0a9810be60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009ca0 tx=0x7f0a9002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 -- 172.21.15.78:0/1808149746 shutdown_connections 2024-04-25T14:10:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1808149746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98109a40 0x7f0a9810be60 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1808149746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a98104d40 0x7f0a98105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1808149746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98103b00 0x7f0a98104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 -- 172.21.15.78:0/1808149746 >> 172.21.15.78:0/1808149746 conn(0x7f0a980fed10 msgr2=0x7f0a981011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 -- 172.21.15.78:0/1808149746 shutdown_connections 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.148+0000 7f0aa0735640 1 -- 172.21.15.78:0/1808149746 wait complete. 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 Processor -- start 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 -- start start 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98103b00 0x7f0a9819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a98104d40 0x7f0a9819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98109a40 0x7f0a981a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810f610 con 0x7f0a98103b00 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810f490 con 0x7f0a98109a40 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0aa0735640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810f790 con 0x7f0a98104d40 2024-04-25T14:10:05.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98103b00 0x7f0a9819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98103b00 0x7f0a9819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55816/0 (socket says 172.21.15.78:55816) 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1819228474 learned_addr learned my addr 172.21.15.78:0/1819228474 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9ecd2640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98109a40 0x7f0a981a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a98104d40 0x7f0a9819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1819228474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a98104d40 msgr2=0x7f0a9819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a98104d40 0x7f0a9819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1819228474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98109a40 msgr2=0x7f0a981a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98109a40 0x7f0a981a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.149+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1819228474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a981a1d50 con 0x7f0a98103b00 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.150+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98103b00 0x7f0a9819a210 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d740 tx=0x7f0a9400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.150+0000 7f0a8f7fe640 1 -- 172.21.15.78:0/1819228474 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94017070 con 0x7f0a98103b00 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.150+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a981a1fe0 con 0x7f0a98103b00 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.150+0000 7f0a8f7fe640 1 -- 172.21.15.78:0/1819228474 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a9400de50 con 0x7f0a98103b00 2024-04-25T14:10:05.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.150+0000 7f0a8f7fe640 1 -- 172.21.15.78:0/1819228474 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94003c30 con 0x7f0a98103b00 2024-04-25T14:10:05.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.150+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a981a24f0 con 0x7f0a98103b00 2024-04-25T14:10:05.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.152+0000 7f0aa75df740 1 Processor -- start 2024-04-25T14:10:05.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.152+0000 7f0a8f7fe640 1 -- 172.21.15.78:0/1819228474 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0a94003dd0 con 0x7f0a98103b00 2024-04-25T14:10:05.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.152+0000 7f0aa75df740 1 -- start start 2024-04-25T14:10:05.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337593a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56337556f4d0 0x56337593ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633755776b0 0x56337593ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0aa75df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56337589ed80 con 0x56337556f4d0 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0aa75df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563375858a20 con 0x563375598390 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0aa75df740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56337589ad40 con 0x5633755776b0 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56337556f4d0 0x56337593ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56337556f4d0 0x56337593ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55818/0 (socket says 172.21.15.78:55818) 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1444248985 learned_addr learned my addr 172.21.15.78:0/1444248985 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9ecd2640 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337593a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633755776b0 0x56337593ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a8f7fe640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a840afaf0 0x7f0a840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1444248985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633755776b0 msgr2=0x56337593ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633755776b0 0x56337593ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1444248985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 msgr2=0x56337593a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337593a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 -- 172.21.15.78:0/1444248985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563375863c30 con 0x56337556f4d0 2024-04-25T14:10:05.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9ecd2640 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337593a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9ecd2640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a840afaf0 0x7f0a840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a8f7fe640 1 -- 172.21.15.78:0/1819228474 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0a940de5e0 con 0x7f0a98103b00 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.153+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56337556f4d0 0x56337593ad90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f0a94085a40 tx=0x7f0a9406d080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b46324d2e0a0a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0a8cff9640 1 -- 172.21.15.78:0/1444248985 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a940a6930 con 0x56337556f4d0 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0a9ecd2640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a840afaf0 0x7f0a840b2000 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56337593b860 tx=0x7f0a900095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0a8cff9640 1 -- 172.21.15.78:0/1444248985 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a940a6670 con 0x56337556f4d0 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0a8cff9640 1 -- 172.21.15.78:0/1444248985 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94085640 con 0x56337556f4d0 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0aa75df740 1 -- 172.21.15.78:0/1444248985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56337556f4d0 msgr2=0x56337593ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56337556f4d0 0x56337593ad90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f0a94085a40 tx=0x7f0a9406d080 comp rx=0 tx=0).stop 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0aa75df740 1 -- 172.21.15.78:0/1444248985 shutdown_connections 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5633755776b0 0x56337593ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56337556f4d0 0x56337593ad90 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1444248985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337593a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.154+0000 7f0aa75df740 1 -- 172.21.15.78:0/1444248985 >> 172.21.15.78:0/1444248985 conn(0x56337558c4e0 msgr2=0x563375936020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 -- 172.21.15.78:0/1444248985 shutdown_connections 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 -- 172.21.15.78:0/1444248985 wait complete. 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 -- start start 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 Processor -- start 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375944500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375944a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 0x56337594b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56337593c4d0 con 0x5633755776b0 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563375825dc0 con 0x56337556f4d0 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56337593c650 con 0x563375598390 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0aa75df740 10 client.0 _ll_register_callbacks cb 0x563375841600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0a9dcd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375944a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0a9dcd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375944a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55834/0 (socket says 172.21.15.78:55834) 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/754920945 learned_addr learned my addr 172.21.15.78:0/754920945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375944500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.155+0000 7f0a9ecd2640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 0x56337594b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/754920945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 msgr2=0x56337594b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 0x56337594b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 msgr2=0x563375944500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375944500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/754920945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633755b03f0 con 0x5633755776b0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375944500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375944a40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800bf80 tx=0x7f0a880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc14d06384aafbf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a81ffb640 1 -- 172.21.15.78:0/754920945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88004020 con 0x5633755776b0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a81ffb640 1 -- 172.21.15.78:0/754920945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0a880041c0 con 0x5633755776b0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a940a6670 con 0x5633755776b0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a81ffb640 1 -- 172.21.15.78:0/754920945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0a880043c0 con 0x5633755776b0 2024-04-25T14:10:05.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0aa75df740 10 client.18648 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56337567a7d0 con 0x5633755776b0 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a81ffb640 1 -- 172.21.15.78:0/754920945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a940a6670 con 0x5633755776b0 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.156+0000 7f0a81ffb640 1 -- 172.21.15.78:0/754920945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0a880107c0 con 0x5633755776b0 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0aa75df740 10 client.18648 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0aa75df740 10 client.18648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0aa75df740 20 client.18648 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0aa75df740 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563375556f60 0x5633759521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0aa75df740 4 client.18648 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633758ac990 con 0x563375556f60 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563375556f60 0x5633759521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0a81ffb640 10 client.18648 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.157+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563375556f60 0x5633759521e0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198bae5edc9b6051 server_cookie=53c3dc85f077ad41 in_seq=0 out_seq=0 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0a81ffb640 1 -- 172.21.15.78:0/754920945 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633758ac990 con 0x563375556f60 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0a81ffb640 10 client.18648 handle_command_reply: tid=0 2024-04-25T14:10:05.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 client.18648 shutdown 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 msgr2=0x563375944a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375944a40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800bf80 tx=0x7f0a880118d0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 shutdown_connections 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x563375556f60 0x5633759521e0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 0x56337594b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375944a40 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0a81ffb640 0 client.18648 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 --2- 172.21.15.78:0/754920945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375944500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 >> 172.21.15.78:0/754920945 conn(0x56337558c4e0 msgr2=0x563375938f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 shutdown_connections 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 1 -- 172.21.15.78:0/754920945 wait complete. 2024-04-25T14:10:05.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.158+0000 7f0aa75df740 20 client.18648 trim_cache size 0 max 16384 2024-04-25T14:10:05.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 Processor -- start 2024-04-25T14:10:05.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 -- start start 2024-04-25T14:10:05.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375934610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x56337594b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 0x563375934b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981adc90 con 0x5633755776b0 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981ab560 con 0x56337556f4d0 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0aa75df740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981a93b0 con 0x563375598390 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x56337594b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x56337594b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55840/0 (socket says 172.21.15.78:55840) 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/167624111 learned_addr learned my addr 172.21.15.78:0/167624111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/167624111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 msgr2=0x563375934b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/167624111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 0x563375934b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/167624111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 msgr2=0x563375934610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/167624111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375934610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/167624111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633759429e0 con 0x5633755776b0 2024-04-25T14:10:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/167624111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x56337594b850 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001eb0 tx=0x7f0a8800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff64496ea494df31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a83fff640 1 -- 172.21.15.78:0/167624111 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88000e60 con 0x5633755776b0 2024-04-25T14:10:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a83fff640 1 -- 172.21.15.78:0/167624111 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a88010da0 con 0x5633755776b0 2024-04-25T14:10:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.172+0000 7f0a83fff640 1 -- 172.21.15.78:0/167624111 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8802b040 con 0x5633755776b0 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 -- 172.21.15.78:0/167624111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 msgr2=0x56337594b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 --2- 172.21.15.78:0/167624111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x56337594b850 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001eb0 tx=0x7f0a8800b5c0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 -- 172.21.15.78:0/167624111 shutdown_connections 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 --2- 172.21.15.78:0/167624111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563375598390 0x563375934b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 --2- 172.21.15.78:0/167624111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x56337594b850 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 --2- 172.21.15.78:0/167624111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56337556f4d0 0x563375934610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 -- 172.21.15.78:0/167624111 >> 172.21.15.78:0/167624111 conn(0x56337558c4e0 msgr2=0x56337593f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 -- 172.21.15.78:0/167624111 shutdown_connections 2024-04-25T14:10:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.173+0000 7f0aa75df740 1 -- 172.21.15.78:0/167624111 wait complete. 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 -- start start 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 Processor -- start 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56337556f4d0 0x5633759490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375949620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337594a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981a9a20 con 0x5633755776b0 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981aae90 con 0x563375598390 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981aa0f0 con 0x56337556f4d0 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0aa75df740 10 client.0 _ll_register_callbacks cb 0x7f0a8800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375949620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375949620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55844/0 (socket says 172.21.15.78:55844) 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/1881625584 learned_addr learned my addr 172.21.15.78:0/1881625584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56337556f4d0 0x5633759490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9ecd2640 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337594a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/1881625584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56337556f4d0 msgr2=0x5633759490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56337556f4d0 0x5633759490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 msgr2=0x56337594a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337594a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.174+0000 7f0a9dcd0640 1 -- 172.21.15.78:0/1881625584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337594bd90 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a9dcd0640 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375949620 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0a88010d20 tx=0x7f0a88005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71acd8ca4ee806e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a81ffb640 1 -- 172.21.15.78:0/1881625584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8802b070 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a81ffb640 1 -- 172.21.15.78:0/1881625584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f0a880048e0 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a88010da0 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a81ffb640 1 -- 172.21.15.78:0/1881625584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0a88000e40 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 10 client.18654 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56337594bbc0 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a81ffb640 1 -- 172.21.15.78:0/1881625584 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a88010da0 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a81ffb640 1 -- 172.21.15.78:0/1881625584 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0a88025940 con 0x5633755776b0 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 10 client.18654 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 10 client.18654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 20 client.18654 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56337552c3a0 0x5633759bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 4 client.18654 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633759435c0 con 0x56337552c3a0 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.175+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56337552c3a0 0x5633759bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0a81ffb640 10 client.18654 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0a9e4d1640 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56337552c3a0 0x5633759bcd00 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57279f187809b9e server_cookie=bdb467cb2745f31b in_seq=0 out_seq=0 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0a81ffb640 1 -- 172.21.15.78:0/1881625584 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633759435c0 con 0x56337552c3a0 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0a81ffb640 10 client.18654 handle_command_reply: tid=0 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 client.18654 shutdown 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 msgr2=0x563375949620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375949620 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0a88010d20 tx=0x7f0a88005660 comp rx=0 tx=0).stop 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 shutdown_connections 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563375598390 0x56337594a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5633755776b0 0x563375949620 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56337552c3a0 0x5633759bcd00 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 --2- 172.21.15.78:0/1881625584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56337556f4d0 0x5633759490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0a81ffb640 0 client.18654 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 >> 172.21.15.78:0/1881625584 conn(0x56337558c4e0 msgr2=0x56337593f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 shutdown_connections 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 1 -- 172.21.15.78:0/1881625584 wait complete. 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.176+0000 7f0aa75df740 20 client.18654 trim_cache size 0 max 16384 2024-04-25T14:10:05.183 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:10:05.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a840afaf0 msgr2=0x7f0a840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a840afaf0 0x7f0a840b2000 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56337593b860 tx=0x7f0a900095c0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98103b00 msgr2=0x7f0a9819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:05.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98103b00 0x7f0a9819a210 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d740 tx=0x7f0a9400dc00 comp rx=0 tx=0).stop 2024-04-25T14:10:05.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 shutdown_connections 2024-04-25T14:10:05.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a98109a40 0x7f0a981a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a98104d40 0x7f0a9819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f0a840afaf0 0x7f0a840b2000 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 --2- 172.21.15.78:0/1819228474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a98103b00 0x7f0a9819a210 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 >> 172.21.15.78:0/1819228474 conn(0x7f0a980fed10 msgr2=0x7f0a98105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 shutdown_connections 2024-04-25T14:10:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:05.177+0000 7f0aa0735640 1 -- 172.21.15.78:0/1819228474 wait complete. 2024-04-25T14:10:05.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:05 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2948316589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:05.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:05 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2948316589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:05.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:05 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2948316589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:06.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:06 smithi112 ceph-mon[32617]: pgmap v665: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:10:06.084 INFO:tasks.workunit.client.0.smithi078.stdout:499/600: open fd = 3 2024-04-25T14:10:06.084 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.084 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.084 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.084 INFO:tasks.workunit.client.0.smithi078.stdout:500/600: open fd = 3 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:501/600: open fd = 3 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:502/600: open fd = 3 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:503/600: open fd = 3 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.085 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:504/600: open fd = 3 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:505/600: open fd = 3 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:506/600: open fd = 3 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.086 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:507/600: open fd = 3 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:508/600: open fd = 3 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:509/600: open fd = 3 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:510/600: open fd = 3 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.087 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:511/600: open fd = 3 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:512/600: open fd = 3 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:513/600: open fd = 3 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.088 INFO:tasks.workunit.client.0.smithi078.stdout:514/600: open fd = 3 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:515/600: open fd = 3 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:516/600: open fd = 3 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:517/600: open fd = 3 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.089 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:518/600: open fd = 3 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:519/600: open fd = 3 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:520/600: open fd = 3 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.090 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:521/600: open fd = 3 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:522/600: open fd = 3 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:523/600: open fd = 3 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:524/600: open fd = 3 2024-04-25T14:10:06.091 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:525/600: open fd = 3 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:526/600: open fd = 3 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:527/600: open fd = 3 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.092 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:528/600: open fd = 3 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:529/600: open fd = 3 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:530/600: open fd = 3 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:531/600: open fd = 3 2024-04-25T14:10:06.093 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:532/600: open fd = 3 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:533/600: open fd = 3 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:534/600: open fd = 3 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.094 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:535/600: open fd = 3 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:536/600: open fd = 3 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:537/600: open fd = 3 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:538/600: open fd = 3 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.095 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:539/600: open fd = 3 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:540/600: open fd = 3 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:541/600: open fd = 3 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:542/600: open fd = 3 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.096 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:543/600: open fd = 3 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:544/600: open fd = 3 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:545/600: open fd = 3 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:546/600: open fd = 3 2024-04-25T14:10:06.097 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:547/600: open fd = 3 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:548/600: open fd = 3 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:549/600: open fd = 3 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.098 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:550/600: open fd = 3 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:551/600: open fd = 3 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:552/600: open fd = 3 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:553/600: open fd = 3 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.099 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:554/600: open fd = 3 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:555/600: open fd = 3 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:556/600: open fd = 3 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:557/600: open fd = 3 2024-04-25T14:10:06.100 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:558/600: open fd = 3 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:559/600: open fd = 3 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:560/600: open fd = 3 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:06.101 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:06.188 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:10:06.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:06.188 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 -- 172.21.15.78:0/81642359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce4108920 msgr2=0x7fbce410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 --2- 172.21.15.78:0/81642359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce4108920 0x7fbce410ad40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009ca0 tx=0x7fbcdc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 -- 172.21.15.78:0/81642359 shutdown_connections 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 --2- 172.21.15.78:0/81642359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce4108920 0x7fbce410ad40 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 --2- 172.21.15.78:0/81642359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce4103c20 0x7fbce4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 --2- 172.21.15.78:0/81642359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce41029e0 0x7fbce41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 -- 172.21.15.78:0/81642359 >> 172.21.15.78:0/81642359 conn(0x7fbce40fdbd0 msgr2=0x7fbce4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 -- 172.21.15.78:0/81642359 shutdown_connections 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 -- 172.21.15.78:0/81642359 wait complete. 2024-04-25T14:10:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.309+0000 7fbcec32d640 1 Processor -- start 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcec32d640 1 -- start start 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcec32d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce41029e0 0x7fbce419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcec32d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4103c20 0x7fbce419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcec32d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce4108920 0x7fbce41a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcec32d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbce410f610 con 0x7fbce41029e0 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcec32d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbce410f490 con 0x7fbce4108920 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcec32d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbce410f790 con 0x7fbce4103c20 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce41029e0 0x7fbce419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce41029e0 0x7fbce419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55862/0 (socket says 172.21.15.78:55862) 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3051368860 learned_addr learned my addr 172.21.15.78:0/3051368860 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea8ca640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce4108920 0x7fbce41a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4103c20 0x7fbce419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3051368860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4103c20 msgr2=0x7fbce419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4103c20 0x7fbce419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3051368860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce4108920 msgr2=0x7fbce41a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce4108920 0x7fbce41a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3051368860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce41a1f50 con 0x7fbce41029e0 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea8ca640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce4108920 0x7fbce41a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce41029e0 0x7fbce419a4c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c970 tx=0x7fbcd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.310+0000 7fbcd77fe640 1 -- 172.21.15.78:0/3051368860 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8007c70 con 0x7fbce41029e0 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.311+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce41a2240 con 0x7fbce41029e0 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.311+0000 7fbcd77fe640 1 -- 172.21.15.78:0/3051368860 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8007e10 con 0x7fbce41029e0 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.311+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce41a2790 con 0x7fbce41029e0 2024-04-25T14:10:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.311+0000 7fbcd77fe640 1 -- 172.21.15.78:0/3051368860 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8018780 con 0x7fbce41029e0 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.313+0000 7fbcd77fe640 1 -- 172.21.15.78:0/3051368860 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbcd800fab0 con 0x7fbce41029e0 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.313+0000 7fbcf31dd740 1 Processor -- start 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcf31dd740 1 -- start start 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4cb7410 0x55d0f50584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c8e550 0x55d0f5058a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c96730 0x55d0f505f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d0f4fbdb00 con 0x55d0f4c96730 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0f4f59920 con 0x55d0f4cb7410 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d0f4fb9ac0 con 0x55d0f4c8e550 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcd77fe640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcd00afaf0 0x7fbcd00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c96730 0x55d0f505f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c96730 0x55d0f505f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55870/0 (socket says 172.21.15.78:55870) 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 -- 172.21.15.78:0/3708093853 learned_addr learned my addr 172.21.15.78:0/3708093853 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c8e550 0x55d0f5058a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcea8ca640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcd00afaf0 0x7fbcd00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcea8ca640 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4cb7410 0x55d0f50584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbcd77fe640 1 -- 172.21.15.78:0/3051368860 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbcd80ded10 con 0x7fbce41029e0 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 -- 172.21.15.78:0/3708093853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c8e550 msgr2=0x55d0f5058a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c8e550 0x55d0f5058a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 -- 172.21.15.78:0/3708093853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4cb7410 msgr2=0x55d0f50584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4cb7410 0x55d0f50584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.314+0000 7fbce98c8640 1 -- 172.21.15.78:0/3708093853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f4d977e0 con 0x55d0f4c96730 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.315+0000 7fbcea8ca640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcd00afaf0 0x7fbcd00b2000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55d0f5059670 tx=0x7fbcdc02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.315+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c96730 0x55d0f505f920 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fbce000bc90 tx=0x7fbce000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1a708b453ebc0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.315+0000 7fbcd4bfe640 1 -- 172.21.15.78:0/3708093853 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0015070 con 0x55d0f4c96730 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.315+0000 7fbcd4bfe640 1 -- 172.21.15.78:0/3708093853 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce001a910 con 0x55d0f4c96730 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.315+0000 7fbcd4bfe640 1 -- 172.21.15.78:0/3708093853 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0011890 con 0x55d0f4c96730 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3708093853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c96730 msgr2=0x55d0f505f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c96730 0x55d0f505f920 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fbce000bc90 tx=0x7fbce000f730 comp rx=0 tx=0).stop 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3708093853 shutdown_connections 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c96730 0x55d0f505f920 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c8e550 0x55d0f5058a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3708093853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4cb7410 0x55d0f50584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3708093853 >> 172.21.15.78:0/3708093853 conn(0x55d0f4cab560 msgr2=0x55d0f5053e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3708093853 shutdown_connections 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3708093853 wait complete. 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- start start 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 Processor -- start 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f5062290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f50627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4cb7410 0x55d0f5069730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d0f5060370 con 0x55d0f4c8e550 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d0f4f575d0 con 0x55d0f4c96730 2024-04-25T14:10:06.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.316+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d0f50604f0 con 0x55d0f4cb7410 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcf31dd740 10 client.0 _ll_register_callbacks cb 0x55d0f4f45080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f5062290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f5062290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55884/0 (socket says 172.21.15.78:55884) 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 -- 172.21.15.78:0/411044154 learned_addr learned my addr 172.21.15.78:0/411044154 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea8ca640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4cb7410 0x55d0f5069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbce98c8640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f50627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 -- 172.21.15.78:0/411044154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4cb7410 msgr2=0x55d0f5069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4cb7410 0x55d0f5069730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 -- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 msgr2=0x55d0f50627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f50627d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 -- 172.21.15.78:0/411044154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f4df7f70 con 0x55d0f4c8e550 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbce98c8640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f50627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f5062290 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800fe90 tx=0x7fbcd8072930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e16da41a4700a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:06.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbccdffb640 1 -- 172.21.15.78:0/411044154 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80a6cf0 con 0x55d0f4c8e550 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbccdffb640 1 -- 172.21.15.78:0/411044154 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbcd80a6650 con 0x55d0f4c8e550 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce001a910 con 0x55d0f4c8e550 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbccdffb640 1 -- 172.21.15.78:0/411044154 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbcd80a78e0 con 0x55d0f4c8e550 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcf31dd740 10 client.18666 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d0f4c8f170 con 0x55d0f4c8e550 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.317+0000 7fbccdffb640 1 -- 172.21.15.78:0/411044154 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce001a910 con 0x55d0f4c8e550 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbccdffb640 1 -- 172.21.15.78:0/411044154 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbcd8020030 con 0x55d0f4c8e550 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbcf31dd740 10 client.18666 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbcf31dd740 10 client.18666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbcf31dd740 20 client.18666 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4c761f0 0x55d0f506fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbcf31dd740 4 client.18666 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0f4fcb710 con 0x55d0f4c761f0 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbce98c8640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4c761f0 0x55d0f506fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbccdffb640 10 client.18666 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.318+0000 7fbce98c8640 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4c761f0 0x55d0f506fc10 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580461e9af606c86 server_cookie=58576ed5c6feed13 in_seq=0 out_seq=0 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbccdffb640 1 -- 172.21.15.78:0/411044154 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0f4fcb710 con 0x55d0f4c761f0 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbccdffb640 10 client.18666 handle_command_reply: tid=0 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 client.18666 shutdown 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 msgr2=0x55d0f5062290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f5062290 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800fe90 tx=0x7fbcd8072930 comp rx=0 tx=0).stop 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 shutdown_connections 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4c761f0 0x55d0f506fc10 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4cb7410 0x55d0f5069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f50627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbccdffb640 0 client.18666 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/411044154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f5062290 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 >> 172.21.15.78:0/411044154 conn(0x55d0f4cab560 msgr2=0x55d0f5052ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 shutdown_connections 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.319+0000 7fbcf31dd740 1 -- 172.21.15.78:0/411044154 wait complete. 2024-04-25T14:10:06.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.320+0000 7fbcf31dd740 20 client.18666 trim_cache size 0 max 16384 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 Processor -- start 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 -- start start 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c761f0 0x55d0f5062570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f50696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f5069c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41ada20 con 0x55d0f4c8e550 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41ab2f0 con 0x55d0f4c96730 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.337+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a9140 con 0x55d0f4c761f0 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f50696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f50696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55890/0 (socket says 172.21.15.78:55890) 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 -- 172.21.15.78:0/1281372722 learned_addr learned my addr 172.21.15.78:0/1281372722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 -- 172.21.15.78:0/1281372722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c761f0 msgr2=0x55d0f5062570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c761f0 0x55d0f5062570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c761f0 0x55d0f5062570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 -- 172.21.15.78:0/1281372722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 msgr2=0x55d0f5069c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f5069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 -- 172.21.15.78:0/1281372722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f504e160 con 0x55d0f4c8e550 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c761f0 0x55d0f5062570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbce98c8640 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f50696c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55d0f5054bc0 tx=0x7fbce000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9d246a19128a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbccffff640 1 -- 172.21.15.78:0/1281372722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0015070 con 0x55d0f4c8e550 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbccffff640 1 -- 172.21.15.78:0/1281372722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce002b070 con 0x55d0f4c8e550 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.338+0000 7fbccffff640 1 -- 172.21.15.78:0/1281372722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004900 con 0x55d0f4c8e550 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 -- 172.21.15.78:0/1281372722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 msgr2=0x55d0f50696c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f50696c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55d0f5054bc0 tx=0x7fbce000b5a0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 -- 172.21.15.78:0/1281372722 shutdown_connections 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c96730 0x55d0f5069c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c8e550 0x55d0f50696c0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/1281372722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c761f0 0x55d0f5062570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 -- 172.21.15.78:0/1281372722 >> 172.21.15.78:0/1281372722 conn(0x55d0f4cab560 msgr2=0x55d0f5061e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 -- 172.21.15.78:0/1281372722 shutdown_connections 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 -- 172.21.15.78:0/1281372722 wait complete. 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 -- start start 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 Processor -- start 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.339+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c761f0 0x55d0f50670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c8e550 0x55d0f50675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c96730 0x55d0f5068520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a97b0 con 0x55d0f4c761f0 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41aac20 con 0x55d0f4c8e550 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41a9e80 con 0x55d0f4c96730 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 10 client.0 _ll_register_callbacks cb 0x7fbcd80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:06.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c761f0 0x55d0f50670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c761f0 0x55d0f50670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55894/0 (socket says 172.21.15.78:55894) 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3007220939 learned_addr learned my addr 172.21.15.78:0/3007220939 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c8e550 0x55d0f50675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea8ca640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c96730 0x55d0f5068520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3007220939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c96730 msgr2=0x55d0f5068520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c96730 0x55d0f5068520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c8e550 msgr2=0x55d0f50675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c8e550 0x55d0f50675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 -- 172.21.15.78:0/3007220939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f5068ae0 con 0x55d0f4c761f0 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c8e550 0x55d0f50675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcea0c9640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c761f0 0x55d0f50670b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80a6ee0 tx=0x7fbcd80a31c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bf4230922bb258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbccdffb640 1 -- 172.21.15.78:0/3007220939 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80a71e0 con 0x55d0f4c761f0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbccdffb640 1 -- 172.21.15.78:0/3007220939 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbcd80a2ba0 con 0x55d0f4c761f0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce002b070 con 0x55d0f4c761f0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbccdffb640 1 -- 172.21.15.78:0/3007220939 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbcd80a2da0 con 0x55d0f4c761f0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 10 client.18672 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.340+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d0f5069d70 con 0x55d0f4c761f0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbccdffb640 1 -- 172.21.15.78:0/3007220939 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce002b070 con 0x55d0f4c761f0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbccdffb640 1 -- 172.21.15.78:0/3007220939 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbcd801f360 con 0x55d0f4c761f0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbcf31dd740 10 client.18672 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbcf31dd740 10 client.18672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbcf31dd740 20 client.18672 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4cb7410 0x55d0f50da550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbcf31dd740 4 client.18672 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d0f4fc5580 con 0x55d0f4cb7410 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.341+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4cb7410 0x55d0f50da550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.342+0000 7fbccdffb640 10 client.18672 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.342+0000 7fbce98c8640 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4cb7410 0x55d0f50da550 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dee1f5de0e381b5 server_cookie=dfbe6c7f1e94115a in_seq=0 out_seq=0 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbccdffb640 1 -- 172.21.15.78:0/3007220939 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d0f4fc5580 con 0x55d0f4cb7410 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbccdffb640 10 client.18672 handle_command_reply: tid=0 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 client.18672 shutdown 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c761f0 msgr2=0x55d0f50670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c761f0 0x55d0f50670b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80a6ee0 tx=0x7fbcd80a31c0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 shutdown_connections 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55d0f4cb7410 0x55d0f50da550 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0f4c96730 0x55d0f5068520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d0f4c8e550 0x55d0f50675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 --2- 172.21.15.78:0/3007220939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d0f4c761f0 0x55d0f50670b0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 >> 172.21.15.78:0/3007220939 conn(0x55d0f4cab560 msgr2=0x55d0f50625f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbccdffb640 0 client.18672 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 shutdown_connections 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 1 -- 172.21.15.78:0/3007220939 wait complete. 2024-04-25T14:10:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcf31dd740 20 client.18672 trim_cache size 0 max 16384 2024-04-25T14:10:06.349 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:10:06.349 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:10:06.349 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "84c4532c-14ee-4185-95c6-778a711c85db", 2024-04-25T14:10:06.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:10:06.349 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcd00afaf0 msgr2=0x7fbcd00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcd00afaf0 0x7fbcd00b2000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55d0f5059670 tx=0x7fbcdc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce41029e0 msgr2=0x7fbce419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce41029e0 0x7fbce419a4c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c970 tx=0x7fbcd800ce30 comp rx=0 tx=0).stop 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 shutdown_connections 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbce4108920 0x7fbce41a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4103c20 0x7fbce419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbcd00afaf0 0x7fbcd00b2000 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.343+0000 7fbcec32d640 1 --2- 172.21.15.78:0/3051368860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbce41029e0 0x7fbce419a4c0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.344+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 >> 172.21.15.78:0/3051368860 conn(0x7fbce40fdbd0 msgr2=0x7fbce4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.344+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 shutdown_connections 2024-04-25T14:10:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.344+0000 7fbcec32d640 1 -- 172.21.15.78:0/3051368860 wait complete. 2024-04-25T14:10:06.353 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:06 smithi078 ceph-mon[29163]: pgmap v665: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:10:06.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:10:06.354 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:10:06.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:06 smithi129 ceph-mon[32569]: pgmap v665: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-25T14:10:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.464+0000 7f210b90d640 1 -- 172.21.15.78:0/853633145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 msgr2=0x7f210410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.464+0000 7f210b90d640 1 --2- 172.21.15.78:0/853633145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f210410bcd0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009ca0 tx=0x7f20fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.464+0000 7f210b90d640 1 -- 172.21.15.78:0/853633145 shutdown_connections 2024-04-25T14:10:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.464+0000 7f210b90d640 1 --2- 172.21.15.78:0/853633145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f210410bcd0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.464+0000 7f210b90d640 1 --2- 172.21.15.78:0/853633145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2104106f50 0x7f2104109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.464+0000 7f210b90d640 1 --2- 172.21.15.78:0/853633145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21041045f0 0x7f2104106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.464+0000 7f210b90d640 1 -- 172.21.15.78:0/853633145 >> 172.21.15.78:0/853633145 conn(0x7f21040fdbf0 msgr2=0x7f21041000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 -- 172.21.15.78:0/853633145 shutdown_connections 2024-04-25T14:10:06.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 -- 172.21.15.78:0/853633145 wait complete. 2024-04-25T14:10:06.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 Processor -- start 2024-04-25T14:10:06.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 -- start start 2024-04-25T14:10:06.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21041045f0 0x7f210419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2104106f50 0x7f210419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f21041a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f210410f630 con 0x7f21041098b0 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f210410f4b0 con 0x7f21041045f0 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f210b90d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f210410f7b0 con 0x7f2104106f50 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.465+0000 7f2109eaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f21041a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f21041a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55916/0 (socket says 172.21.15.78:55916) 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 -- 172.21.15.78:0/2116581378 learned_addr learned my addr 172.21.15.78:0/2116581378 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f21096a9640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21041045f0 0x7f210419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2108ea8640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2104106f50 0x7f210419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 -- 172.21.15.78:0/2116581378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2104106f50 msgr2=0x7f210419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2104106f50 0x7f210419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 -- 172.21.15.78:0/2116581378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21041045f0 msgr2=0x7f210419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21041045f0 0x7f210419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 -- 172.21.15.78:0/2116581378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21041a1d40 con 0x7f21041098b0 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f2109eaa640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f21041a16d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0317c0 tx=0x7f20fc0317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f20fa7fc640 1 -- 172.21.15.78:0/2116581378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc003b70 con 0x7f21041098b0 2024-04-25T14:10:06.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f20fa7fc640 1 -- 172.21.15.78:0/2116581378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20fc003d10 con 0x7f21041098b0 2024-04-25T14:10:06.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21041a1fd0 con 0x7f21041098b0 2024-04-25T14:10:06.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f20fa7fc640 1 -- 172.21.15.78:0/2116581378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc02fbd0 con 0x7f21041098b0 2024-04-25T14:10:06.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.466+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041a2430 con 0x7f21041098b0 2024-04-25T14:10:06.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 Processor -- start 2024-04-25T14:10:06.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f20fa7fc640 1 -- 172.21.15.78:0/2116581378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f20fc02fd70 con 0x7f21041098b0 2024-04-25T14:10:06.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 -- start start 2024-04-25T14:10:06.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177487ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177487cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177487dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5617747e1d80 con 0x5617744ba6b0 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56177479b8b0 con 0x5617744db390 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.469+0000 7f21127c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5617747ddd40 con 0x5617744b24d0 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177487dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177487dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55926/0 (socket says 172.21.15.78:55926) 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 -- 172.21.15.78:0/17211899 learned_addr learned my addr 172.21.15.78:0/17211899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177487ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f21096a9640 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177487cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 -- 172.21.15.78:0/17211899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 msgr2=0x56177487cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177487cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f20fa7fc640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20f00afaf0 0x7f20f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 -- 172.21.15.78:0/17211899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 msgr2=0x56177487ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177487ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 -- 172.21.15.78:0/17211899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56177476ca40 con 0x5617744ba6b0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2109eaa640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20f00afaf0 0x7f20f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f2108ea8640 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177487dc70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x56177487dab0 tx=0x7f20f4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee175e76481bb81d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f20fa7fc640 1 -- 172.21.15.78:0/2116581378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f20fc1001b0 con 0x7f21041098b0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f20ef7fe640 1 -- 172.21.15.78:0/17211899 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4004030 con 0x5617744ba6b0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f20ef7fe640 1 -- 172.21.15.78:0/17211899 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f400fc40 con 0x5617744ba6b0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.470+0000 7f20ef7fe640 1 -- 172.21.15.78:0/17211899 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40038e0 con 0x5617744ba6b0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 -- 172.21.15.78:0/17211899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 msgr2=0x56177487dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f2109eaa640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20f00afaf0 0x7f20f00b2000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0bdc20 tx=0x7f20fc0f6dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177487dc70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x56177487dab0 tx=0x7f20f4005850 comp rx=0 tx=0).stop 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 -- 172.21.15.78:0/17211899 shutdown_connections 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177487dc70 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177487cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 --2- 172.21.15.78:0/17211899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177487ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 -- 172.21.15.78:0/17211899 >> 172.21.15.78:0/17211899 conn(0x5617744cf4e0 msgr2=0x561774878270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 -- 172.21.15.78:0/17211899 shutdown_connections 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.471+0000 7f21127c1740 1 -- 172.21.15.78:0/17211899 wait complete. 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 -- start start 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 Processor -- start 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744b24d0 0x561774886930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744ba6b0 0x561774886e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177488ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56177487e780 con 0x5617744b24d0 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561774768d10 con 0x5617744db390 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56177487e900 con 0x5617744ba6b0 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21127c1740 10 client.0 _ll_register_callbacks cb 0x561774795210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:06.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f2108ea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744b24d0 0x561774886930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f2108ea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744b24d0 0x561774886930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55940/0 (socket says 172.21.15.78:55940) 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f2108ea8640 1 -- 172.21.15.78:0/1750870926 learned_addr learned my addr 172.21.15.78:0/1750870926 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177488ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.472+0000 7f21096a9640 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744ba6b0 0x561774886e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f2108ea8640 1 -- 172.21.15.78:0/1750870926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744ba6b0 msgr2=0x561774886e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744ba6b0 0x561774886e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f2108ea8640 1 -- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 msgr2=0x56177488ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177488ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f2108ea8640 1 -- 172.21.15.78:0/1750870926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617745bd7d0 con 0x5617744b24d0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744b24d0 0x561774886930 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f20f400a010 tx=0x7f20f4009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c14592bc7ac902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f20ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f20ecff9640 1 -- 172.21.15.78:0/1750870926 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4029020 con 0x5617744b24d0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f20ecff9640 1 -- 172.21.15.78:0/1750870926 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f20f4019070 con 0x5617744b24d0 2024-04-25T14:10:06.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f400fc40 con 0x5617744b24d0 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f20ecff9640 1 -- 172.21.15.78:0/1750870926 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f20f4034050 con 0x5617744b24d0 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 10 client.18684 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5617744f33f0 con 0x5617744b24d0 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f20ecff9640 1 -- 172.21.15.78:0/1750870926 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f400fc40 con 0x5617744b24d0 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f20ecff9640 1 -- 172.21.15.78:0/1750870926 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f20f4004aa0 con 0x5617744b24d0 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 10 client.18684 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 10 client.18684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 20 client.18684 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561774499f60 0x561774893fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 4 client.18684 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.473+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617747ef990 con 0x561774499f60 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.474+0000 7f21096a9640 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561774499f60 0x561774893fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.474+0000 7f20ecff9640 10 client.18684 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.474+0000 7f21096a9640 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561774499f60 0x561774893fe0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de961864a5e23101 server_cookie=851a6412fad8fa91 in_seq=0 out_seq=0 2024-04-25T14:10:06.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f20ecff9640 1 -- 172.21.15.78:0/1750870926 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617747ef990 con 0x561774499f60 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f20ecff9640 10 client.18684 handle_command_reply: tid=0 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 client.18684 shutdown 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744b24d0 msgr2=0x561774886930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744b24d0 0x561774886930 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f20f400a010 tx=0x7f20f4009d60 comp rx=0 tx=0).stop 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 shutdown_connections 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x56177488ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x561774499f60 0x561774893fe0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744ba6b0 0x561774886e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 --2- 172.21.15.78:0/1750870926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744b24d0 0x561774886930 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f20ecff9640 0 client.18684 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 >> 172.21.15.78:0/1750870926 conn(0x5617744cf4e0 msgr2=0x56177487b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 shutdown_connections 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 1 -- 172.21.15.78:0/1750870926 wait complete. 2024-04-25T14:10:06.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.475+0000 7f21127c1740 20 client.18684 trim_cache size 0 max 16384 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 Processor -- start 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 -- start start 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x5617748855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744ba6b0 0x56177488dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744db390 0x561774881680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21041adb40 con 0x5617744db390 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21041ab410 con 0x5617744ba6b0 2024-04-25T14:10:06.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21127c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21041a9260 con 0x5617744b24d0 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744db390 0x561774881680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744db390 0x561774881680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55956/0 (socket says 172.21.15.78:55956) 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 -- 172.21.15.78:0/1253425370 learned_addr learned my addr 172.21.15.78:0/1253425370 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f21096a9640 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744ba6b0 0x56177488dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x5617748855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 -- 172.21.15.78:0/1253425370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 msgr2=0x5617748855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x5617748855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 -- 172.21.15.78:0/1253425370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744ba6b0 msgr2=0x56177488dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744ba6b0 0x56177488dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.493+0000 7f2108ea8640 1 -- 172.21.15.78:0/1253425370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617744b30f0 con 0x5617744db390 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.494+0000 7f2108ea8640 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744db390 0x561774881680 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f20f40097a0 tx=0x7f20f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300b523bae5cb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.494+0000 7f20eeffd640 1 -- 172.21.15.78:0/1253425370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4029040 con 0x5617744db390 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.494+0000 7f20eeffd640 1 -- 172.21.15.78:0/1253425370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f400f8d0 con 0x5617744db390 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.494+0000 7f20eeffd640 1 -- 172.21.15.78:0/1253425370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4023030 con 0x5617744db390 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.494+0000 7f21127c1740 1 -- 172.21.15.78:0/1253425370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744db390 msgr2=0x561774881680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.494+0000 7f21127c1740 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744db390 0x561774881680 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f20f40097a0 tx=0x7f20f4005f90 comp rx=0 tx=0).stop 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- 172.21.15.78:0/1253425370 shutdown_connections 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744db390 0x561774881680 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744ba6b0 0x56177488dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 --2- 172.21.15.78:0/1253425370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x5617748855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- 172.21.15.78:0/1253425370 >> 172.21.15.78:0/1253425370 conn(0x5617744cf4e0 msgr2=0x561774886500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- 172.21.15.78:0/1253425370 shutdown_connections 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- 172.21.15.78:0/1253425370 wait complete. 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- start start 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 Processor -- start 2024-04-25T14:10:06.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177488ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177488b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x5617748fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21041a98d0 con 0x5617744ba6b0 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21041aad40 con 0x5617744db390 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21041a9fa0 con 0x5617744b24d0 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21127c1740 10 client.0 _ll_register_callbacks cb 0x7f20f4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.495+0000 7f21096a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177488b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177488b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55966/0 (socket says 172.21.15.78:55966) 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 -- 172.21.15.78:0/2699476028 learned_addr learned my addr 172.21.15.78:0/2699476028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f2108ea8640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177488ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f2108ea8640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x5617748fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 -- 172.21.15.78:0/2699476028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 msgr2=0x56177488ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177488ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 -- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 msgr2=0x5617748fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x5617748fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 -- 172.21.15.78:0/2699476028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617747a24b0 con 0x5617744ba6b0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f2108ea8640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x5617748fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21096a9640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177488b260 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f2100005d00 tx=0x7f210001da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef959540cca12fae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f20ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f20ecff9640 1 -- 172.21.15.78:0/2699476028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100011de0 con 0x5617744ba6b0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f20ecff9640 1 -- 172.21.15.78:0/2699476028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2100005020 con 0x5617744ba6b0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20f400f8d0 con 0x5617744ba6b0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f20ecff9640 1 -- 172.21.15.78:0/2699476028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2100005220 con 0x5617744ba6b0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21127c1740 10 client.18690 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5617748fb9d0 con 0x5617744ba6b0 2024-04-25T14:10:06.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.496+0000 7f20ecff9640 1 -- 172.21.15.78:0/2699476028 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f400f8d0 con 0x5617744ba6b0 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f20ecff9640 1 -- 172.21.15.78:0/2699476028 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2100025540 con 0x5617744ba6b0 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f21127c1740 10 client.18690 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f21127c1740 10 client.18690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f21127c1740 20 client.18690 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f21127c1740 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56177446f3a0 0x5617748fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f21127c1740 4 client.18690 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617747e9800 con 0x56177446f3a0 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f2108ea8640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56177446f3a0 0x5617748fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f20ecff9640 10 client.18690 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.497+0000 7f2108ea8640 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56177446f3a0 0x5617748fe3a0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224e7ce9149bfcd7 server_cookie=ef463f951cdb67a4 in_seq=0 out_seq=0 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f20ecff9640 1 -- 172.21.15.78:0/2699476028 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5617747e9800 con 0x56177446f3a0 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f20ecff9640 10 client.18690 handle_command_reply: tid=0 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 client.18690 shutdown 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 msgr2=0x56177488b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177488b260 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f2100005d00 tx=0x7f210001da00 comp rx=0 tx=0).stop 2024-04-25T14:10:06.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 shutdown_connections 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5617744db390 0x5617748fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617744ba6b0 0x56177488b260 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56177446f3a0 0x5617748fe3a0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 --2- 172.21.15.78:0/2699476028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617744b24d0 0x56177488ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f20ecff9640 0 client.18690 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 >> 172.21.15.78:0/2699476028 conn(0x5617744cf4e0 msgr2=0x5617748769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 shutdown_connections 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 1 -- 172.21.15.78:0/2699476028 wait complete. 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f21127c1740 20 client.18690 trim_cache size 0 max 16384 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:10:06.505 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout: "84c4532c-14ee-4185-95c6-778a711c85db": { 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "84c4532c-14ee-4185-95c6-778a711c85db", 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:10:06.506 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20f00afaf0 msgr2=0x7f20f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f210b90d640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20f00afaf0 0x7f20f00b2000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0bdc20 tx=0x7f20fc0f6dd0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 msgr2=0x7f21041a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.498+0000 7f210b90d640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f21041a16d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0317c0 tx=0x7f20fc0317f0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 shutdown_connections 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20f00afaf0 0x7f20f00b2000 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21041098b0 0x7f21041a16d0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2104106f50 0x7f210419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 --2- 172.21.15.78:0/2116581378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21041045f0 0x7f210419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:06.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 >> 172.21.15.78:0/2116581378 conn(0x7f21040fdbf0 msgr2=0x7f2104105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:06.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 shutdown_connections 2024-04-25T14:10:06.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:06.499+0000 7f210b90d640 1 -- 172.21.15.78:0/2116581378 wait complete. 2024-04-25T14:10:06.509 INFO:tasks.cephfs.filesystem:scrub status for tag:84c4532c-14ee-4185-95c6-778a711c85db - {'path': '/', 'tag': '84c4532c-14ee-4185-95c6-778a711c85db', 'options': 'recursive,force'} 2024-04-25T14:10:08.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:08 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:10:08.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:08 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:08.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:08 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:10:08.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:08 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:08.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:08 smithi112 ceph-mon[32617]: pgmap v666: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:08.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:08 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:10:08.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:08 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:08.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:08 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:10:08.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:08 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:08.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:08 smithi129 ceph-mon[32569]: pgmap v666: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:08 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:10:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:08 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:08 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:10:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:08 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:08 smithi078 ceph-mon[29163]: pgmap v666: 97 pgs: 97 active+clean; 3.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:10.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:10 smithi112 ceph-mon[32617]: pgmap v667: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:10:10.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:10 smithi129 ceph-mon[32569]: pgmap v667: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:10:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:10 smithi078 ceph-mon[29163]: pgmap v667: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:10:12.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:12 smithi112 ceph-mon[32617]: pgmap v668: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:12.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:12 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:10:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:12 smithi129 ceph-mon[32569]: pgmap v668: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:12.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:12 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:10:12.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:12 smithi078 ceph-mon[29163]: pgmap v668: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:12 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:10:14.266 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:14 smithi112 ceph-mon[32617]: pgmap v669: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:14.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:14 smithi129 ceph-mon[32569]: pgmap v669: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:14 smithi078 ceph-mon[29163]: pgmap v669: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 -- 172.21.15.78:0/2577467453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71900760a0 msgr2=0x7f71900764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 --2- 172.21.15.78:0/2577467453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71900760a0 0x7f71900764b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7188009ca0 tx=0x7f718802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 -- 172.21.15.78:0/2577467453 shutdown_connections 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 --2- 172.21.15.78:0/2577467453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f719010baa0 0x7f719010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 --2- 172.21.15.78:0/2577467453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71900769f0 0x7f719010b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 --2- 172.21.15.78:0/2577467453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71900760a0 0x7f71900764b0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 -- 172.21.15.78:0/2577467453 >> 172.21.15.78:0/2577467453 conn(0x7f71900fed40 msgr2=0x7f71901011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 -- 172.21.15.78:0/2577467453 shutdown_connections 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 -- 172.21.15.78:0/2577467453 wait complete. 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 Processor -- start 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.103+0000 7f7197bb7640 1 -- start start 2024-04-25T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7197bb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71900760a0 0x7f719019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7197bb7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71900769f0 0x7f719019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7197bb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f719010baa0 0x7f71901a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7197bb7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7190111840 con 0x7f719010baa0 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7197bb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71901116c0 con 0x7f71900760a0 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7197bb7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71901119c0 con 0x7f71900769f0 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f719010baa0 0x7f71901a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f719010baa0 0x7f71901a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41676/0 (socket says 172.21.15.78:41676) 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 -- 172.21.15.78:0/3779712993 learned_addr learned my addr 172.21.15.78:0/3779712993 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7195152640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71900769f0 0x7f719019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7195953640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71900760a0 0x7f719019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 -- 172.21.15.78:0/3779712993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71900769f0 msgr2=0x7f719019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71900769f0 0x7f719019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 -- 172.21.15.78:0/3779712993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71900760a0 msgr2=0x7f719019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71900760a0 0x7f719019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 -- 172.21.15.78:0/3779712993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71901a3ed0 con 0x7f719010baa0 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7196154640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f719010baa0 0x7f71901a3910 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f718c00e970 tx=0x7f718c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.104+0000 7f7186ffd640 1 -- 172.21.15.78:0/3779712993 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c00ccc0 con 0x7f719010baa0 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.105+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71901a41c0 con 0x7f719010baa0 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.105+0000 7f7186ffd640 1 -- 172.21.15.78:0/3779712993 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718c0054d0 con 0x7f719010baa0 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.105+0000 7f7186ffd640 1 -- 172.21.15.78:0/3779712993 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c010430 con 0x7f719010baa0 2024-04-25T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.105+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71901a46c0 con 0x7f719010baa0 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.107+0000 7f7186ffd640 1 -- 172.21.15.78:0/3779712993 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f718c005020 con 0x7f719010baa0 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.107+0000 7f719ea6e740 1 Processor -- start 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.107+0000 7f719ea6e740 1 -- start start 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626facb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626facbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f70a4d0 0x55626fad2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f719ea6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55626fa31460 con 0x55626f70a4d0 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f719ea6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55626f9eb590 con 0x55626f7022f0 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f719ea6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55626fa2d420 con 0x55626f72b1b0 2024-04-25T14:10:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f70a4d0 0x55626fad2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f70a4d0 0x55626fad2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41680/0 (socket says 172.21.15.78:41680) 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 -- 172.21.15.78:0/1662533493 learned_addr learned my addr 172.21.15.78:0/1662533493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7186ffd640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f717c0afae0 0x7f717c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195953640 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626facbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626facb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7196154640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f717c0afae0 0x7f717c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 -- 172.21.15.78:0/1662533493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 msgr2=0x55626facb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626facb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 -- 172.21.15.78:0/1662533493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 msgr2=0x55626facbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626facbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 -- 172.21.15.78:0/1662533493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626f9f80d0 con 0x55626f70a4d0 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195953640 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626facbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7186ffd640 1 -- 172.21.15.78:0/3779712993 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f718c0df110 con 0x7f719010baa0 2024-04-25T14:10:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.108+0000 7f7195152640 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f70a4d0 0x55626fad2df0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f71800124c0 tx=0x7f718000b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b304c2d22f148b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.109+0000 7f717bfff640 1 -- 172.21.15.78:0/1662533493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180018030 con 0x55626f70a4d0 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.109+0000 7f717bfff640 1 -- 172.21.15.78:0/1662533493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180010b50 con 0x55626f70a4d0 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.109+0000 7f717bfff640 1 -- 172.21.15.78:0/1662533493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718000eb00 con 0x55626f70a4d0 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 -- 172.21.15.78:0/1662533493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f70a4d0 msgr2=0x55626fad2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f7196154640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f717c0afae0 0x7f717c0b1ff0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f718c0a3660 tx=0x7f718c0d5d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f70a4d0 0x55626fad2df0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f71800124c0 tx=0x7f718000b640 comp rx=0 tx=0).stop 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 -- 172.21.15.78:0/1662533493 shutdown_connections 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f70a4d0 0x55626fad2df0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626facbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 --2- 172.21.15.78:0/1662533493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626facb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 -- 172.21.15.78:0/1662533493 >> 172.21.15.78:0/1662533493 conn(0x55626f71f300 msgr2=0x55626fac7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 -- 172.21.15.78:0/1662533493 shutdown_connections 2024-04-25T14:10:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 -- 172.21.15.78:0/1662533493 wait complete. 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 -- start start 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.110+0000 7f719ea6e740 1 Processor -- start 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fad5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fad5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fadcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55626fad3840 con 0x55626f72b1b0 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55626f9b8b60 con 0x55626f7022f0 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55626fad39c0 con 0x55626f70a4d0 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 10 client.0 _ll_register_callbacks cb 0x55626f9dd2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fadcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fadcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41694/0 (socket says 172.21.15.78:41694) 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 -- 172.21.15.78:0/2275955805 learned_addr learned my addr 172.21.15.78:0/2275955805 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195953640 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fad5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fad5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 -- 172.21.15.78:0/2275955805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 msgr2=0x55626fad5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fad5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 -- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 msgr2=0x55626fad5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fad5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 -- 172.21.15.78:0/2275955805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626f743210 con 0x55626f72b1b0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f7195152640 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fadcab0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f718000e5e0 tx=0x7f718000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c1980798c52715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f71797fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f71797fa640 1 -- 172.21.15.78:0/2275955805 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180018070 con 0x55626f72b1b0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f71797fa640 1 -- 172.21.15.78:0/2275955805 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f71800108b0 con 0x55626f72b1b0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.111+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7180010b50 con 0x55626f72b1b0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f71797fa640 1 -- 172.21.15.78:0/2275955805 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7180005020 con 0x55626f72b1b0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 10 client.18702 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55626f80f620 con 0x55626f72b1b0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f71797fa640 1 -- 172.21.15.78:0/2275955805 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180010b50 con 0x55626f72b1b0 2024-04-25T14:10:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f71797fa640 1 -- 172.21.15.78:0/2275955805 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f71800047e0 con 0x55626f72b1b0 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 10 client.18702 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 10 client.18702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 20 client.18702 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6e9d80 0x55626fae30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 4 client.18702 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55626fa3f070 con 0x55626f6e9d80 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.112+0000 7f7195953640 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6e9d80 0x55626fae30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.113+0000 7f71797fa640 10 client.18702 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.113+0000 7f7195953640 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6e9d80 0x55626fae30c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288f2d00fd6b3231 server_cookie=a1854131e0e3dfad in_seq=0 out_seq=0 2024-04-25T14:10:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f71797fa640 1 -- 172.21.15.78:0/2275955805 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55626fa3f070 con 0x55626f6e9d80 2024-04-25T14:10:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f71797fa640 10 client.18702 handle_command_reply: tid=0 2024-04-25T14:10:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 client.18702 shutdown 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 msgr2=0x55626fadcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fadcab0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f718000e5e0 tx=0x7f718000bec0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 shutdown_connections 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fadcab0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6e9d80 0x55626fae30c0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fad5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 --2- 172.21.15.78:0/2275955805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fad5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f71797fa640 0 client.18702 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 >> 172.21.15.78:0/2275955805 conn(0x55626f71f300 msgr2=0x55626faca210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 shutdown_connections 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 1 -- 172.21.15.78:0/2275955805 wait complete. 2024-04-25T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.114+0000 7f719ea6e740 20 client.18702 trim_cache size 0 max 16384 2024-04-25T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 Processor -- start 2024-04-25T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 -- start start 2024-04-25T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fac6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fadc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fac7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71901afd30 con 0x55626f72b1b0 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71901ad600 con 0x55626f7022f0 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.129+0000 7f719ea6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71901ab450 con 0x55626f70a4d0 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fac7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fac7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41704/0 (socket says 172.21.15.78:41704) 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 -- 172.21.15.78:0/3876313820 learned_addr learned my addr 172.21.15.78:0/3876313820 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195953640 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fadc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fac6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 -- 172.21.15.78:0/3876313820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 msgr2=0x55626fadc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fadc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 -- 172.21.15.78:0/3876313820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 msgr2=0x55626fac6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fac6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 -- 172.21.15.78:0/3876313820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55626fad0500 con 0x55626f72b1b0 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195953640 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fadc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f7195152640 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fac7270 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f718000b680 tx=0x7f7180005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf98836f0d8e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f717b7fe640 1 -- 172.21.15.78:0/3876313820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180018040 con 0x55626f72b1b0 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f717b7fe640 1 -- 172.21.15.78:0/3876313820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718000ee60 con 0x55626f72b1b0 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.130+0000 7f717b7fe640 1 -- 172.21.15.78:0/3876313820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180010b10 con 0x55626f72b1b0 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 -- 172.21.15.78:0/3876313820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 msgr2=0x55626fac7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fac7270 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f718000b680 tx=0x7f7180005b10 comp rx=0 tx=0).stop 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 -- 172.21.15.78:0/3876313820 shutdown_connections 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f72b1b0 0x55626fac7270 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f70a4d0 0x55626fadc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 --2- 172.21.15.78:0/3876313820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f7022f0 0x55626fac6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 -- 172.21.15.78:0/3876313820 >> 172.21.15.78:0/3876313820 conn(0x55626f71f300 msgr2=0x55626facb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 -- 172.21.15.78:0/3876313820 shutdown_connections 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 -- 172.21.15.78:0/3876313820 wait complete. 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 -- start start 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.131+0000 7f719ea6e740 1 Processor -- start 2024-04-25T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f7022f0 0x55626fad9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f70a4d0 0x55626fada440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f719ea6e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626fadb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f719ea6e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71901abac0 con 0x55626f7022f0 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f719ea6e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f71901acf30 con 0x55626f70a4d0 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f719ea6e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71901ac190 con 0x55626f72b1b0 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f719ea6e740 10 client.0 _ll_register_callbacks cb 0x7f718000bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f7022f0 0x55626fad9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f7022f0 0x55626fad9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41706/0 (socket says 172.21.15.78:41706) 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 -- 172.21.15.78:0/4010938424 learned_addr learned my addr 172.21.15.78:0/4010938424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195953640 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f70a4d0 0x55626fada440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626fadb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 -- 172.21.15.78:0/4010938424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 msgr2=0x55626fadb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626fadb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 -- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f70a4d0 msgr2=0x55626fada440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f70a4d0 0x55626fada440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 -- 172.21.15.78:0/4010938424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55626fadcf10 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f7195152640 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f7022f0 0x55626fad9f00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f718000ee30 tx=0x7f718000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4fe87f9f152f3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.132+0000 7f71797fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f71797fa640 1 -- 172.21.15.78:0/4010938424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180018040 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f71797fa640 1 -- 172.21.15.78:0/4010938424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f718002d090 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718000ee60 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f71797fa640 1 -- 172.21.15.78:0/4010938424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7180046050 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 10 client.18708 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55626fadc9f0 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f71797fa640 1 -- 172.21.15.78:0/4010938424 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718000ee60 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f71797fa640 1 -- 172.21.15.78:0/4010938424 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f718002d290 con 0x55626f7022f0 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 10 client.18708 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 10 client.18708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 20 client.18708 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6bf1c0 0x55626fb56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 4 client.18708 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55626fa38ee0 con 0x55626f6bf1c0 2024-04-25T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.133+0000 7f7195953640 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6bf1c0 0x55626fb56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.134+0000 7f71797fa640 10 client.18708 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.134+0000 7f7195953640 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6bf1c0 0x55626fb56080 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eae13e672832270 server_cookie=d90e6fdba16d58bb in_seq=0 out_seq=0 2024-04-25T14:10:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f71797fa640 1 -- 172.21.15.78:0/4010938424 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55626fa38ee0 con 0x55626f6bf1c0 2024-04-25T14:10:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f71797fa640 10 client.18708 handle_command_reply: tid=0 2024-04-25T14:10:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 client.18708 shutdown 2024-04-25T14:10:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f7022f0 msgr2=0x55626fad9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f7022f0 0x55626fad9f00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f718000ee30 tx=0x7f718000b730 comp rx=0 tx=0).stop 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 shutdown_connections 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55626f6bf1c0 0x55626fb56080 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55626f72b1b0 0x55626fadb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55626f70a4d0 0x55626fada440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 --2- 172.21.15.78:0/4010938424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55626f7022f0 0x55626fad9f00 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f71797fa640 0 client.18708 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 >> 172.21.15.78:0/4010938424 conn(0x55626f71f300 msgr2=0x55626faca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 shutdown_connections 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 1 -- 172.21.15.78:0/4010938424 wait complete. 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.148+0000 7f719ea6e740 20 client.18708 trim_cache size 0 max 16384 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:10:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:10:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:10:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:10:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f717c0afae0 msgr2=0x7f717c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f717c0afae0 0x7f717c0b1ff0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f718c0a3660 tx=0x7f718c0d5d30 comp rx=0 tx=0).stop 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f719010baa0 msgr2=0x7f71901a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f719010baa0 0x7f71901a3910 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f718c00e970 tx=0x7f718c00ee30 comp rx=0 tx=0).stop 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 shutdown_connections 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f717c0afae0 0x7f717c0b1ff0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f719010baa0 0x7f71901a3910 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71900769f0 0x7f719019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 --2- 172.21.15.78:0/3779712993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71900760a0 0x7f719019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 >> 172.21.15.78:0/3779712993 conn(0x7f71900fed40 msgr2=0x7f7190107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 shutdown_connections 2024-04-25T14:10:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:15.149+0000 7f7197bb7640 1 -- 172.21.15.78:0/3779712993 wait complete. 2024-04-25T14:10:16.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:16 smithi129 ceph-mon[32569]: pgmap v670: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:10:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:16 smithi078 ceph-mon[29163]: pgmap v670: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:10:16.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:16 smithi112 ceph-mon[32617]: pgmap v670: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-25T14:10:18.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:18 smithi129 ceph-mon[32569]: pgmap v671: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:18.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:18 smithi078 ceph-mon[29163]: pgmap v671: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:18.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:18 smithi112 ceph-mon[32617]: pgmap v671: 97 pgs: 97 active+clean; 3.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:20.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:20 smithi129 ceph-mon[32569]: pgmap v672: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:10:20.528 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:20 smithi112 ceph-mon[32617]: pgmap v672: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:10:20.528 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:20 smithi078 ceph-mon[29163]: pgmap v672: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:10:22.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:22 smithi129 ceph-mon[32569]: pgmap v673: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:22 smithi078 ceph-mon[29163]: pgmap v673: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:22.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:22 smithi112 ceph-mon[32617]: pgmap v673: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:24.318 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:24 smithi112 ceph-mon[32617]: pgmap v674: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:24.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:24 smithi129 ceph-mon[32569]: pgmap v674: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:24 smithi078 ceph-mon[29163]: pgmap v674: 97 pgs: 97 active+clean; 3.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:26.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:26 smithi129 ceph-mon[32569]: pgmap v675: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:26 smithi078 ceph-mon[29163]: pgmap v675: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:26.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:26 smithi112 ceph-mon[32617]: pgmap v675: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 -- 172.21.15.78:0/686042414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 msgr2=0x7f2058104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 --2- 172.21.15.78:0/686042414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f2058104840 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f2050009ca0 tx=0x7f205002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 -- 172.21.15.78:0/686042414 shutdown_connections 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 --2- 172.21.15.78:0/686042414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2058109a80 0x7f205810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 --2- 172.21.15.78:0/686042414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2058104d80 0x7f2058105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 --2- 172.21.15.78:0/686042414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f2058104840 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 -- 172.21.15.78:0/686042414 >> 172.21.15.78:0/686042414 conn(0x7f20580fed30 msgr2=0x7f20581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 -- 172.21.15.78:0/686042414 shutdown_connections 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.254+0000 7f205f306640 1 -- 172.21.15.78:0/686042414 wait complete. 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 Processor -- start 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 -- start start 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f205819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2058104d80 0x7f205819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2058109a80 0x7f20581a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f205810f640 con 0x7f2058103b40 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f205810f4c0 con 0x7f2058104d80 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205f306640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f205810f7c0 con 0x7f2058109a80 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205d0a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f205819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205d0a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f205819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56242/0 (socket says 172.21.15.78:56242) 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205d0a2640 1 -- 172.21.15.78:0/4253501802 learned_addr learned my addr 172.21.15.78:0/4253501802 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205c8a1640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2058104d80 0x7f205819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.255+0000 7f205d8a3640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2058109a80 0x7f20581a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205d0a2640 1 -- 172.21.15.78:0/4253501802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2058109a80 msgr2=0x7f20581a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205d0a2640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2058109a80 0x7f20581a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205d0a2640 1 -- 172.21.15.78:0/4253501802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2058104d80 msgr2=0x7f205819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205d0a2640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2058104d80 0x7f205819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205d0a2640 1 -- 172.21.15.78:0/4253501802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20581a1db0 con 0x7f2058103b40 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205d0a2640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f205819a270 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f205002ade0 tx=0x7f20500316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f20567fc640 1 -- 172.21.15.78:0/4253501802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050003b70 con 0x7f2058103b40 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f20567fc640 1 -- 172.21.15.78:0/4253501802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2050003d10 con 0x7f2058103b40 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20581a2040 con 0x7f2058103b40 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f20567fc640 1 -- 172.21.15.78:0/4253501802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205002fb90 con 0x7f2058103b40 2024-04-25T14:10:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.256+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20581a2590 con 0x7f2058103b40 2024-04-25T14:10:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.258+0000 7f20567fc640 1 -- 172.21.15.78:0/4253501802 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f205002fd30 con 0x7f2058103b40 2024-04-25T14:10:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 Processor -- start 2024-04-25T14:10:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 -- start start 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad3799970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad3799ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ad36ff460 con 0x559ad33d02f0 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559ad36b9590 con 0x559ad33f91b0 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20661b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ad36fb420 con 0x559ad33d84d0 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f20567fc640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20400afae0 0x7f20400b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f205d0a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad3799ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f205d0a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad3799ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56248/0 (socket says 172.21.15.78:56248) 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f205d0a2640 1 -- 172.21.15.78:0/3235013569 learned_addr learned my addr 172.21.15.78:0/3235013569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f205d8a3640 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad3799970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f205c8a1640 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.259+0000 7f205d8a3640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20400afae0 0x7f20400b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f20567fc640 1 -- 172.21.15.78:0/4253501802 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2050100170 con 0x7f2058103b40 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f205d0a2640 1 -- 172.21.15.78:0/3235013569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 msgr2=0x559ad37a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f205d0a2640 1 -- 172.21.15.78:0/3235013569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 msgr2=0x559ad3799970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad3799970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f205d0a2640 1 -- 172.21.15.78:0/3235013569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad36c60d0 con 0x559ad33d02f0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad3799ef0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f2050045000 tx=0x7f205002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7225a7a5c3ae2551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f205d8a3640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20400afae0 0x7f20400b1ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x559ad379ab40 tx=0x7f2048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f203f7fe640 1 -- 172.21.15.78:0/3235013569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20500c7f90 con 0x559ad33d02f0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f203f7fe640 1 -- 172.21.15.78:0/3235013569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2050036070 con 0x559ad33d02f0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.260+0000 7f203f7fe640 1 -- 172.21.15.78:0/3235013569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20500c7db0 con 0x559ad33d02f0 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 -- 172.21.15.78:0/3235013569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 msgr2=0x559ad3799ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad3799ef0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f2050045000 tx=0x7f205002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 -- 172.21.15.78:0/3235013569 shutdown_connections 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad3799ef0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 --2- 172.21.15.78:0/3235013569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad3799970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 -- 172.21.15.78:0/3235013569 >> 172.21.15.78:0/3235013569 conn(0x559ad33ed300 msgr2=0x559ad3795300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 -- 172.21.15.78:0/3235013569 shutdown_connections 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 -- 172.21.15.78:0/3235013569 wait complete. 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 -- start start 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.261+0000 7f20661b9740 1 Processor -- start 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f20661b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d02f0 0x559ad37a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f20661b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d84d0 0x559ad37a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f20661b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f20661b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ad36ff460 con 0x559ad33d84d0 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f20661b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559ad3686b60 con 0x559ad33f91b0 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f20661b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ad37a1600 con 0x559ad33d02f0 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f20661b9740 10 client.0 _ll_register_callbacks cb 0x559ad36ab2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d84d0 0x559ad37a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d84d0 0x559ad37a3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56264/0 (socket says 172.21.15.78:56264) 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 -- 172.21.15.78:0/3812791769 learned_addr learned my addr 172.21.15.78:0/3812791769 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d02f0 0x559ad37a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205d8a3640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 -- 172.21.15.78:0/3812791769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d02f0 msgr2=0x559ad37a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d02f0 0x559ad37a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 -- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 msgr2=0x559ad37aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 -- 172.21.15.78:0/3812791769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad3411210 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205d8a3640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f205c8a1640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d84d0 0x559ad37a3750 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f204c0126d0 tx=0x7f204c012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaca97522a29c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.262+0000 7f203cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f203cff9640 1 -- 172.21.15.78:0/3812791769 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c010e80 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2050036070 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f203cff9640 1 -- 172.21.15.78:0/3812791769 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f204c004020 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f203cff9640 1 -- 172.21.15.78:0/3812791769 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f204c004220 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 10 client.18720 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559ad34dd620 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f203cff9640 1 -- 172.21.15.78:0/3812791769 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2050036070 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f203cff9640 1 -- 172.21.15.78:0/3812791769 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f204c01a4c0 con 0x559ad33d84d0 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 10 client.18720 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 10 client.18720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 20 client.18720 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad33b7d80 0x559ad37b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 4 client.18720 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.263+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ad370d070 con 0x559ad33b7d80 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.264+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad33b7d80 0x559ad37b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.264+0000 7f203cff9640 10 client.18720 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.264+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad33b7d80 0x559ad37b1460 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67952945457ab62 server_cookie=6e414d5ec979b3cd in_seq=0 out_seq=0 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f203cff9640 1 -- 172.21.15.78:0/3812791769 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ad370d070 con 0x559ad33b7d80 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f203cff9640 10 client.18720 handle_command_reply: tid=0 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 client.18720 shutdown 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d84d0 msgr2=0x559ad37a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d84d0 0x559ad37a3750 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f204c0126d0 tx=0x7f204c012b90 comp rx=0 tx=0).stop 2024-04-25T14:10:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 shutdown_connections 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d84d0 0x559ad37a3750 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad33b7d80 0x559ad37b1460 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 --2- 172.21.15.78:0/3812791769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d02f0 0x559ad37a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 >> 172.21.15.78:0/3812791769 conn(0x559ad33ed300 msgr2=0x559ad3798210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f203cff9640 0 client.18720 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 shutdown_connections 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 1 -- 172.21.15.78:0/3812791769 wait complete. 2024-04-25T14:10:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.265+0000 7f20661b9740 20 client.18720 trim_cache size 0 max 16384 2024-04-25T14:10:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 Processor -- start 2024-04-25T14:10:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 -- start start 2024-04-25T14:10:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad379dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad379dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20581adc00 con 0x559ad33d02f0 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20581ab4d0 con 0x559ad33f91b0 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f20661b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f20581a9320 con 0x559ad33d84d0 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad379dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad379dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56278/0 (socket says 172.21.15.78:56278) 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 -- 172.21.15.78:0/861966099 learned_addr learned my addr 172.21.15.78:0/861966099 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d8a3640 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205c8a1640 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad379dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 -- 172.21.15.78:0/861966099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 msgr2=0x559ad379dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad379dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 -- 172.21.15.78:0/861966099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 msgr2=0x559ad37aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 -- 172.21.15.78:0/861966099 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad37aab10 con 0x559ad33d02f0 2024-04-25T14:10:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f205d0a2640 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad379dab0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f20500049d0 tx=0x7f20500c4630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb369549ac55dda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f203effd640 1 -- 172.21.15.78:0/861966099 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20500c8d40 con 0x559ad33d02f0 2024-04-25T14:10:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f203effd640 1 -- 172.21.15.78:0/861966099 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2050036070 con 0x559ad33d02f0 2024-04-25T14:10:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.281+0000 7f203effd640 1 -- 172.21.15.78:0/861966099 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20500d14a0 con 0x559ad33d02f0 2024-04-25T14:10:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 -- 172.21.15.78:0/861966099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 msgr2=0x559ad379dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad379dab0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f20500049d0 tx=0x7f20500c4630 comp rx=0 tx=0).stop 2024-04-25T14:10:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 -- 172.21.15.78:0/861966099 shutdown_connections 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33f91b0 0x559ad37aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad379dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 --2- 172.21.15.78:0/861966099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33d02f0 0x559ad379dab0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 -- 172.21.15.78:0/861966099 >> 172.21.15.78:0/861966099 conn(0x559ad33ed300 msgr2=0x559ad37998f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 -- 172.21.15.78:0/861966099 shutdown_connections 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.282+0000 7f20661b9740 1 -- 172.21.15.78:0/861966099 wait complete. 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 -- start start 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 Processor -- start 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33d02f0 0x559ad37aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33f91b0 0x559ad37a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20581a9990 con 0x559ad33f91b0 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20581aae00 con 0x559ad33d02f0 2024-04-25T14:10:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f20581aa060 con 0x559ad33d84d0 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f20661b9740 10 client.0 _ll_register_callbacks cb 0x7f204c00a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f205d8a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33f91b0 0x559ad37a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f205d8a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33f91b0 0x559ad37a74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56284/0 (socket says 172.21.15.78:56284) 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f205d8a3640 1 -- 172.21.15.78:0/3579540964 learned_addr learned my addr 172.21.15.78:0/3579540964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f205c8a1640 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.283+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33d02f0 0x559ad37aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f205d8a3640 1 -- 172.21.15.78:0/3579540964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 msgr2=0x559ad37a6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f205d8a3640 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f205d8a3640 1 -- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33d02f0 msgr2=0x559ad37aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f205d8a3640 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33d02f0 0x559ad37aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f205d8a3640 1 -- 172.21.15.78:0/3579540964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad3817800 con 0x559ad33f91b0 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f205d8a3640 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33f91b0 0x559ad37a74f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f20480141b0 tx=0x7f20480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d925bdd91529f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f203cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f203cff9640 1 -- 172.21.15.78:0/3579540964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048027070 con 0x559ad33f91b0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f203cff9640 1 -- 172.21.15.78:0/3579540964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f20480047a0 con 0x559ad33f91b0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2050036070 con 0x559ad33f91b0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f203cff9640 1 -- 172.21.15.78:0/3579540964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f20480049a0 con 0x559ad33f91b0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f20661b9740 10 client.18726 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559ad3818a40 con 0x559ad33f91b0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f203cff9640 1 -- 172.21.15.78:0/3579540964 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2050036070 con 0x559ad33f91b0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f203cff9640 1 -- 172.21.15.78:0/3579540964 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2048020040 con 0x559ad33f91b0 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f20661b9740 10 client.18726 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.284+0000 7f20661b9740 10 client.18726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.285+0000 7f20661b9740 20 client.18726 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.285+0000 7f20661b9740 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad338d1c0 0x559ad381b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.285+0000 7f20661b9740 4 client.18726 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:10:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.285+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ad3706ee0 con 0x559ad338d1c0 2024-04-25T14:10:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.285+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad338d1c0 0x559ad381b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.285+0000 7f203cff9640 10 client.18726 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.285+0000 7f205d0a2640 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad338d1c0 0x559ad381b460 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c424d6b58ca2b47 server_cookie=c959c9e826497416 in_seq=0 out_seq=0 2024-04-25T14:10:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f203cff9640 1 -- 172.21.15.78:0/3579540964 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ad3706ee0 con 0x559ad338d1c0 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f203cff9640 10 client.18726 handle_command_reply: tid=0 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 client.18726 shutdown 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33f91b0 msgr2=0x559ad37a74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33f91b0 0x559ad37a74f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f20480141b0 tx=0x7f20480046c0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 shutdown_connections 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ad33f91b0 0x559ad37a74f0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559ad338d1c0 0x559ad381b460 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ad33d84d0 0x559ad37a6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 --2- 172.21.15.78:0/3579540964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559ad33d02f0 0x559ad37aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f203cff9640 0 client.18726 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 >> 172.21.15.78:0/3579540964 conn(0x559ad33ed300 msgr2=0x559ad379dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 shutdown_connections 2024-04-25T14:10:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 1 -- 172.21.15.78:0/3579540964 wait complete. 2024-04-25T14:10:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f20661b9740 20 client.18726 trim_cache size 0 max 16384 2024-04-25T14:10:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:10:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:10:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:10:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:10:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20400afae0 msgr2=0x7f20400b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20400afae0 0x7f20400b1ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x559ad379ab40 tx=0x7f2048005f90 comp rx=0 tx=0).stop 2024-04-25T14:10:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 msgr2=0x7f205819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f205819a270 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f205002ade0 tx=0x7f20500316e0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 shutdown_connections 2024-04-25T14:10:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2058109a80 0x7f20581a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2058104d80 0x7f205819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20400afae0 0x7f20400b1ff0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 --2- 172.21.15.78:0/4253501802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2058103b40 0x7f205819a270 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 >> 172.21.15.78:0/4253501802 conn(0x7f20580fed30 msgr2=0x7f2058105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 shutdown_connections 2024-04-25T14:10:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:28.301+0000 7f205f306640 1 -- 172.21.15.78:0/4253501802 wait complete. 2024-04-25T14:10:28.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:28 smithi129 ceph-mon[32569]: pgmap v676: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:28.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:28 smithi078 ceph-mon[29163]: pgmap v676: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:28.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:28 smithi112 ceph-mon[32617]: pgmap v676: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:30.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:30 smithi129 ceph-mon[32569]: pgmap v677: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:10:30.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:30 smithi078 ceph-mon[29163]: pgmap v677: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:10:30.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:30 smithi112 ceph-mon[32617]: pgmap v677: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-25T14:10:32.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:32 smithi129 ceph-mon[32569]: pgmap v678: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:10:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:32 smithi078 ceph-mon[29163]: pgmap v678: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:10:32.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:32 smithi112 ceph-mon[32617]: pgmap v678: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:10:34.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:34 smithi129 ceph-mon[32569]: pgmap v679: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:10:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:34 smithi078 ceph-mon[29163]: pgmap v679: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:10:34.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:34 smithi112 ceph-mon[32617]: pgmap v679: 97 pgs: 97 active+clean; 3.8 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T14:10:36.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:36 smithi129 ceph-mon[32569]: pgmap v680: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:36 smithi078 ceph-mon[29163]: pgmap v680: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:36.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:10:36.510 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:10:36.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:36 smithi112 ceph-mon[32617]: pgmap v680: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:36.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 -- 172.21.15.78:0/1031894420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 msgr2=0x7f5ff0105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/1031894420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff0105150 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009900 tx=0x7f5fe402a410 comp rx=0 tx=0).stop 2024-04-25T14:10:36.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 -- 172.21.15.78:0/1031894420 shutdown_connections 2024-04-25T14:10:36.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/1031894420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff0109a40 0x7f5ff010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/1031894420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff0105150 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/1031894420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff0103b00 0x7f5ff0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 -- 172.21.15.78:0/1031894420 >> 172.21.15.78:0/1031894420 conn(0x7f5ff00fed10 msgr2=0x7f5ff01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 -- 172.21.15.78:0/1031894420 shutdown_connections 2024-04-25T14:10:36.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.635+0000 7f5ff75f3640 1 -- 172.21.15.78:0/1031894420 wait complete. 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 Processor -- start 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 -- start start 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff0103b00 0x7f5ff019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff0109a40 0x7f5ff01a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff010f630 con 0x7f5ff0104d40 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff010f4b0 con 0x7f5ff0103b00 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff75f3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff010f7b0 con 0x7f5ff0109a40 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff4b8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff4b8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42054/0 (socket says 172.21.15.78:42054) 2024-04-25T14:10:36.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/775836679 learned_addr learned my addr 172.21.15.78:0/775836679 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff0109a40 0x7f5ff01a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.636+0000 7f5ff538f640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff0103b00 0x7f5ff019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/775836679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff0109a40 msgr2=0x7f5ff01a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff0109a40 0x7f5ff01a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/775836679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff0103b00 msgr2=0x7f5ff019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff0103b00 0x7f5ff019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/775836679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff01a1de0 con 0x7f5ff0104d40 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff0109a40 0x7f5ff01a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff538f640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff0103b00 0x7f5ff019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff019a7e0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4006010 tx=0x7f5fe4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5fe27fc640 1 -- 172.21.15.78:0/775836679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe403f030 con 0x7f5ff0104d40 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5fe27fc640 1 -- 172.21.15.78:0/775836679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe4003cb0 con 0x7f5ff0104d40 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff01a2070 con 0x7f5ff0104d40 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5fe27fc640 1 -- 172.21.15.78:0/775836679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4035070 con 0x7f5ff0104d40 2024-04-25T14:10:36.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.637+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff01a2560 con 0x7f5ff0104d40 2024-04-25T14:10:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 Processor -- start 2024-04-25T14:10:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5fe27fc640 1 -- 172.21.15.78:0/775836679 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5fe4042030 con 0x7f5ff0104d40 2024-04-25T14:10:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 -- start start 2024-04-25T14:10:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87fe9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87fe9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87feac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d87f4ed80 con 0x559d87c48390 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d87f088b0 con 0x559d87c276b0 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d87f4ad40 con 0x559d87c1f4d0 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ff5b90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87fe9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ff538f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87fe9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ff5b90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87fe9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42066/0 (socket says 172.21.15.78:42066) 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2581158254 learned_addr learned my addr 172.21.15.78:0/2581158254 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87feac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.640+0000 7f5fe27fc640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2581158254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 msgr2=0x559d87fe9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87fe9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2581158254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 msgr2=0x559d87feac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87feac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2581158254 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d87ed9a40 con 0x559d87c48390 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff538f640 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87fe9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:10:36.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5fe27fc640 1 -- 172.21.15.78:0/775836679 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5fe40ff2f0 con 0x7f5ff0104d40 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87fe9a60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8008f50 tx=0x7f5fe800d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6efe74024ca414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5fdb7fe640 1 -- 172.21.15.78:0/2581158254 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe801a8e0 con 0x559d87c48390 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5fdb7fe640 1 -- 172.21.15.78:0/2581158254 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe8019410 con 0x559d87c48390 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5fe801e000 tx=0x7f5fe801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.641+0000 7f5fdb7fe640 1 -- 172.21.15.78:0/2581158254 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800e610 con 0x559d87c48390 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2581158254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 msgr2=0x559d87fe9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87fe9a60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8008f50 tx=0x7f5fe800d7d0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2581158254 shutdown_connections 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87feac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87fe9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2581158254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87fe9a60 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2581158254 >> 172.21.15.78:0/2581158254 conn(0x559d87c3c4e0 msgr2=0x559d87fe5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2581158254 shutdown_connections 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2581158254 wait complete. 2024-04-25T14:10:36.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 -- start start 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.642+0000 7f5ffe57d740 1 Processor -- start 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d87ff3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d87ffaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d87feb780 con 0x559d87c276b0 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559d87ed5d10 con 0x559d87c48390 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d87feb900 con 0x559d87c1f4d0 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ffe57d740 10 client.0 _ll_register_callbacks cb 0x559d87f02210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d87ff3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d87ff3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42080/0 (socket says 172.21.15.78:42080) 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/4247372480 learned_addr learned my addr 172.21.15.78:0/4247372480 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff538f640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d87ffaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/4247372480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 msgr2=0x559d87ff3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 msgr2=0x559d87ffaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d87ffaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/4247372480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d87d2a7d0 con 0x559d87c276b0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d87ffaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff538f640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d87ff3f40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40fcde0 tx=0x7f5fe4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7f745d2b65e693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.643+0000 7f5fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/4247372480 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4049070 con 0x559d87c276b0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/4247372480 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5fe40b00d0 con 0x559d87c276b0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe8019410 con 0x559d87c276b0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/4247372480 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5fe4003e60 con 0x559d87c276b0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 10 client.18738 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559d87c603f0 con 0x559d87c276b0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/4247372480 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe8019410 con 0x559d87c276b0 2024-04-25T14:10:36.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/4247372480 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5fe40c7a20 con 0x559d87c276b0 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 10 client.18738 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 20 client.18738 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87c06f60 0x559d88001310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 4 client.18738 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d87f5c990 con 0x559d87c06f60 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.644+0000 7f5ff538f640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87c06f60 0x559d88001310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.645+0000 7f5fd8ff9640 10 client.18738 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.645+0000 7f5ff538f640 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87c06f60 0x559d88001310 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cdc020beca727e9 server_cookie=9156ab97dcdec290 in_seq=0 out_seq=0 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/4247372480 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d87f5c990 con 0x559d87c06f60 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5fd8ff9640 10 client.18738 handle_command_reply: tid=0 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 client.18738 shutdown 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 msgr2=0x559d87ff3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d87ff3f40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40fcde0 tx=0x7f5fe4009560 comp rx=0 tx=0).stop 2024-04-25T14:10:36.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 shutdown_connections 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d87ffaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d87ff3f40 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87c06f60 0x559d88001310 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/4247372480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5fd8ff9640 0 client.18738 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 >> 172.21.15.78:0/4247372480 conn(0x559d87c3c4e0 msgr2=0x559d87fe43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 shutdown_connections 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 1 -- 172.21.15.78:0/4247372480 wait complete. 2024-04-25T14:10:36.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.646+0000 7f5ffe57d740 20 client.18738 trim_cache size 0 max 16384 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 Processor -- start 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 -- start start 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d88005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d88006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d88006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01adc10 con 0x559d87c276b0 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01ab4e0 con 0x559d87c48390 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01a9330 con 0x559d87c1f4d0 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d88006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d88006040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42094/0 (socket says 172.21.15.78:42094) 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/3933700961 learned_addr learned my addr 172.21.15.78:0/3933700961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff538f640 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d88005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/3933700961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 msgr2=0x559d88005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d88005b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d88006580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/3933700961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 msgr2=0x559d88006580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d88006580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 -- 172.21.15.78:0/3933700961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d87c200f0 con 0x559d87c276b0 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d88006580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.664+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d88006040 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40c4a30 tx=0x7f5fe40fce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcffbaf77e9c87b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5fdaffd640 1 -- 172.21.15.78:0/3933700961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4049070 con 0x559d87c276b0 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5fdaffd640 1 -- 172.21.15.78:0/3933700961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40c72d0 con 0x559d87c276b0 2024-04-25T14:10:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5fdaffd640 1 -- 172.21.15.78:0/3933700961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe403be60 con 0x559d87c276b0 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 -- 172.21.15.78:0/3933700961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 msgr2=0x559d88006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d88006040 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40c4a30 tx=0x7f5fe40fce50 comp rx=0 tx=0).stop 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 -- 172.21.15.78:0/3933700961 shutdown_connections 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c48390 0x559d88006580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c276b0 0x559d88006040 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/3933700961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d88005b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 -- 172.21.15.78:0/3933700961 >> 172.21.15.78:0/3933700961 conn(0x559d87c3c4e0 msgr2=0x559d87fee730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 -- 172.21.15.78:0/3933700961 shutdown_connections 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.665+0000 7f5ffe57d740 1 -- 172.21.15.78:0/3933700961 wait complete. 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 -- start start 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 Processor -- start 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87ff7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87ff8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01a99a0 con 0x559d87c48390 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01aae10 con 0x559d87c276b0 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01aa070 con 0x559d87c1f4d0 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ffe57d740 10 client.0 _ll_register_callbacks cb 0x7f5fe4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ff5b90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87ff8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ff5b90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87ff8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42096/0 (socket says 172.21.15.78:42096) 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2218126297 learned_addr learned my addr 172.21.15.78:0/2218126297 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.666+0000 7f5ff538f640 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2218126297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 msgr2=0x559d87ff7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ff4b8e640 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87ff7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 msgr2=0x559d87ff7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:36.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87ff7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ff5b90640 1 -- 172.21.15.78:0/2218126297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d87f0f4b0 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ff5b90640 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87ff8f20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000f80 tx=0x7f5fe800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b91c4c18ffd575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/2218126297 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe801a860 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/2218126297 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5fe8020040 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe40c72d0 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/2218126297 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5fe801aa00 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 10 client.18744 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559d87ffa5b0 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/2218126297 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40c72d0 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/2218126297 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5fe8024930 con 0x559d87c48390 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 10 client.18744 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 10 client.18744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 20 client.18744 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87bdc3a0 0x559d8806bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 4 client.18744 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.667+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d87f56800 con 0x559d87bdc3a0 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5ff538f640 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87bdc3a0 0x559d8806bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5ff538f640 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87bdc3a0 0x559d8806bb10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65855fb098893b7 server_cookie=3a30ebd1dad15fb1 in_seq=0 out_seq=0 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5fd8ff9640 10 client.18744 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5fd8ff9640 1 -- 172.21.15.78:0/2218126297 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d87f56800 con 0x559d87bdc3a0 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5fd8ff9640 10 client.18744 handle_command_reply: tid=0 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5ffe57d740 1 client.18744 shutdown 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 msgr2=0x559d87ff8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87ff8f20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000f80 tx=0x7f5fe800eb50 comp rx=0 tx=0).stop 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.668+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 shutdown_connections 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d87c48390 0x559d87ff8f20 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559d87c276b0 0x559d87ff7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x559d87bdc3a0 0x559d8806bb10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 1 --2- 172.21.15.78:0/2218126297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d87c1f4d0 0x559d87ff7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5fd8ff9640 0 client.18744 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 >> 172.21.15.78:0/2218126297 conn(0x559d87c3c4e0 msgr2=0x559d87fe8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 shutdown_connections 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 1 -- 172.21.15.78:0/2218126297 wait complete. 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ffe57d740 20 client.18744 trim_cache size 0 max 16384 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:10:36.675 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:10:36.676 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:10:36.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5fdc0afaf0 msgr2=0x7f5fdc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5fe801e000 tx=0x7f5fe801f000 comp rx=0 tx=0).stop 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 msgr2=0x7f5ff019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff019a7e0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4006010 tx=0x7f5fe4004340 comp rx=0 tx=0).stop 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 shutdown_connections 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff0109a40 0x7f5ff01a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ff0104d40 0x7f5ff019a7e0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 --2- 172.21.15.78:0/775836679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff0103b00 0x7f5ff019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 >> 172.21.15.78:0/775836679 conn(0x7f5ff00fed10 msgr2=0x7f5ff0105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 shutdown_connections 2024-04-25T14:10:36.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.669+0000 7f5ff75f3640 1 -- 172.21.15.78:0/775836679 wait complete. 2024-04-25T14:10:36.681 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:10:36.681 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:10:36.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 -- 172.21.15.78:0/82124265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744103c20 msgr2=0x7f4744104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 --2- 172.21.15.78:0/82124265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744103c20 0x7f4744104030 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f4730009900 tx=0x7f473002a410 comp rx=0 tx=0).stop 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 -- 172.21.15.78:0/82124265 shutdown_connections 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 --2- 172.21.15.78:0/82124265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4744108920 0x7f474410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 --2- 172.21.15.78:0/82124265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744103c20 0x7f4744104030 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 --2- 172.21.15.78:0/82124265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f47441029e0 0x7f47441036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 -- 172.21.15.78:0/82124265 >> 172.21.15.78:0/82124265 conn(0x7f47440fdbf0 msgr2=0x7f4744100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 -- 172.21.15.78:0/82124265 shutdown_connections 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.788+0000 7f474ab7a640 1 -- 172.21.15.78:0/82124265 wait complete. 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 Processor -- start 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 -- start start 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47441029e0 0x7f474419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4744103c20 0x7f474419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744108920 0x7f47441a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f474410e4e0 con 0x7f4744108920 2024-04-25T14:10:36.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f474410e360 con 0x7f47441029e0 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f474ab7a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f474410e660 con 0x7f4744103c20 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f4749117640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744108920 0x7f47441a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f4749117640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744108920 0x7f47441a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42116/0 (socket says 172.21.15.78:42116) 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f4749117640 1 -- 172.21.15.78:0/2585880950 learned_addr learned my addr 172.21.15.78:0/2585880950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.789+0000 7f4748916640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47441029e0 0x7f474419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4743fff640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4744103c20 0x7f474419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4749117640 1 -- 172.21.15.78:0/2585880950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4744103c20 msgr2=0x7f474419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4749117640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4744103c20 0x7f474419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4749117640 1 -- 172.21.15.78:0/2585880950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47441029e0 msgr2=0x7f474419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4749117640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47441029e0 0x7f474419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4749117640 1 -- 172.21.15.78:0/2585880950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47441a1d00 con 0x7f4744108920 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4749117640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744108920 0x7f47441a1740 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f473c00cce0 tx=0x7f473c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4741ffb640 1 -- 172.21.15.78:0/2585880950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c00f450 con 0x7f4744108920 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4741ffb640 1 -- 172.21.15.78:0/2585880950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c00faa0 con 0x7f4744108920 2024-04-25T14:10:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f4741ffb640 1 -- 172.21.15.78:0/2585880950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c018b70 con 0x7f4744108920 2024-04-25T14:10:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47441a1ff0 con 0x7f4744108920 2024-04-25T14:10:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.790+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47441a2450 con 0x7f4744108920 2024-04-25T14:10:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.791+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47441a2710 con 0x7f4744108920 2024-04-25T14:10:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.793+0000 7f4741ffb640 1 -- 172.21.15.78:0/2585880950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f473c018d10 con 0x7f4744108920 2024-04-25T14:10:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.793+0000 7f4741ffb640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47340afb70 0x7f47340b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.793+0000 7f4748916640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47340afb70 0x7f47340b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.793+0000 7f4741ffb640 1 -- 172.21.15.78:0/2585880950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f473c0df010 con 0x7f4744108920 2024-04-25T14:10:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.794+0000 7f4748916640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47340afb70 0x7f47340b2080 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f47441035e0 tx=0x7f473800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:36.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:36.798+0000 7f4741ffb640 1 -- 172.21.15.78:0/2585880950 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f473c014030 con 0x7f4744108920 2024-04-25T14:10:37.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.010+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47441a28e0 con 0x7f4744108920 2024-04-25T14:10:37.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.011+0000 7f4741ffb640 1 -- 172.21.15.78:0/2585880950 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f473c0a7be0 con 0x7f4744108920 2024-04-25T14:10:37.010 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:10:37.011 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:10:37.011 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47340afb70 msgr2=0x7f47340b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47340afb70 0x7f47340b2080 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f47441035e0 tx=0x7f473800b390 comp rx=0 tx=0).stop 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744108920 msgr2=0x7f47441a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744108920 0x7f47441a1740 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f473c00cce0 tx=0x7f473c00a620 comp rx=0 tx=0).stop 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 shutdown_connections 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f47340afb70 0x7f47340b2080 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4744108920 0x7f47441a1740 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4744103c20 0x7f474419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 --2- 172.21.15.78:0/2585880950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47441029e0 0x7f474419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 >> 172.21.15.78:0/2585880950 conn(0x7f47440fdbf0 msgr2=0x7f4744109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 shutdown_connections 2024-04-25T14:10:37.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.013+0000 7f474ab7a640 1 -- 172.21.15.78:0/2585880950 wait complete. 2024-04-25T14:10:37.024 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/2708896327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc109790 msgr2=0x7fbfbc10bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/2708896327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc109790 0x7fbfbc10bca0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009ca0 tx=0x7fbfb402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/2708896327 shutdown_connections 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/2708896327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc109790 0x7fbfbc10bca0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/2708896327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfbc106c20 0x7fbfbc109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/2708896327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc101e40 0x7fbfbc102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/2708896327 >> 172.21.15.78:0/2708896327 conn(0x7fbfbc0fd350 msgr2=0x7fbfbc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/2708896327 shutdown_connections 2024-04-25T14:10:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.134+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/2708896327 wait complete. 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 Processor -- start 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 -- start start 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc101e40 0x7fbfbc19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc106c20 0x7fbfbc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfbc109790 0x7fbfbc1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc10f620 con 0x7fbfbc101e40 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc10f4a0 con 0x7fbfbc106c20 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfc0f9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc10f7a0 con 0x7fbfbc109790 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc101e40 0x7fbfbc19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc101e40 0x7fbfbc19a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42128/0 (socket says 172.21.15.78:42128) 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 -- 172.21.15.78:0/810251127 learned_addr learned my addr 172.21.15.78:0/810251127 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc106c20 0x7fbfbc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfbc109790 0x7fbfbc1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 -- 172.21.15.78:0/810251127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfbc109790 msgr2=0x7fbfbc1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfbc109790 0x7fbfbc1a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 -- 172.21.15.78:0/810251127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc106c20 msgr2=0x7fbfbc19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc106c20 0x7fbfbc19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 -- 172.21.15.78:0/810251127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc1a1cd0 con 0x7fbfbc101e40 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfbc109790 0x7fbfbc1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc106c20 0x7fbfbc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfba59c640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc101e40 0x7fbfbc19a190 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005fd0 tx=0x7fbfac00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.135+0000 7fbfab7fe640 1 -- 172.21.15.78:0/810251127 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac017070 con 0x7fbfbc101e40 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.136+0000 7fbfab7fe640 1 -- 172.21.15.78:0/810251127 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac009c30 con 0x7fbfbc101e40 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.136+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc1a1fc0 con 0x7fbfbc101e40 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.136+0000 7fbfab7fe640 1 -- 172.21.15.78:0/810251127 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac003d30 con 0x7fbfbc101e40 2024-04-25T14:10:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.136+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc1a2420 con 0x7fbfbc101e40 2024-04-25T14:10:37.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.138+0000 7fbfc7e50740 1 Processor -- start 2024-04-25T14:10:37.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.138+0000 7fbfab7fe640 1 -- 172.21.15.78:0/810251127 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbfac003ed0 con 0x7fbfbc101e40 2024-04-25T14:10:37.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.138+0000 7fbfc7e50740 1 -- start start 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e9978b4d0 0x559e99b56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997936b0 0x559e99b57a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e99abad80 con 0x559e9978b4d0 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e99a74a20 con 0x559e997936b0 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559e99ab6d40 con 0x559e997b4390 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e9978b4d0 0x559e99b56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e9978b4d0 0x559e99b56d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42142/0 (socket says 172.21.15.78:42142) 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 -- 172.21.15.78:0/1716969845 learned_addr learned my addr 172.21.15.78:0/1716969845 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997936b0 0x559e99b57a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfab7fe640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbfa40afaf0 0x7fbfa40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 -- 172.21.15.78:0/1716969845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 msgr2=0x559e99b56810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b56810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 -- 172.21.15.78:0/1716969845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997936b0 msgr2=0x559e99b57a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997936b0 0x559e99b57a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 -- 172.21.15.78:0/1716969845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e99a7fc30 con 0x559e9978b4d0 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b56810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbfa40afaf0 0x7fbfa40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfab7fe640 1 -- 172.21.15.78:0/810251127 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbfac0de9d0 con 0x7fbfbc101e40 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.139+0000 7fbfba59c640 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e9978b4d0 0x559e99b56d90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbfac018000 tx=0x7fbfac007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262525ad0905156f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.140+0000 7fbfa8ff9640 1 -- 172.21.15.78:0/1716969845 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0a67f0 con 0x559e9978b4d0 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.140+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbfa40afaf0 0x7fbfa40b2000 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x559e99b57860 tx=0x7fbfb40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.140+0000 7fbfa8ff9640 1 -- 172.21.15.78:0/1716969845 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac0a69b0 con 0x559e9978b4d0 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.140+0000 7fbfa8ff9640 1 -- 172.21.15.78:0/1716969845 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac015110 con 0x559e9978b4d0 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.140+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1716969845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e9978b4d0 msgr2=0x559e99b56d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.140+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e9978b4d0 0x559e99b56d90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbfac018000 tx=0x7fbfac007f90 comp rx=0 tx=0).stop 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1716969845 shutdown_connections 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997936b0 0x559e99b57a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e9978b4d0 0x559e99b56d90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1716969845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b56810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1716969845 >> 172.21.15.78:0/1716969845 conn(0x559e997a84e0 msgr2=0x559e99b52020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1716969845 shutdown_connections 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1716969845 wait complete. 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- start start 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 Processor -- start 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b60500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997936b0 0x559e99b60a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997b4390 0x559e99b67930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e99b584d0 con 0x559e997b4390 2024-04-25T14:10:37.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559e99a41dc0 con 0x559e9978b4d0 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559e99b58650 con 0x559e997936b0 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfc7e50740 10 client.0 _ll_register_callbacks cb 0x559e99a5d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.141+0000 7fbfbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997b4390 0x559e99b67930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997b4390 0x559e99b67930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42152/0 (socket says 172.21.15.78:42152) 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 -- 172.21.15.78:0/1512261185 learned_addr learned my addr 172.21.15.78:0/1512261185 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfba59c640 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b60500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997936b0 0x559e99b60a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 -- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997936b0 msgr2=0x559e99b60a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997936b0 0x559e99b60a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 -- 172.21.15.78:0/1512261185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 msgr2=0x559e99b60500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b60500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 -- 172.21.15.78:0/1512261185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e997cc3f0 con 0x559e997b4390 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997b4390 0x559e99b67930 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4003bf0 tx=0x7fbfb4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61763f73aea81e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/1512261185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40047c0 con 0x559e997b4390 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/1512261185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbfb402f660 con 0x559e997b4390 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac0a69b0 con 0x559e997b4390 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/1512261185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbfb402f860 con 0x559e997b4390 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfc7e50740 10 client.18762 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.142+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559e998967d0 con 0x559e997b4390 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/1512261185 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac0a69b0 con 0x559e997b4390 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/1512261185 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbfb4030ce0 con 0x559e997b4390 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfc7e50740 10 client.18762 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfc7e50740 10 client.18762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfc7e50740 20 client.18762 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e99772f60 0x559e99b6e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfc7e50740 4 client.18762 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e99ac8990 con 0x559e99772f60 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfba59c640 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e99772f60 0x559e99b6e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfa1ffb640 10 client.18762 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.143+0000 7fbfba59c640 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e99772f60 0x559e99b6e1d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9b9b75f98879b64 server_cookie=37fbbedf817247d4 in_seq=0 out_seq=0 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/1512261185 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e99ac8990 con 0x559e99772f60 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfa1ffb640 10 client.18762 handle_command_reply: tid=0 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 client.18762 shutdown 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997b4390 msgr2=0x559e99b67930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997b4390 0x559e99b67930 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4003bf0 tx=0x7fbfb4005fd0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 shutdown_connections 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e99772f60 0x559e99b6e1d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997b4390 0x559e99b67930 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997936b0 0x559e99b60a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/1512261185 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b60500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfa1ffb640 0 client.18762 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 >> 172.21.15.78:0/1512261185 conn(0x559e997a84e0 msgr2=0x559e99b54f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 shutdown_connections 2024-04-25T14:10:37.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.144+0000 7fbfc7e50740 1 -- 172.21.15.78:0/1512261185 wait complete. 2024-04-25T14:10:37.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.145+0000 7fbfc7e50740 20 client.18762 trim_cache size 0 max 16384 2024-04-25T14:10:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 Processor -- start 2024-04-25T14:10:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 -- start start 2024-04-25T14:10:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e9978b4d0 0x559e99b5b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997b4390 0x559e99b67850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1ada80 con 0x559e997936b0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1ab350 con 0x559e997b4390 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a91a0 con 0x559e9978b4d0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42162/0 (socket says 172.21.15.78:42162) 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/3534179334 learned_addr learned my addr 172.21.15.78:0/3534179334 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997b4390 0x559e99b67850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfba59c640 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e9978b4d0 0x559e99b5b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/3534179334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e9978b4d0 msgr2=0x559e99b5b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e9978b4d0 0x559e99b5b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/3534179334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997b4390 msgr2=0x559e99b67850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997b4390 0x559e99b67850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/3534179334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e99b5e9e0 con 0x559e997936b0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.158+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997b4390 0x559e99b67850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b5b830 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009d60 tx=0x7fbfb0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cd367eba362dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfa3fff640 1 -- 172.21.15.78:0/3534179334 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0020020 con 0x559e997936b0 2024-04-25T14:10:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfa3fff640 1 -- 172.21.15.78:0/3534179334 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb00054d0 con 0x559e997936b0 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfa3fff640 1 -- 172.21.15.78:0/3534179334 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0011760 con 0x559e997936b0 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfc7e50740 1 -- 172.21.15.78:0/3534179334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 msgr2=0x559e99b5b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b5b830 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009d60 tx=0x7fbfb0013830 comp rx=0 tx=0).stop 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfc7e50740 1 -- 172.21.15.78:0/3534179334 shutdown_connections 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e997b4390 0x559e99b67850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b5b830 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.159+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/3534179334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e9978b4d0 0x559e99b5b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 -- 172.21.15.78:0/3534179334 >> 172.21.15.78:0/3534179334 conn(0x559e997a84e0 msgr2=0x559e99b60500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 -- 172.21.15.78:0/3534179334 shutdown_connections 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 -- 172.21.15.78:0/3534179334 wait complete. 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 -- start start 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 Processor -- start 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b64f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a9810 con 0x559e997936b0 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1aac80 con 0x559e9978b4d0 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc1a9ee0 con 0x559e997b4390 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfc7e50740 10 client.0 _ll_register_callbacks cb 0x7fbfb0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfb9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b64f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfb9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b64f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42178/0 (socket says 172.21.15.78:42178) 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/108317803 learned_addr learned my addr 172.21.15.78:0/108317803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfba59c640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 msgr2=0x559e99b65da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b65da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.160+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/108317803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 msgr2=0x559e99b64a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b64a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfb9d9b640 1 -- 172.21.15.78:0/108317803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e99b67d90 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfba59c640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfbad9d640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b65da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfb9d9b640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b64f50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00058c0 tx=0x7fbfb0005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a44d406b065f028 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/108317803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb002a030 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/108317803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fbfb0005230 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb00054d0 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/108317803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbfb0020040 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 10 client.18768 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559e99b67440 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/108317803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb00054d0 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/108317803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbfb001b070 con 0x559e997936b0 2024-04-25T14:10:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 10 client.18768 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 10 client.18768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 20 client.18768 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e997483a0 0x559e99bd85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 4 client.18768 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.161+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e99ac2800 con 0x559e997483a0 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.162+0000 7fbfba59c640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e997483a0 0x559e99bd85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.162+0000 7fbfa1ffb640 10 client.18768 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.162+0000 7fbfba59c640 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e997483a0 0x559e99bd85c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54df56ddd20413d4 server_cookie=a56c46f4399e1567 in_seq=0 out_seq=0 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.162+0000 7fbfa1ffb640 1 -- 172.21.15.78:0/108317803 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e99ac2800 con 0x559e997483a0 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.162+0000 7fbfa1ffb640 10 client.18768 handle_command_reply: tid=0 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.162+0000 7fbfc7e50740 1 client.18768 shutdown 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.162+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 msgr2=0x559e99b64f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b64f50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00058c0 tx=0x7fbfb0005750 comp rx=0 tx=0).stop 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 shutdown_connections 2024-04-25T14:10:37.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x559e997483a0 0x559e99bd85c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e997b4390 0x559e99b65da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e997936b0 0x559e99b64f50 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfa1ffb640 0 client.18768 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 --2- 172.21.15.78:0/108317803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559e9978b4d0 0x559e99b64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 >> 172.21.15.78:0/108317803 conn(0x559e997a84e0 msgr2=0x559e99b604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 shutdown_connections 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 1 -- 172.21.15.78:0/108317803 wait complete. 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc7e50740 20 client.18768 trim_cache size 0 max 16384 2024-04-25T14:10:37.169 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:10:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbfa40afaf0 msgr2=0x7fbfa40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbfa40afaf0 0x7fbfa40b2000 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x559e99b57860 tx=0x7fbfb40095c0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc101e40 msgr2=0x7fbfbc19a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc101e40 0x7fbfbc19a190 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005fd0 tx=0x7fbfac00dde0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 shutdown_connections 2024-04-25T14:10:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfbc109790 0x7fbfbc1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfbc106c20 0x7fbfbc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fbfa40afaf0 0x7fbfa40b2000 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 --2- 172.21.15.78:0/810251127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfbc101e40 0x7fbfbc19a190 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 >> 172.21.15.78:0/810251127 conn(0x7fbfbc0fd350 msgr2=0x7fbfbc107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 shutdown_connections 2024-04-25T14:10:37.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.163+0000 7fbfc0f9d640 1 -- 172.21.15.78:0/810251127 wait complete. 2024-04-25T14:10:37.174 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:10:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 -- 172.21.15.78:0/2204246141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c44108920 msgr2=0x7f5c4410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/2204246141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c44108920 0x7f5c4410ad40 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009ca0 tx=0x7f5c3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 -- 172.21.15.78:0/2204246141 shutdown_connections 2024-04-25T14:10:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/2204246141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c44108920 0x7f5c4410ad40 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/2204246141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c44103c20 0x7f5c44104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/2204246141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c441029e0 0x7f5c441036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 -- 172.21.15.78:0/2204246141 >> 172.21.15.78:0/2204246141 conn(0x7f5c440fdbd0 msgr2=0x7f5c44100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 -- 172.21.15.78:0/2204246141 shutdown_connections 2024-04-25T14:10:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 -- 172.21.15.78:0/2204246141 wait complete. 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 Processor -- start 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.284+0000 7f5c49bd7640 1 -- start start 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c49bd7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c441029e0 0x7f5c4419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c49bd7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c44103c20 0x7f5c4419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c49bd7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c44108920 0x7f5c441a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c49bd7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4410e510 con 0x7f5c441029e0 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c49bd7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4410e390 con 0x7f5c44108920 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c49bd7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4410e690 con 0x7f5c44103c20 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c441029e0 0x7f5c4419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c441029e0 0x7f5c4419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42210/0 (socket says 172.21.15.78:42210) 2024-04-25T14:10:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 -- 172.21.15.78:0/72653044 learned_addr learned my addr 172.21.15.78:0/72653044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c43fff640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c44108920 0x7f5c441a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c44103c20 0x7f5c4419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 -- 172.21.15.78:0/72653044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c44103c20 msgr2=0x7f5c4419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c44103c20 0x7f5c4419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 -- 172.21.15.78:0/72653044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c44108920 msgr2=0x7f5c441a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c44108920 0x7f5c441a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 -- 172.21.15.78:0/72653044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c441a2060 con 0x7f5c441029e0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.285+0000 7f5c437fe640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c441029e0 0x7f5c4419a520 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005d00 tx=0x7f5c3400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.286+0000 7f5c40ff9640 1 -- 172.21.15.78:0/72653044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34017070 con 0x7f5c441029e0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.286+0000 7f5c40ff9640 1 -- 172.21.15.78:0/72653044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c34009c30 con 0x7f5c441029e0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.286+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c441a2350 con 0x7f5c441029e0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.286+0000 7f5c40ff9640 1 -- 172.21.15.78:0/72653044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400f620 con 0x7f5c441029e0 2024-04-25T14:10:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.286+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c441a2850 con 0x7f5c441029e0 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.288+0000 7f5c40ff9640 1 -- 172.21.15.78:0/72653044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5c3401a070 con 0x7f5c441029e0 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.288+0000 7f5c50b65740 1 Processor -- start 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.288+0000 7f5c50b65740 1 -- start start 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.288+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d4e390 0x5587410f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d254d0 0x5587410f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c50b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558741054d80 con 0x558740d2d6b0 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c50b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55874100ea20 con 0x558740d254d0 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c50b65740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558741050d40 con 0x558740d4e390 2024-04-25T14:10:37.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42218/0 (socket says 172.21.15.78:42218) 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 -- 172.21.15.78:0/3319625419 learned_addr learned my addr 172.21.15.78:0/3319625419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c40ff9640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c2c0afaf0 0x7f5c2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c437fe640 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d254d0 0x5587410f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c43fff640 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d4e390 0x5587410f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 -- 172.21.15.78:0/3319625419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d4e390 msgr2=0x5587410f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d4e390 0x5587410f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 -- 172.21.15.78:0/3319625419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d254d0 msgr2=0x5587410f0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d254d0 0x5587410f0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 -- 172.21.15.78:0/3319625419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558741019c30 con 0x558740d2d6b0 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c43fff640 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d4e390 0x5587410f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c43fff640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c2c0afaf0 0x7f5c2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c40ff9640 1 -- 172.21.15.78:0/72653044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5c340de5e0 con 0x7f5c441029e0 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410f1a20 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800bc90 tx=0x7f5c380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d80dc4f7839427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c29ffb640 1 -- 172.21.15.78:0/3319625419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38005220 con 0x558740d2d6b0 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c29ffb640 1 -- 172.21.15.78:0/3319625419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3800be40 con 0x558740d2d6b0 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.289+0000 7f5c29ffb640 1 -- 172.21.15.78:0/3319625419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38011790 con 0x558740d2d6b0 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 -- 172.21.15.78:0/3319625419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 msgr2=0x5587410f1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c43fff640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c2c0afaf0 0x7f5c2c0b2000 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c004af0 tx=0x7f5c3c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410f1a20 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800bc90 tx=0x7f5c380107a0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 -- 172.21.15.78:0/3319625419 shutdown_connections 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410f1a20 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d254d0 0x5587410f0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 --2- 172.21.15.78:0/3319625419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d4e390 0x5587410f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 -- 172.21.15.78:0/3319625419 >> 172.21.15.78:0/3319625419 conn(0x558740d424e0 msgr2=0x5587410ec020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 -- 172.21.15.78:0/3319625419 shutdown_connections 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.290+0000 7f5c50b65740 1 -- 172.21.15.78:0/3319625419 wait complete. 2024-04-25T14:10:37.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 -- start start 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 Processor -- start 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410faa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x5587411019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5587410f2530 con 0x558740d2d6b0 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558740fdbdc0 con 0x558740d4e390 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5587410f26b0 con 0x558740d254d0 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c50b65740 10 client.0 _ll_register_callbacks cb 0x558740ff7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410faa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42228/0 (socket says 172.21.15.78:42228) 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c42ffd640 1 -- 172.21.15.78:0/2101653415 learned_addr learned my addr 172.21.15.78:0/2101653415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.291+0000 7f5c437fe640 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c42ffd640 1 -- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 msgr2=0x5587410fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410fa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c43fff640 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x5587411019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c42ffd640 1 -- 172.21.15.78:0/2101653415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 msgr2=0x5587411019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x5587411019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c42ffd640 1 -- 172.21.15.78:0/2101653415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558740d663f0 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410faa80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f5c38000c00 tx=0x7f5c38011820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3c6db0819c76dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c22ffd640 1 -- 172.21.15.78:0/2101653415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3802a060 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c22ffd640 1 -- 172.21.15.78:0/2101653415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5c38011c20 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558741062990 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c22ffd640 1 -- 172.21.15.78:0/2101653415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5c38011e20 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 10 client.18780 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558740e307d0 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c22ffd640 1 -- 172.21.15.78:0/2101653415 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558741062990 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c22ffd640 1 -- 172.21.15.78:0/2101653415 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5c380306c0 con 0x558740d2d6b0 2024-04-25T14:10:37.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 10 client.18780 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 10 client.18780 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 20 client.18780 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740d0cf60 0x558741107e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 4 client.18780 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.292+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55874105c800 con 0x558740d0cf60 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.293+0000 7f5c437fe640 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740d0cf60 0x558741107e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.293+0000 7f5c437fe640 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740d0cf60 0x558741107e00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9364b0d2a1993714 server_cookie=c3f786814f643dca in_seq=0 out_seq=0 2024-04-25T14:10:37.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.293+0000 7f5c22ffd640 10 client.18780 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:37.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c22ffd640 1 -- 172.21.15.78:0/2101653415 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55874105c800 con 0x558740d0cf60 2024-04-25T14:10:37.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c22ffd640 10 client.18780 handle_command_reply: tid=0 2024-04-25T14:10:37.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 client.18780 shutdown 2024-04-25T14:10:37.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 msgr2=0x5587410faa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410faa80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f5c38000c00 tx=0x7f5c38011820 comp rx=0 tx=0).stop 2024-04-25T14:10:37.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 shutdown_connections 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x5587411019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d2d6b0 0x5587410faa80 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740d0cf60 0x558741107e00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2101653415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410fa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c22ffd640 0 client.18780 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 >> 172.21.15.78:0/2101653415 conn(0x558740d424e0 msgr2=0x5587410eef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 shutdown_connections 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 1 -- 172.21.15.78:0/2101653415 wait complete. 2024-04-25T14:10:37.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.294+0000 7f5c50b65740 20 client.18780 trim_cache size 0 max 16384 2024-04-25T14:10:37.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 Processor -- start 2024-04-25T14:10:37.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 -- start start 2024-04-25T14:10:37.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d2d6b0 0x5587410f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d4e390 0x55874110d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441adbd0 con 0x558740d4e390 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441ab4a0 con 0x558740d2d6b0 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c50b65740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441a92f0 con 0x558740d254d0 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d4e390 0x55874110d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d4e390 0x55874110d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42244/0 (socket says 172.21.15.78:42244) 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 -- 172.21.15.78:0/2238260892 learned_addr learned my addr 172.21.15.78:0/2238260892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c437fe640 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 -- 172.21.15.78:0/2238260892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 msgr2=0x5587410f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 -- 172.21.15.78:0/2238260892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d2d6b0 msgr2=0x5587410f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d2d6b0 0x5587410f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 -- 172.21.15.78:0/2238260892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587410fb680 con 0x558740d4e390 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.308+0000 7f5c43fff640 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d4e390 0x55874110d570 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009ca0 tx=0x7f5c3c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e15dbc1c2019aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c297fa640 1 -- 172.21.15.78:0/2238260892 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c003e40 con 0x558740d4e390 2024-04-25T14:10:37.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c297fa640 1 -- 172.21.15.78:0/2238260892 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c036070 con 0x558740d4e390 2024-04-25T14:10:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c297fa640 1 -- 172.21.15.78:0/2238260892 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c02f8b0 con 0x558740d4e390 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 -- 172.21.15.78:0/2238260892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d4e390 msgr2=0x55874110d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d4e390 0x55874110d570 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009ca0 tx=0x7f5c3c041000 comp rx=0 tx=0).stop 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 -- 172.21.15.78:0/2238260892 shutdown_connections 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d4e390 0x55874110d570 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d2d6b0 0x5587410f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 --2- 172.21.15.78:0/2238260892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d254d0 0x5587410f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 -- 172.21.15.78:0/2238260892 >> 172.21.15.78:0/2238260892 conn(0x558740d424e0 msgr2=0x5587410f0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 -- 172.21.15.78:0/2238260892 shutdown_connections 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.309+0000 7f5c50b65740 1 -- 172.21.15.78:0/2238260892 wait complete. 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 -- start start 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 Processor -- start 2024-04-25T14:10:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d254d0 0x558741100580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d2d6b0 0x558741100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x55874116e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441a9960 con 0x558740d254d0 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441aadd0 con 0x558740d4e390 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441aa030 con 0x558740d2d6b0 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c50b65740 10 client.0 _ll_register_callbacks cb 0x7f5c3800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d254d0 0x558741100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d254d0 0x558741100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42248/0 (socket says 172.21.15.78:42248) 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c437fe640 1 -- 172.21.15.78:0/262256370 learned_addr learned my addr 172.21.15.78:0/262256370 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c43fff640 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x55874116e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d2d6b0 0x558741100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c437fe640 1 -- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d2d6b0 msgr2=0x558741100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c437fe640 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d2d6b0 0x558741100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c437fe640 1 -- 172.21.15.78:0/262256370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 msgr2=0x55874116e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.310+0000 7f5c437fe640 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x55874116e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c437fe640 1 -- 172.21.15.78:0/262256370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55874116eb50 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c437fe640 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d254d0 0x558741100580 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f5c340a6860 tx=0x7f5c34082040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528a4e82a228d47c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c22ffd640 1 -- 172.21.15.78:0/262256370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34021040 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c22ffd640 1 -- 172.21.15.78:0/262256370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5c340a3150 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c3c036070 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c22ffd640 1 -- 172.21.15.78:0/262256370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5c340a6670 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 10 client.18786 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55874116fe40 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c22ffd640 1 -- 172.21.15.78:0/262256370 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c036070 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c22ffd640 1 -- 172.21.15.78:0/262256370 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5c340a2040 con 0x558740d254d0 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 10 client.18786 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 10 client.18786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 20 client.18786 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740ce23a0 0x5587411727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 4 client.18786 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.311+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558740fd7610 con 0x558740ce23a0 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740ce23a0 0x5587411727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c22ffd640 10 client.18786 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c42ffd640 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740ce23a0 0x5587411727b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7893ec378ca47278 server_cookie=74e9b734e450d47d in_seq=0 out_seq=0 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c22ffd640 1 -- 172.21.15.78:0/262256370 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558740fd7610 con 0x558740ce23a0 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c22ffd640 10 client.18786 handle_command_reply: tid=0 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 client.18786 shutdown 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d254d0 msgr2=0x558741100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d254d0 0x558741100580 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f5c340a6860 tx=0x7f5c34082040 comp rx=0 tx=0).stop 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 shutdown_connections 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558740d4e390 0x55874116e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x558740ce23a0 0x5587411727b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558740d2d6b0 0x558741100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c22ffd640 0 client.18786 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 --2- 172.21.15.78:0/262256370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558740d254d0 0x558741100580 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 >> 172.21.15.78:0/262256370 conn(0x558740d424e0 msgr2=0x5587410f0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 shutdown_connections 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.312+0000 7f5c50b65740 1 -- 172.21.15.78:0/262256370 wait complete. 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c50b65740 20 client.18786 trim_cache size 0 max 16384 2024-04-25T14:10:37.318 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:10:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c2c0afaf0 msgr2=0x7f5c2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c2c0afaf0 0x7f5c2c0b2000 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c004af0 tx=0x7f5c3c009360 comp rx=0 tx=0).stop 2024-04-25T14:10:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c441029e0 msgr2=0x7f5c4419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c441029e0 0x7f5c4419a520 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005d00 tx=0x7f5c3400dde0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 shutdown_connections 2024-04-25T14:10:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c44108920 0x7f5c441a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c44103c20 0x7f5c4419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5c2c0afaf0 0x7f5c2c0b2000 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 --2- 172.21.15.78:0/72653044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c441029e0 0x7f5c4419a520 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 >> 172.21.15.78:0/72653044 conn(0x7f5c440fdbd0 msgr2=0x7f5c44109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 shutdown_connections 2024-04-25T14:10:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.313+0000 7f5c49bd7640 1 -- 172.21.15.78:0/72653044 wait complete. 2024-04-25T14:10:37.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:37 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2585880950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:37.324 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:10:37.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:37 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2585880950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:37.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.433+0000 7f861d7ce640 1 -- 172.21.15.78:0/190427853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 msgr2=0x7f8618109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.433+0000 7f861d7ce640 1 --2- 172.21.15.78:0/190427853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f8618109350 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f860c009900 tx=0x7f860c02a410 comp rx=0 tx=0).stop 2024-04-25T14:10:37.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 -- 172.21.15.78:0/190427853 shutdown_connections 2024-04-25T14:10:37.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 --2- 172.21.15.78:0/190427853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8618109890 0x7f861810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 --2- 172.21.15.78:0/190427853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f8618109350 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 --2- 172.21.15.78:0/190427853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86181045d0 0x7f86181069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 -- 172.21.15.78:0/190427853 >> 172.21.15.78:0/190427853 conn(0x7f86180fdbd0 msgr2=0x7f8618100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 -- 172.21.15.78:0/190427853 shutdown_connections 2024-04-25T14:10:37.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 -- 172.21.15.78:0/190427853 wait complete. 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 Processor -- start 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.434+0000 7f861d7ce640 1 -- start start 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86181045d0 0x7f861819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f861819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8618109890 0x7f86181a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f861810f610 con 0x7f8618106f30 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f861810f490 con 0x7f8618109890 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f861810f790 con 0x7f86181045d0 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f861819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f861819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42264/0 (socket says 172.21.15.78:42264) 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 -- 172.21.15.78:0/3671687627 learned_addr learned my addr 172.21.15.78:0/3671687627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f8616ffd640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86181045d0 0x7f861819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86177fe640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8618109890 0x7f86181a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 -- 172.21.15.78:0/3671687627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86181045d0 msgr2=0x7f861819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86181045d0 0x7f861819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 -- 172.21.15.78:0/3671687627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8618109890 msgr2=0x7f86181a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8618109890 0x7f86181a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 -- 172.21.15.78:0/3671687627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86181a2060 con 0x7f8618106f30 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f86167fc640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f861819ab10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f860c006010 tx=0x7f860c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f85fffff640 1 -- 172.21.15.78:0/3671687627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c03f030 con 0x7f8618106f30 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f85fffff640 1 -- 172.21.15.78:0/3671687627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f860c003aa0 con 0x7f8618106f30 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86181a22f0 con 0x7f8618106f30 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f85fffff640 1 -- 172.21.15.78:0/3671687627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c035070 con 0x7f8618106f30 2024-04-25T14:10:37.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.435+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86181a27e0 con 0x7f8618106f30 2024-04-25T14:10:37.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f85fffff640 1 -- 172.21.15.78:0/3671687627 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f860c042030 con 0x7f8618106f30 2024-04-25T14:10:37.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 Processor -- start 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 -- start start 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f85fffff640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f86000afaf0 0x7f86000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556490605d80 con 0x5564902d64d0 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5564905bfa20 con 0x5564902de6b0 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8624680740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556490601d40 con 0x5564902ff390 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f86167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f8616ffd640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f86000afaf0 0x7f86000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f86167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906a61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42266/0 (socket says 172.21.15.78:42266) 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.438+0000 7f86167fc640 1 -- 172.21.15.78:0/2184756203 learned_addr learned my addr 172.21.15.78:0/2184756203 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86177fe640 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f85fffff640 1 -- 172.21.15.78:0/3671687627 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f860c0ff210 con 0x7f8618106f30 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86177fe640 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86167fc640 1 -- 172.21.15.78:0/2184756203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 msgr2=0x5564906a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86167fc640 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86167fc640 1 -- 172.21.15.78:0/2184756203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 msgr2=0x5564906a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86167fc640 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86167fc640 1 -- 172.21.15.78:0/2184756203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564905cac30 con 0x5564902d64d0 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f86167fc640 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906a61a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f860c005f70 tx=0x7f860c0c2fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f83127bae7c0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f8616ffd640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f86000afaf0 0x7f86000b2000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f861806a8b0 tx=0x7f8608005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f85fd7fa640 1 -- 172.21.15.78:0/2184756203 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c049070 con 0x5564902d64d0 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f85fd7fa640 1 -- 172.21.15.78:0/2184756203 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f860c03b460 con 0x5564902d64d0 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.439+0000 7f85fd7fa640 1 -- 172.21.15.78:0/2184756203 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0cba90 con 0x5564902d64d0 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 -- 172.21.15.78:0/2184756203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 msgr2=0x5564906a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906a61a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f860c005f70 tx=0x7f860c0c2fa0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 -- 172.21.15.78:0/2184756203 shutdown_connections 2024-04-25T14:10:37.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906a61a0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 --2- 172.21.15.78:0/2184756203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 -- 172.21.15.78:0/2184756203 >> 172.21.15.78:0/2184756203 conn(0x5564902f34e0 msgr2=0x55649069d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 -- 172.21.15.78:0/2184756203 shutdown_connections 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 -- 172.21.15.78:0/2184756203 wait complete. 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 -- start start 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.440+0000 7f8624680740 1 Processor -- start 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8624680740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8624680740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8624680740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8624680740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5564906a7700 con 0x5564902d64d0 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8624680740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55649058cdc0 con 0x5564902de6b0 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8624680740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564906a7880 con 0x5564902ff390 2024-04-25T14:10:37.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8624680740 10 client.0 _ll_register_callbacks cb 0x5564905a8600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42272/0 (socket says 172.21.15.78:42272) 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 -- 172.21.15.78:0/229647419 learned_addr learned my addr 172.21.15.78:0/229647419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f86167fc640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f86177fe640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 -- 172.21.15.78:0/229647419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 msgr2=0x5564906b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 -- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 msgr2=0x5564906afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 -- 172.21.15.78:0/229647419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564903173f0 con 0x5564902d64d0 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f86177fe640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f8616ffd640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906af780 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f8608003880 tx=0x7f8608004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4685b1fbccba7422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f85faff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:37.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f85faff5640 1 -- 172.21.15.78:0/229647419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860800eac0 con 0x5564902d64d0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.441+0000 7f85faff5640 1 -- 172.21.15.78:0/229647419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f860800ec60 con 0x5564902d64d0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556490613990 con 0x5564902d64d0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f85faff5640 1 -- 172.21.15.78:0/229647419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f860800ee60 con 0x5564902d64d0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 10 client.18798 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5564903e17d0 con 0x5564902d64d0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f85faff5640 1 -- 172.21.15.78:0/229647419 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556490613990 con 0x5564902d64d0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f85faff5640 1 -- 172.21.15.78:0/229647419 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8608031670 con 0x5564902d64d0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 10 client.18798 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 10 client.18798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 20 client.18798 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902bdf60 0x5564906bd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 4 client.18798 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55649060d800 con 0x5564902bdf60 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.442+0000 7f86167fc640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902bdf60 0x5564906bd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.443+0000 7f85faff5640 10 client.18798 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:37.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.443+0000 7f86167fc640 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902bdf60 0x5564906bd530 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee64c5297a09012a server_cookie=bf1d5ef346171fd in_seq=0 out_seq=0 2024-04-25T14:10:37.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f85faff5640 1 -- 172.21.15.78:0/229647419 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55649060d800 con 0x5564902bdf60 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f85faff5640 10 client.18798 handle_command_reply: tid=0 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 client.18798 shutdown 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 msgr2=0x5564906af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906af780 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f8608003880 tx=0x7f8608004730 comp rx=0 tx=0).stop 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 shutdown_connections 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902bdf60 0x5564906bd530 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902ff390 0x5564906b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f85faff5640 0 client.18798 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 --2- 172.21.15.78:0/229647419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906af780 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 >> 172.21.15.78:0/229647419 conn(0x5564902f34e0 msgr2=0x55649069c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 shutdown_connections 2024-04-25T14:10:37.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 1 -- 172.21.15.78:0/229647419 wait complete. 2024-04-25T14:10:37.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.444+0000 7f8624680740 20 client.18798 trim_cache size 0 max 16384 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.457+0000 7f8624680740 1 Processor -- start 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.457+0000 7f8624680740 1 -- start start 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8624680740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902bdf60 0x5564906ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8624680740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902d64d0 0x5564906b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8624680740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902de6b0 0x5564906c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8624680740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86181ab990 con 0x5564902bdf60 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8624680740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86181a9260 con 0x5564902d64d0 2024-04-25T14:10:37.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8624680740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86181a70b0 con 0x5564902de6b0 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902bdf60 0x5564906ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902bdf60 0x5564906ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42282/0 (socket says 172.21.15.78:42282) 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 -- 172.21.15.78:0/2622198067 learned_addr learned my addr 172.21.15.78:0/2622198067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f86167fc640 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902d64d0 0x5564906b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 -- 172.21.15.78:0/2622198067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902de6b0 msgr2=0x5564906c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902de6b0 0x5564906c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 -- 172.21.15.78:0/2622198067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902d64d0 msgr2=0x5564906b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902d64d0 0x5564906b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 -- 172.21.15.78:0/2622198067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564906ae9f0 con 0x5564902bdf60 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f86167fc640 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902d64d0 0x5564906b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f8616ffd640 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902bdf60 0x5564906ae4b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f86080059f0 tx=0x7f8608003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e766f2c291ff7338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f85fcff9640 1 -- 172.21.15.78:0/2622198067 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860803f070 con 0x5564902bdf60 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f85fcff9640 1 -- 172.21.15.78:0/2622198067 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8608031d00 con 0x5564902bdf60 2024-04-25T14:10:37.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.458+0000 7f85fcff9640 1 -- 172.21.15.78:0/2622198067 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608036070 con 0x5564902bdf60 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 -- 172.21.15.78:0/2622198067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902bdf60 msgr2=0x5564906ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902bdf60 0x5564906ae4b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f86080059f0 tx=0x7f8608003e00 comp rx=0 tx=0).stop 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 -- 172.21.15.78:0/2622198067 shutdown_connections 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902de6b0 0x5564906c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902d64d0 0x5564906b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 --2- 172.21.15.78:0/2622198067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902bdf60 0x5564906ae4b0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 -- 172.21.15.78:0/2622198067 >> 172.21.15.78:0/2622198067 conn(0x5564902f34e0 msgr2=0x5564906adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 -- 172.21.15.78:0/2622198067 shutdown_connections 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 -- 172.21.15.78:0/2622198067 wait complete. 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 -- start start 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.459+0000 7f8624680740 1 Processor -- start 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902bdf60 0x5564906b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86181a7720 con 0x5564902d64d0 2024-04-25T14:10:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86181a8b90 con 0x5564902de6b0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86181a7df0 con 0x5564902bdf60 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 10 client.0 _ll_register_callbacks cb 0x7f8608009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42294/0 (socket says 172.21.15.78:42294) 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 -- 172.21.15.78:0/1639641745 learned_addr learned my addr 172.21.15.78:0/1639641745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 -- 172.21.15.78:0/1639641745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902bdf60 msgr2=0x5564906b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902bdf60 0x5564906b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 -- 172.21.15.78:0/1639641745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 msgr2=0x5564906b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 -- 172.21.15.78:0/1639641745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564906b58a0 con 0x5564902d64d0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f86167fc640 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906b4470 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f860c004010 tx=0x7f860c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f0e2097b4d8196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f85faff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f85faff5640 1 -- 172.21.15.78:0/1639641745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0e6070 con 0x5564902d64d0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f85faff5640 1 -- 172.21.15.78:0/1639641745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f860c0b00b0 con 0x5564902d64d0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.460+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8608031d00 con 0x5564902d64d0 2024-04-25T14:10:37.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f85faff5640 1 -- 172.21.15.78:0/1639641745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f860c049040 con 0x5564902d64d0 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 10 client.18804 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5564906b6b60 con 0x5564902d64d0 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f85faff5640 1 -- 172.21.15.78:0/1639641745 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8608031d00 con 0x5564902d64d0 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f85faff5640 1 -- 172.21.15.78:0/1639641745 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f860c0d9050 con 0x5564902d64d0 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 10 client.18804 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 10 client.18804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 20 client.18804 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902ff390 0x556490727990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 4 client.18804 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556490588610 con 0x5564902ff390 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.461+0000 7f8616ffd640 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902ff390 0x556490727990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f85faff5640 10 client.18804 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:37.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8616ffd640 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902ff390 0x556490727990 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed64d3ad377a7ca6 server_cookie=62f64655212f8153 in_seq=0 out_seq=0 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f85faff5640 1 -- 172.21.15.78:0/1639641745 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556490588610 con 0x5564902ff390 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f85faff5640 10 client.18804 handle_command_reply: tid=0 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 client.18804 shutdown 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 msgr2=0x5564906b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906b4470 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f860c004010 tx=0x7f860c02a410 comp rx=0 tx=0).stop 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 shutdown_connections 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5564902de6b0 0x5564906b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564902d64d0 0x5564906b4470 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5564902ff390 0x556490727990 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 --2- 172.21.15.78:0/1639641745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564902bdf60 0x5564906b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f85faff5640 0 client.18804 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:37.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 >> 172.21.15.78:0/1639641745 conn(0x5564902f34e0 msgr2=0x55649069fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 shutdown_connections 2024-04-25T14:10:37.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 1 -- 172.21.15.78:0/1639641745 wait complete. 2024-04-25T14:10:37.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.462+0000 7f8624680740 20 client.18804 trim_cache size 0 max 16384 2024-04-25T14:10:37.468 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f86000afaf0 msgr2=0x7f86000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f86000afaf0 0x7f86000b2000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f861806a8b0 tx=0x7f8608005f90 comp rx=0 tx=0).stop 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 msgr2=0x7f861819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f861819ab10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f860c006010 tx=0x7f860c031a50 comp rx=0 tx=0).stop 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 shutdown_connections 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8618109890 0x7f86181a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f86000afaf0 0x7f86000b2000 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8618106f30 0x7f861819ab10 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 --2- 172.21.15.78:0/3671687627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86181045d0 0x7f861819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 >> 172.21.15.78:0/3671687627 conn(0x7f86180fdbd0 msgr2=0x7f8618104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 shutdown_connections 2024-04-25T14:10:37.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:37.463+0000 7f861d7ce640 1 -- 172.21.15.78:0/3671687627 wait complete. 2024-04-25T14:10:37.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:37 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2585880950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:10:38.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:38 smithi129 ceph-mon[32569]: pgmap v681: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:38 smithi078 ceph-mon[29163]: pgmap v681: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:38.474 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T14:10:38.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:10:38.474 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T14:10:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:38 smithi112 ceph-mon[32617]: pgmap v681: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:38.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.600+0000 7fc268c92640 1 -- 172.21.15.78:0/60201451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 msgr2=0x7fc264101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.600+0000 7fc268c92640 1 --2- 172.21.15.78:0/60201451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc264101bd0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc258009900 tx=0x7fc25802a410 comp rx=0 tx=0).stop 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 -- 172.21.15.78:0/60201451 shutdown_connections 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 --2- 172.21.15.78:0/60201451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc264102260 0x7fc26410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 --2- 172.21.15.78:0/60201451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc264101bd0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 --2- 172.21.15.78:0/60201451 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2641050b0 0x7fc2641054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 -- 172.21.15.78:0/60201451 >> 172.21.15.78:0/60201451 conn(0x7fc2640fd380 msgr2=0x7fc2640ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 -- 172.21.15.78:0/60201451 shutdown_connections 2024-04-25T14:10:38.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 -- 172.21.15.78:0/60201451 wait complete. 2024-04-25T14:10:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 Processor -- start 2024-04-25T14:10:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.601+0000 7fc268c92640 1 -- start start 2024-04-25T14:10:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc26419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc264102260 0x7fc26419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2641050b0 0x7fc2641a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410f640 con 0x7fc264101760 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410f4c0 con 0x7fc2641050b0 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410f7c0 con 0x7fc264102260 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc26419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc26419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42320/0 (socket says 172.21.15.78:42320) 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 -- 172.21.15.78:0/2054270667 learned_addr learned my addr 172.21.15.78:0/2054270667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc261d9b640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc264102260 0x7fc26419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 -- 172.21.15.78:0/2054270667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc264102260 msgr2=0x7fc26419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc264102260 0x7fc26419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 -- 172.21.15.78:0/2054270667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2641050b0 msgr2=0x7fc2641a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2641050b0 0x7fc2641a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 -- 172.21.15.78:0/2054270667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2641a1d80 con 0x7fc264101760 2024-04-25T14:10:38.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc261d9b640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc264102260 0x7fc26419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc26259c640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc26419a2b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc254005fd0 tx=0x7fc25400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc2537fe640 1 -- 172.21.15.78:0/2054270667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254017070 con 0x7fc264101760 2024-04-25T14:10:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2641a2070 con 0x7fc264101760 2024-04-25T14:10:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc2537fe640 1 -- 172.21.15.78:0/2054270667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc254009c30 con 0x7fc264101760 2024-04-25T14:10:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc2537fe640 1 -- 172.21.15.78:0/2054270667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254003d10 con 0x7fc264101760 2024-04-25T14:10:38.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.602+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2641a2610 con 0x7fc264101760 2024-04-25T14:10:38.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 Processor -- start 2024-04-25T14:10:38.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc2537fe640 1 -- 172.21.15.78:0/2054270667 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc254003eb0 con 0x7fc264101760 2024-04-25T14:10:38.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 -- start start 2024-04-25T14:10:38.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b85a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b498aeab00 con 0x55b4987bb550 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b498a86920 con 0x55b4987e4410 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26fc17740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b498ae6ac0 con 0x55b4987c3730 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.605+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b85a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b85a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42326/0 (socket says 172.21.15.78:42326) 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 -- 172.21.15.78:0/2894210528 learned_addr learned my addr 172.21.15.78:0/2894210528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc262d9d640 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc2537fe640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc24c0afaf0 0x7fc24c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc261d9b640 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc262d9d640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc24c0afaf0 0x7fc24c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 -- 172.21.15.78:0/2894210528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 msgr2=0x55b498b8c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 -- 172.21.15.78:0/2894210528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 msgr2=0x55b498b854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b854a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 -- 172.21.15.78:0/2894210528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4988c47e0 con 0x55b4987bb550 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc261d9b640 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc2537fe640 1 -- 172.21.15.78:0/2054270667 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc2540de8e0 con 0x7fc264101760 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc26259c640 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b85a20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc2540a6ab0 tx=0x7fc254061b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d379cb71f4538e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc262d9d640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc24c0afaf0 0x7fc24c0b2000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009270 tx=0x7fc25c009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc250ff9640 1 -- 172.21.15.78:0/2894210528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254022300 con 0x55b4987bb550 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.606+0000 7fc250ff9640 1 -- 172.21.15.78:0/2894210528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2540aa020 con 0x55b4987bb550 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc250ff9640 1 -- 172.21.15.78:0/2894210528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254017210 con 0x55b4987bb550 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 -- 172.21.15.78:0/2894210528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 msgr2=0x55b498b85a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b85a20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc2540a6ab0 tx=0x7fc254061b20 comp rx=0 tx=0).stop 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 -- 172.21.15.78:0/2894210528 shutdown_connections 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b85a20 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 --2- 172.21.15.78:0/2894210528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 -- 172.21.15.78:0/2894210528 >> 172.21.15.78:0/2894210528 conn(0x55b4987d8560 msgr2=0x55b498b80e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 -- 172.21.15.78:0/2894210528 shutdown_connections 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.607+0000 7fc26fc17740 1 -- 172.21.15.78:0/2894210528 wait complete. 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 -- start start 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 Processor -- start 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b498b8d310 con 0x55b4987bb550 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b498a845d0 con 0x55b4987e4410 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b498b8d490 con 0x55b4987c3730 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26fc17740 10 client.0 _ll_register_callbacks cb 0x55b498a72080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b8f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42332/0 (socket says 172.21.15.78:42332) 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26259c640 1 -- 172.21.15.78:0/1046417142 learned_addr learned my addr 172.21.15.78:0/1046417142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc262d9d640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc261d9b640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26259c640 1 -- 172.21.15.78:0/1046417142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 msgr2=0x55b498b8f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.608+0000 7fc26259c640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26259c640 1 -- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 msgr2=0x55b498b96700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26259c640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b96700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26259c640 1 -- 172.21.15.78:0/1046417142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b498924f70 con 0x55b4987bb550 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc262d9d640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b96700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26259c640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b8f260 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc2540dc420 tx=0x7fc2540a68c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e582429e7fc21adf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc249ffb640 1 -- 172.21.15.78:0/1046417142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254014040 con 0x55b4987bb550 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc249ffb640 1 -- 172.21.15.78:0/1046417142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc2540a3040 con 0x55b4987bb550 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2540aa020 con 0x55b4987bb550 2024-04-25T14:10:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc249ffb640 1 -- 172.21.15.78:0/1046417142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc254009de0 con 0x55b4987bb550 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 10 client.18816 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b4987bc170 con 0x55b4987bb550 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc249ffb640 1 -- 172.21.15.78:0/1046417142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2540aa020 con 0x55b4987bb550 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc249ffb640 1 -- 172.21.15.78:0/1046417142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc2540041a0 con 0x55b4987bb550 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 10 client.18816 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 10 client.18816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 20 client.18816 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4987a31f0 0x55b498b9c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 4 client.18816 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.609+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b498af8710 con 0x55b4987a31f0 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.610+0000 7fc261d9b640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4987a31f0 0x55b498b9c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.610+0000 7fc249ffb640 10 client.18816 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.610+0000 7fc261d9b640 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4987a31f0 0x55b498b9c7c0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77585718341c793f server_cookie=3d57a0a7128ae924 in_seq=0 out_seq=0 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc249ffb640 1 -- 172.21.15.78:0/1046417142 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b498af8710 con 0x55b4987a31f0 2024-04-25T14:10:38.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc249ffb640 10 client.18816 handle_command_reply: tid=0 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 client.18816 shutdown 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 msgr2=0x55b498b8f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b8f260 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc2540dc420 tx=0x7fc2540a68c0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 shutdown_connections 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498b96700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b4987a31f0 0x55b498b9c7c0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b8f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc249ffb640 0 client.18816 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 --2- 172.21.15.78:0/1046417142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b8f260 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 >> 172.21.15.78:0/1046417142 conn(0x55b4987d8560 msgr2=0x55b498b7fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 shutdown_connections 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 1 -- 172.21.15.78:0/1046417142 wait complete. 2024-04-25T14:10:38.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.611+0000 7fc26fc17740 20 client.18816 trim_cache size 0 max 16384 2024-04-25T14:10:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 Processor -- start 2024-04-25T14:10:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 -- start start 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987bb550 0x55b498b89d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987c3730 0x55b498b8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987e4410 0x55b498b84340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641adcb0 con 0x55b4987e4410 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ab580 con 0x55b4987c3730 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc26fc17740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641a93f0 con 0x55b4987bb550 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc262d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987e4410 0x55b498b84340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc262d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987e4410 0x55b498b84340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42340/0 (socket says 172.21.15.78:42340) 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.629+0000 7fc262d9d640 1 -- 172.21.15.78:0/853057831 learned_addr learned my addr 172.21.15.78:0/853057831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc261d9b640 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987c3730 0x55b498b8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc26259c640 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987bb550 0x55b498b89d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc262d9d640 1 -- 172.21.15.78:0/853057831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987bb550 msgr2=0x55b498b89d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc262d9d640 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987bb550 0x55b498b89d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc262d9d640 1 -- 172.21.15.78:0/853057831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987c3730 msgr2=0x55b498b8e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc262d9d640 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987c3730 0x55b498b8e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc262d9d640 1 -- 172.21.15.78:0/853057831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b498b900c0 con 0x55b4987e4410 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc26259c640 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987bb550 0x55b498b89d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc261d9b640 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987c3730 0x55b498b8e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc262d9d640 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987e4410 0x55b498b84340 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc25c003cc0 tx=0x7fc25c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cf518d86ab5d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc24bfff640 1 -- 172.21.15.78:0/853057831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c01c030 con 0x55b4987e4410 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc24bfff640 1 -- 172.21.15.78:0/853057831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c01d040 con 0x55b4987e4410 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.630+0000 7fc24bfff640 1 -- 172.21.15.78:0/853057831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c005240 con 0x55b4987e4410 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- 172.21.15.78:0/853057831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987e4410 msgr2=0x55b498b84340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987e4410 0x55b498b84340 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc25c003cc0 tx=0x7fc25c0096e0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- 172.21.15.78:0/853057831 shutdown_connections 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987e4410 0x55b498b84340 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987c3730 0x55b498b8e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 --2- 172.21.15.78:0/853057831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987bb550 0x55b498b89d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- 172.21.15.78:0/853057831 >> 172.21.15.78:0/853057831 conn(0x55b4987d8560 msgr2=0x55b498b83c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- 172.21.15.78:0/853057831 shutdown_connections 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- 172.21.15.78:0/853057831 wait complete. 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- start start 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 Processor -- start 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b94d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b95260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498c02b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641a9a40 con 0x55b4987bb550 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641aaeb0 con 0x55b4987e4410 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641aa110 con 0x55b4987c3730 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26fc17740 10 client.0 _ll_register_callbacks cb 0x7fc2540ab220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:38.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b94d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.631+0000 7fc26259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b94d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42350/0 (socket says 172.21.15.78:42350) 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26259c640 1 -- 172.21.15.78:0/3624303819 learned_addr learned my addr 172.21.15.78:0/3624303819 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc262d9d640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498c02b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc261d9b640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b95260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26259c640 1 -- 172.21.15.78:0/3624303819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 msgr2=0x55b498b95260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26259c640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b95260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26259c640 1 -- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 msgr2=0x55b498c02b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26259c640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498c02b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26259c640 1 -- 172.21.15.78:0/3624303819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b498c03060 con 0x55b4987bb550 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc262d9d640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498c02b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26259c640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b94d20 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc2540a4020 tx=0x7fc2540a2c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c181472716de5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc249ffb640 1 -- 172.21.15.78:0/3624303819 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254014040 con 0x55b4987bb550 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc249ffb640 1 -- 172.21.15.78:0/3624303819 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fc2540a3040 con 0x55b4987bb550 2024-04-25T14:10:38.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc25c01d040 con 0x55b4987bb550 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc249ffb640 1 -- 172.21.15.78:0/3624303819 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc2540a7010 con 0x55b4987bb550 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26fc17740 10 client.18822 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b498c03060 con 0x55b4987bb550 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.632+0000 7fc249ffb640 1 -- 172.21.15.78:0/3624303819 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c01d040 con 0x55b4987bb550 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc249ffb640 1 -- 172.21.15.78:0/3624303819 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc2540041a0 con 0x55b4987bb550 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc26fc17740 10 client.18822 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc26fc17740 10 client.18822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc26fc17740 20 client.18822 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc26fc17740 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b498778630 0x55b498c067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc26fc17740 4 client.18822 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b498af2580 con 0x55b498778630 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc261d9b640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b498778630 0x55b498c067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc249ffb640 10 client.18822 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.633+0000 7fc261d9b640 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b498778630 0x55b498c067e0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e27ca73b44d5aa server_cookie=2d46e9e6603141df in_seq=0 out_seq=0 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc249ffb640 1 -- 172.21.15.78:0/3624303819 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b498af2580 con 0x55b498778630 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc249ffb640 10 client.18822 handle_command_reply: tid=0 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 client.18822 shutdown 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 msgr2=0x55b498b94d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b94d20 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc2540a4020 tx=0x7fc2540a2c70 comp rx=0 tx=0).stop 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 shutdown_connections 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b4987e4410 0x55b498c02b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55b498778630 0x55b498c067e0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4987c3730 0x55b498b95260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 --2- 172.21.15.78:0/3624303819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4987bb550 0x55b498b94d20 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.634+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 >> 172.21.15.78:0/3624303819 conn(0x55b4987d8560 msgr2=0x55b498b84660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc249ffb640 0 client.18822 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 shutdown_connections 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc26fc17740 1 -- 172.21.15.78:0/3624303819 wait complete. 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc26fc17740 20 client.18822 trim_cache size 0 max 16384 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c466a4ea-ac16-4c0b-8339-4faa6d696e71", 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-04-25T14:10:38.641 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:10:38.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc24c0afaf0 msgr2=0x7fc24c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc24c0afaf0 0x7fc24c0b2000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009270 tx=0x7fc25c009200 comp rx=0 tx=0).stop 2024-04-25T14:10:38.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 msgr2=0x7fc26419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc26419a2b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc254005fd0 tx=0x7fc25400dde0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 shutdown_connections 2024-04-25T14:10:38.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2641050b0 0x7fc2641a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc264102260 0x7fc26419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fc24c0afaf0 0x7fc24c0b2000 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 --2- 172.21.15.78:0/2054270667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc264101760 0x7fc26419a2b0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 >> 172.21.15.78:0/2054270667 conn(0x7fc2640fd380 msgr2=0x7fc264105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 shutdown_connections 2024-04-25T14:10:38.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.635+0000 7fc268c92640 1 -- 172.21.15.78:0/2054270667 wait complete. 2024-04-25T14:10:38.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:10:38.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/1771984424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 msgr2=0x7f4d9410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/1771984424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d9410ad40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009ca0 tx=0x7f4d8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/1771984424 shutdown_connections 2024-04-25T14:10:38.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/1771984424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d9410ad40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/1771984424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d94103c20 0x7f4d94104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/1771984424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d941029e0 0x7f4d941036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/1771984424 >> 172.21.15.78:0/1771984424 conn(0x7f4d940fdbd0 msgr2=0x7f4d94100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/1771984424 shutdown_connections 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.755+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/1771984424 wait complete. 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 Processor -- start 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 -- start start 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d941029e0 0x7f4d9419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d94103c20 0x7f4d9419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d941a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9410e510 con 0x7f4d94108920 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9410e390 con 0x7f4d94103c20 2024-04-25T14:10:38.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9ad7d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9410e690 con 0x7f4d941029e0 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d941a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d941a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42370/0 (socket says 172.21.15.78:42370) 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 -- 172.21.15.78:0/91538346 learned_addr learned my addr 172.21.15.78:0/91538346 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d93fff640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d94103c20 0x7f4d9419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d98b19640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d941029e0 0x7f4d9419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 -- 172.21.15.78:0/91538346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d941029e0 msgr2=0x7f4d9419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d941029e0 0x7f4d9419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 -- 172.21.15.78:0/91538346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d94103c20 msgr2=0x7f4d9419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d94103c20 0x7f4d9419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d9931a640 1 -- 172.21.15.78:0/91538346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d941a1dd0 con 0x7f4d94108920 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.756+0000 7f4d93fff640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d94103c20 0x7f4d9419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.757+0000 7f4d9931a640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d941a1760 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c005e30 tx=0x7f4d8c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.757+0000 7f4d91ffb640 1 -- 172.21.15.78:0/91538346 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c003bf0 con 0x7f4d94108920 2024-04-25T14:10:38.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.757+0000 7f4d91ffb640 1 -- 172.21.15.78:0/91538346 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d8c003d90 con 0x7f4d94108920 2024-04-25T14:10:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.757+0000 7f4d91ffb640 1 -- 172.21.15.78:0/91538346 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c02fb90 con 0x7f4d94108920 2024-04-25T14:10:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.757+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d941a2060 con 0x7f4d94108920 2024-04-25T14:10:38.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.757+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d941a2600 con 0x7f4d94108920 2024-04-25T14:10:38.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d91ffb640 1 -- 172.21.15.78:0/91538346 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4d8c02fd30 con 0x7f4d94108920 2024-04-25T14:10:38.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 Processor -- start 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 -- start start 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e568a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e19e4d0 0x55a71e568fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1a66b0 0x55a71e569c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a71e4cdd80 con 0x55a71e1c7390 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a71e4878b0 con 0x55a71e1a66b0 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a71e4c9d40 con 0x55a71e19e4d0 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e568a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e568a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42378/0 (socket says 172.21.15.78:42378) 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 -- 172.21.15.78:0/2516145800 learned_addr learned my addr 172.21.15.78:0/2516145800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d98b19640 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e19e4d0 0x55a71e568fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1a66b0 0x55a71e569c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d91ffb640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d840afb70 0x7f4d840b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 -- 172.21.15.78:0/2516145800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e19e4d0 msgr2=0x55a71e568fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e19e4d0 0x55a71e568fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 -- 172.21.15.78:0/2516145800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1a66b0 msgr2=0x55a71e569c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1a66b0 0x55a71e569c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d93fff640 1 -- 172.21.15.78:0/2516145800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71e458a40 con 0x55a71e1c7390 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.760+0000 7f4d9931a640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d840afb70 0x7f4d840b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.761+0000 7f4d93fff640 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e568a60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000b450 tx=0x7f4d8000b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a7ceded95ef1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.761+0000 7f4d91ffb640 1 -- 172.21.15.78:0/91538346 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4d8c1003c0 con 0x7f4d94108920 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.761+0000 7f4d7effd640 1 -- 172.21.15.78:0/2516145800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80004230 con 0x55a71e1c7390 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.761+0000 7f4d7effd640 1 -- 172.21.15.78:0/2516145800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80017070 con 0x55a71e1c7390 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.761+0000 7f4d7effd640 1 -- 172.21.15.78:0/2516145800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8001b6c0 con 0x55a71e1c7390 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 -- 172.21.15.78:0/2516145800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 msgr2=0x55a71e568a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4d9931a640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d840afb70 0x7f4d840b2080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0fe220 tx=0x7f4d8c0f6fe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e568a60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000b450 tx=0x7f4d8000b770 comp rx=0 tx=0).stop 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 -- 172.21.15.78:0/2516145800 shutdown_connections 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1a66b0 0x55a71e569c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e19e4d0 0x55a71e568fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/2516145800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e568a60 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 -- 172.21.15.78:0/2516145800 >> 172.21.15.78:0/2516145800 conn(0x55a71e1bb4e0 msgr2=0x55a71e564270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 -- 172.21.15.78:0/2516145800 shutdown_connections 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 -- 172.21.15.78:0/2516145800 wait complete. 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 -- start start 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.762+0000 7f4da1c2f740 1 Processor -- start 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e572980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e572ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e579e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a71e56a780 con 0x55a71e19e4d0 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a71e454d10 con 0x55a71e1c7390 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a71e56a900 con 0x55a71e1a66b0 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 10 client.0 _ll_register_callbacks cb 0x55a71e481210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e572980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e572980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42382/0 (socket says 172.21.15.78:42382) 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 -- 172.21.15.78:0/783933976 learned_addr learned my addr 172.21.15.78:0/783933976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d98b19640 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e572ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e579e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 -- 172.21.15.78:0/783933976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 msgr2=0x55a71e572ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e572ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 -- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 msgr2=0x55a71e579e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e579e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 -- 172.21.15.78:0/783933976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71e2a97d0 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d93fff640 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e572980 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4d80016040 tx=0x7f4d80015f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e805b3fa008105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d73fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d73fff640 1 -- 172.21.15.78:0/783933976 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800052d0 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d73fff640 1 -- 172.21.15.78:0/783933976 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4d80005470 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d80017070 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4d73fff640 1 -- 172.21.15.78:0/783933976 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4d800035f0 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.763+0000 7f4da1c2f740 10 client.18834 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a71e1df3f0 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4d73fff640 1 -- 172.21.15.78:0/783933976 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80017070 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4d73fff640 1 -- 172.21.15.78:0/783933976 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4d80004b20 con 0x55a71e19e4d0 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4da1c2f740 10 client.18834 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:38.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4da1c2f740 10 client.18834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4da1c2f740 20 client.18834 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e185f60 0x55a71e57ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4da1c2f740 4 client.18834 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a71e4db990 con 0x55a71e185f60 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.764+0000 7f4d98b19640 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e185f60 0x55a71e57ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.765+0000 7f4d73fff640 10 client.18834 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.765+0000 7f4d98b19640 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e185f60 0x55a71e57ff40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd43163a89f96ee server_cookie=caa6065a89418604 in_seq=0 out_seq=0 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.765+0000 7f4d73fff640 1 -- 172.21.15.78:0/783933976 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a71e4db990 con 0x55a71e185f60 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.765+0000 7f4d73fff640 10 client.18834 handle_command_reply: tid=0 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 client.18834 shutdown 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 msgr2=0x55a71e572980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e572980 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4d80016040 tx=0x7f4d80015f70 comp rx=0 tx=0).stop 2024-04-25T14:10:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 shutdown_connections 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e579e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e185f60 0x55a71e57ff40 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e572ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4d73fff640 0 client.18834 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/783933976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e572980 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 >> 172.21.15.78:0/783933976 conn(0x55a71e1bb4e0 msgr2=0x55a71e567180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 shutdown_connections 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 1 -- 172.21.15.78:0/783933976 wait complete. 2024-04-25T14:10:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.766+0000 7f4da1c2f740 20 client.18834 trim_cache size 0 max 16384 2024-04-25T14:10:38.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 Processor -- start 2024-04-25T14:10:38.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 -- start start 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e19e4d0 0x55a71e5715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e579db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e56d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941adc00 con 0x55a71e1c7390 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941ab4d0 con 0x55a71e19e4d0 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941a9320 con 0x55a71e1a66b0 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e56d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e56d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42390/0 (socket says 172.21.15.78:42390) 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 -- 172.21.15.78:0/3730365585 learned_addr learned my addr 172.21.15.78:0/3730365585 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d98b19640 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e579db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e19e4d0 0x55a71e5715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 -- 172.21.15.78:0/3730365585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 msgr2=0x55a71e579db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e579db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 -- 172.21.15.78:0/3730365585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e19e4d0 msgr2=0x55a71e5715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e19e4d0 0x55a71e5715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d93fff640 1 -- 172.21.15.78:0/3730365585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71e19f0f0 con 0x55a71e1c7390 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.784+0000 7f4d98b19640 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e579db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e56d6a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4d800059f0 tx=0x7f4d8001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19c4870d04352a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4d7e7fc640 1 -- 172.21.15.78:0/3730365585 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000b4f0 con 0x55a71e1c7390 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4d7e7fc640 1 -- 172.21.15.78:0/3730365585 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80017070 con 0x55a71e1c7390 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4d7e7fc640 1 -- 172.21.15.78:0/3730365585 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8002cc10 con 0x55a71e1c7390 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3730365585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 msgr2=0x55a71e56d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e56d6a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4d800059f0 tx=0x7f4d8001d040 comp rx=0 tx=0).stop 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3730365585 shutdown_connections 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e1c7390 0x55a71e56d6a0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e579db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3730365585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e19e4d0 0x55a71e5715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3730365585 >> 172.21.15.78:0/3730365585 conn(0x55a71e1bb4e0 msgr2=0x55a71e572550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.785+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3730365585 shutdown_connections 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3730365585 wait complete. 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 -- start start 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 Processor -- start 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e577370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e5778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e5787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941a9990 con 0x55a71e19e4d0 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941aae00 con 0x55a71e1c7390 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941aa060 con 0x55a71e1a66b0 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4da1c2f740 10 client.0 _ll_register_callbacks cb 0x7f4d88009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e577370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e577370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42392/0 (socket says 172.21.15.78:42392) 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.786+0000 7f4d93fff640 1 -- 172.21.15.78:0/3060397799 learned_addr learned my addr 172.21.15.78:0/3060397799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d98b19640 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e5778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e5787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d93fff640 1 -- 172.21.15.78:0/3060397799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 msgr2=0x55a71e5778b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e5778b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d93fff640 1 -- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 msgr2=0x55a71e5787e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e5787e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d93fff640 1 -- 172.21.15.78:0/3060397799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71e48e4b0 con 0x55a71e19e4d0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e577370 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004290 tx=0x7f4d8000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fef973f95a943fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d73fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d73fff640 1 -- 172.21.15.78:0/3060397799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8003c070 con 0x55a71e19e4d0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d73fff640 1 -- 172.21.15.78:0/3060397799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4d8002c9b0 con 0x55a71e19e4d0 2024-04-25T14:10:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d80017070 con 0x55a71e19e4d0 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d73fff640 1 -- 172.21.15.78:0/3060397799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4d8000b4f0 con 0x55a71e19e4d0 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 10 client.18840 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a71e579e60 con 0x55a71e19e4d0 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d73fff640 1 -- 172.21.15.78:0/3060397799 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80017070 con 0x55a71e19e4d0 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4d73fff640 1 -- 172.21.15.78:0/3060397799 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4d8002cbb0 con 0x55a71e19e4d0 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 10 client.18840 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 10 client.18840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 20 client.18840 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e15b3a0 0x55a71e5ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 4 client.18840 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.787+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a71e4d5800 con 0x55a71e15b3a0 2024-04-25T14:10:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.788+0000 7f4d98b19640 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e15b3a0 0x55a71e5ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.788+0000 7f4d98b19640 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e15b3a0 0x55a71e5ea640 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daccf6a12177601c server_cookie=a6760bf6864422e in_seq=0 out_seq=0 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.788+0000 7f4d73fff640 10 client.18840 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.788+0000 7f4d73fff640 1 -- 172.21.15.78:0/3060397799 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a71e4d5800 con 0x55a71e15b3a0 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.788+0000 7f4d73fff640 10 client.18840 handle_command_reply: tid=0 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.788+0000 7f4da1c2f740 1 client.18840 shutdown 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 msgr2=0x55a71e577370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e577370 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004290 tx=0x7f4d8000bcb0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 shutdown_connections 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a71e1c7390 0x55a71e5787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x55a71e15b3a0 0x55a71e5ea640 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a71e1a66b0 0x55a71e5778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 --2- 172.21.15.78:0/3060397799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a71e19e4d0 0x55a71e577370 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d73fff640 0 client.18840 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 >> 172.21.15.78:0/3060397799 conn(0x55a71e1bb4e0 msgr2=0x55a71e5629c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 shutdown_connections 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 1 -- 172.21.15.78:0/3060397799 wait complete. 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4da1c2f740 20 client.18840 trim_cache size 0 max 16384 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: "c466a4ea-ac16-4c0b-8339-4faa6d696e71": { 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c466a4ea-ac16-4c0b-8339-4faa6d696e71", 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:10:38.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:10:38.797 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:10:38.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d840afb70 msgr2=0x7f4d840b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d840afb70 0x7f4d840b2080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0fe220 tx=0x7f4d8c0f6fe0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 msgr2=0x7f4d941a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d941a1760 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c005e30 tx=0x7f4d8c02f380 comp rx=0 tx=0).stop 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 shutdown_connections 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f4d840afb70 0x7f4d840b2080 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94108920 0x7f4d941a1760 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4d94103c20 0x7f4d9419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 --2- 172.21.15.78:0/91538346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d941029e0 0x7f4d9419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 >> 172.21.15.78:0/91538346 conn(0x7f4d940fdbd0 msgr2=0x7f4d94109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.789+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 shutdown_connections 2024-04-25T14:10:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:10:38.790+0000 7f4d9ad7d640 1 -- 172.21.15.78:0/91538346 wait complete. 2024-04-25T14:10:38.800 INFO:tasks.cephfs.filesystem:scrub status for tag:c466a4ea-ac16-4c0b-8339-4faa6d696e71 - {'path': '/', 'tag': 'c466a4ea-ac16-4c0b-8339-4faa6d696e71', 'options': 'recursive,force'} 2024-04-25T14:10:39.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:39 smithi129 ceph-mon[32569]: scrub queued for path: / 2024-04-25T14:10:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:39 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:39 smithi129 ceph-mon[32569]: scrub summary: active paths [/] 2024-04-25T14:10:39.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:39 smithi129 ceph-mon[32569]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:39 smithi078 ceph-mon[29163]: scrub queued for path: / 2024-04-25T14:10:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:39 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:39 smithi078 ceph-mon[29163]: scrub summary: active paths [/] 2024-04-25T14:10:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:39 smithi078 ceph-mon[29163]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:39 smithi112 ceph-mon[32617]: scrub queued for path: / 2024-04-25T14:10:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:39 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:39 smithi112 ceph-mon[32617]: scrub summary: active paths [/] 2024-04-25T14:10:39.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:39 smithi112 ceph-mon[32617]: scrub summary: idle+waiting paths [/] 2024-04-25T14:10:40.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:40 smithi129 ceph-mon[32569]: pgmap v682: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:40 smithi078 ceph-mon[29163]: pgmap v682: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:40.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:40 smithi112 ceph-mon[32617]: pgmap v682: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 -- 172.21.15.78:0/1100474906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30108110 msgr2=0x7f5d3010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 --2- 172.21.15.78:0/1100474906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30108110 0x7f5d3010a530 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009900 tx=0x7f5d2802a410 comp rx=0 tx=0).stop 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 -- 172.21.15.78:0/1100474906 shutdown_connections 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 --2- 172.21.15.78:0/1100474906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30108110 0x7f5d3010a530 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 --2- 172.21.15.78:0/1100474906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d30103410 0x7f5d30103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 --2- 172.21.15.78:0/1100474906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d301021d0 0x7f5d30102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 -- 172.21.15.78:0/1100474906 >> 172.21.15.78:0/1100474906 conn(0x7f5d300fd3c0 msgr2=0x7f5d300ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 -- 172.21.15.78:0/1100474906 shutdown_connections 2024-04-25T14:10:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 -- 172.21.15.78:0/1100474906 wait complete. 2024-04-25T14:10:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.412+0000 7f5d358a8640 1 Processor -- start 2024-04-25T14:10:41.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:41 smithi129 ceph-mon[32569]: scrub summary: idle 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d358a8640 1 -- start start 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d358a8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d301021d0 0x7f5d3019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d358a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30103410 0x7f5d3019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d358a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d30108110 0x7f5d301a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d358a8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3010dcf0 con 0x7f5d30103410 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d358a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3010db70 con 0x7f5d30108110 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d358a8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3010de70 con 0x7f5d301021d0 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30103410 0x7f5d3019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30103410 0x7f5d3019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42420/0 (socket says 172.21.15.78:42420) 2024-04-25T14:10:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3016072635 learned_addr learned my addr 172.21.15.78:0/3016072635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2effd640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d301021d0 0x7f5d3019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d30108110 0x7f5d301a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3016072635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d301021d0 msgr2=0x7f5d3019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d301021d0 0x7f5d3019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3016072635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d30108110 msgr2=0x7f5d301a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d30108110 0x7f5d301a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3016072635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d301a1d30 con 0x7f5d30103410 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2effd640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d301021d0 0x7f5d3019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.413+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d30108110 0x7f5d301a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.414+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30103410 0x7f5d3019a730 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400d770 tx=0x7f5d2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.414+0000 7f5d348a6640 1 -- 172.21.15.78:0/3016072635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24017070 con 0x7f5d30103410 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.414+0000 7f5d348a6640 1 -- 172.21.15.78:0/3016072635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d240054d0 con 0x7f5d30103410 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.414+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d301a1fc0 con 0x7f5d30103410 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.414+0000 7f5d348a6640 1 -- 172.21.15.78:0/3016072635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24004c30 con 0x7f5d30103410 2024-04-25T14:10:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.414+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d301a2460 con 0x7f5d30103410 2024-04-25T14:10:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 Processor -- start 2024-04-25T14:10:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d348a6640 1 -- 172.21.15.78:0/3016072635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5d24004e10 con 0x7f5d30103410 2024-04-25T14:10:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 -- start start 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33559970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33559ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33560df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564b334bf460 con 0x564b331902f0 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564b33479590 con 0x564b331b91b0 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d3c837740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564b334bb420 con 0x564b331984d0 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.416+0000 7f5d2effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33559ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33559ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42434/0 (socket says 172.21.15.78:42434) 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 -- 172.21.15.78:0/36630711 learned_addr learned my addr 172.21.15.78:0/36630711 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33559970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33560df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d348a6640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5d180afae0 0x7f5d180b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 -- 172.21.15.78:0/36630711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 msgr2=0x564b33560df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33560df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 -- 172.21.15.78:0/36630711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 msgr2=0x564b33559970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33559970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 -- 172.21.15.78:0/36630711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b334860d0 con 0x564b331902f0 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5d180afae0 0x7f5d180b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d2effd640 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33559ef0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009fd0 tx=0x7f5d20010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f82368b3d8d88cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d348a6640 1 -- 172.21.15.78:0/3016072635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5d240de7a0 con 0x7f5d30103410 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d15ffb640 1 -- 172.21.15.78:0/36630711 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20016070 con 0x564b331902f0 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d15ffb640 1 -- 172.21.15.78:0/36630711 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d20012bc0 con 0x564b331902f0 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.417+0000 7f5d15ffb640 1 -- 172.21.15.78:0/36630711 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2001bcb0 con 0x564b331902f0 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 -- 172.21.15.78:0/36630711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 msgr2=0x564b33559ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33559ef0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009fd0 tx=0x7f5d20010800 comp rx=0 tx=0).stop 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5d180afae0 0x7f5d180b1ff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 -- 172.21.15.78:0/36630711 shutdown_connections 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33560df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33559ef0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 --2- 172.21.15.78:0/36630711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33559970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 -- 172.21.15.78:0/36630711 >> 172.21.15.78:0/36630711 conn(0x564b331ad300 msgr2=0x564b33555300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 -- 172.21.15.78:0/36630711 shutdown_connections 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 -- 172.21.15.78:0/36630711 wait complete. 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.418+0000 7f5d3c837740 1 -- start start 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 1 Processor -- start 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331902f0 0x564b33563640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33563b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331b91b0 0x564b3356aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564b33561840 con 0x564b331b91b0 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564b33446b60 con 0x564b331902f0 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564b335619c0 con 0x564b331984d0 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d3c837740 10 client.0 _ll_register_callbacks cb 0x564b3346b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d2f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331b91b0 0x564b3356aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d2f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331b91b0 0x564b3356aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42440/0 (socket says 172.21.15.78:42440) 2024-04-25T14:10:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d2f7fe640 1 -- 172.21.15.78:0/1355781444 learned_addr learned my addr 172.21.15.78:0/1355781444 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33563b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.419+0000 7f5d2effd640 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331902f0 0x564b33563640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d2f7fe640 1 -- 172.21.15.78:0/1355781444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 msgr2=0x564b33563b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33563b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d2f7fe640 1 -- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331902f0 msgr2=0x564b33563640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331902f0 0x564b33563640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d2f7fe640 1 -- 172.21.15.78:0/1355781444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b331d1210 con 0x564b331b91b0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331b91b0 0x564b3356aae0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009900 tx=0x7f5d280058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b428d55f79005218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d0effd640 1 -- 172.21.15.78:0/1355781444 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2804c020 con 0x564b331b91b0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d0effd640 1 -- 172.21.15.78:0/1355781444 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5d28004250 con 0x564b331b91b0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d20012bc0 con 0x564b331b91b0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d0effd640 1 -- 172.21.15.78:0/1355781444 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5d28057050 con 0x564b331b91b0 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d3c837740 10 client.18852 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564b3329d620 con 0x564b331b91b0 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d0effd640 1 -- 172.21.15.78:0/1355781444 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d20012bc0 con 0x564b331b91b0 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.420+0000 7f5d0effd640 1 -- 172.21.15.78:0/1355781444 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5d280036f0 con 0x564b331b91b0 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d3c837740 10 client.18852 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d3c837740 10 client.18852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d3c837740 20 client.18852 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d3c837740 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b33177d80 0x564b335713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d3c837740 4 client.18852 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b334cd070 con 0x564b33177d80 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d2effd640 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b33177d80 0x564b335713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d2effd640 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b33177d80 0x564b335713a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e92c4b1b117872 server_cookie=77be7925950ef869 in_seq=0 out_seq=0 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.421+0000 7f5d0effd640 10 client.18852 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d0effd640 1 -- 172.21.15.78:0/1355781444 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b334cd070 con 0x564b33177d80 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d0effd640 10 client.18852 handle_command_reply: tid=0 2024-04-25T14:10:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 client.18852 shutdown 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331b91b0 msgr2=0x564b3356aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331b91b0 0x564b3356aae0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009900 tx=0x7f5d280058d0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 shutdown_connections 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331b91b0 0x564b3356aae0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b33177d80 0x564b335713a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331984d0 0x564b33563b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 --2- 172.21.15.78:0/1355781444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331902f0 0x564b33563640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d0effd640 0 client.18852 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 >> 172.21.15.78:0/1355781444 conn(0x564b331ad300 msgr2=0x564b33558210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 shutdown_connections 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 1 -- 172.21.15.78:0/1355781444 wait complete. 2024-04-25T14:10:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.422+0000 7f5d3c837740 20 client.18852 trim_cache size 0 max 16384 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.437+0000 7f5d3c837740 1 Processor -- start 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.437+0000 7f5d3c837740 1 -- start start 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.437+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331902f0 0x564b33554cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.437+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331984d0 0x564b3356aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33555200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d3c837740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301adb20 con 0x564b331984d0 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d3c837740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301ab3f0 con 0x564b331b91b0 2024-04-25T14:10:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d3c837740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a9240 con 0x564b331902f0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331984d0 0x564b3356aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331984d0 0x564b3356aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42452/0 (socket says 172.21.15.78:42452) 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3141882505 learned_addr learned my addr 172.21.15.78:0/3141882505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2effd640 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331902f0 0x564b33554cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3141882505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331902f0 msgr2=0x564b33554cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331902f0 0x564b33554cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33555200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3141882505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 msgr2=0x564b33555200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33555200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 -- 172.21.15.78:0/3141882505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3356af40 con 0x564b331984d0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2effd640 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331902f0 0x564b33554cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331984d0 0x564b3356aa00 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5d240a2cf0 tx=0x7f5d24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1aea2d1f9cfee46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d157fa640 1 -- 172.21.15.78:0/3141882505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24020040 con 0x564b331984d0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d157fa640 1 -- 172.21.15.78:0/3141882505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d240a6c70 con 0x564b331984d0 2024-04-25T14:10:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.438+0000 7f5d157fa640 1 -- 172.21.15.78:0/3141882505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d240a6ed0 con 0x564b331984d0 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 -- 172.21.15.78:0/3141882505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331984d0 msgr2=0x564b3356aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331984d0 0x564b3356aa00 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5d240a2cf0 tx=0x7f5d24009f90 comp rx=0 tx=0).stop 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 -- 172.21.15.78:0/3141882505 shutdown_connections 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331b91b0 0x564b33555200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331984d0 0x564b3356aa00 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 --2- 172.21.15.78:0/3141882505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331902f0 0x564b33554cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 -- 172.21.15.78:0/3141882505 >> 172.21.15.78:0/3141882505 conn(0x564b331ad300 msgr2=0x564b3355da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 -- 172.21.15.78:0/3141882505 shutdown_connections 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 -- 172.21.15.78:0/3141882505 wait complete. 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.439+0000 7f5d3c837740 1 -- start start 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 1 Processor -- start 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33567c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331984d0 0x564b335681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331b91b0 0x564b335690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a98b0 con 0x564b331902f0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301aad20 con 0x564b331984d0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a9f80 con 0x564b331b91b0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d3c837740 10 client.0 _ll_register_callbacks cb 0x7f5d28031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33567c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33567c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42458/0 (socket says 172.21.15.78:42458) 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 -- 172.21.15.78:0/4151966091 learned_addr learned my addr 172.21.15.78:0/4151966091 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331b91b0 0x564b335690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 -- 172.21.15.78:0/4151966091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331b91b0 msgr2=0x564b335690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331b91b0 0x564b335690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331984d0 0x564b335681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 -- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331984d0 msgr2=0x564b335681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331984d0 0x564b335681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 -- 172.21.15.78:0/4151966091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b335696a0 con 0x564b331902f0 2024-04-25T14:10:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2f7fe640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331b91b0 0x564b335690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d2effd640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33567c70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x564b3355ad50 tx=0x7f5d20009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd0ebee67219b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.440+0000 7f5d0effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d0effd640 1 -- 172.21.15.78:0/4151966091 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2002d030 con 0x564b331902f0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d0effd640 1 -- 172.21.15.78:0/4151966091 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f5d20004c10 con 0x564b331902f0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d240a6c70 con 0x564b331902f0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d0effd640 1 -- 172.21.15.78:0/4151966091 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5d20016040 con 0x564b331902f0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 10 client.18858 fetch_fsmap learned FSMap version 22 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564b3356a9d0 con 0x564b331902f0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d0effd640 1 -- 172.21.15.78:0/4151966091 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d240a6c70 con 0x564b331902f0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d0effd640 1 -- 172.21.15.78:0/4151966091 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5d2000f040 con 0x564b331902f0 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 10 client.18858 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 10 client.18858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 20 client.18858 populate_metadata read hostname 'smithi078' 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b3314d1c0 0x564b335db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:10:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 4 client.18858 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T14:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b334c6ee0 con 0x564b3314d1c0 2024-04-25T14:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.441+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b3314d1c0 0x564b335db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.442+0000 7f5d0effd640 10 client.18858 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.442+0000 7f5d2e7fc640 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b3314d1c0 0x564b335db900 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b891592ebaa99f19 server_cookie=7c8d143832251a6 in_seq=0 out_seq=0 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d0effd640 1 -- 172.21.15.78:0/4151966091 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b334c6ee0 con 0x564b3314d1c0 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d0effd640 10 client.18858 handle_command_reply: tid=0 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 client.18858 shutdown 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 msgr2=0x564b33567c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33567c70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x564b3355ad50 tx=0x7f5d20009d00 comp rx=0 tx=0).stop 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 shutdown_connections 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x564b3314d1c0 0x564b335db900 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b331b91b0 0x564b335690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564b331984d0 0x564b335681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d0effd640 0 client.18858 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 --2- 172.21.15.78:0/4151966091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b331902f0 0x564b33567c70 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 >> 172.21.15.78:0/4151966091 conn(0x564b331ad300 msgr2=0x564b33558100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 shutdown_connections 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 1 -- 172.21.15.78:0/4151966091 wait complete. 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d3c837740 20 client.18858 trim_cache size 0 max 16384 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-04-25T14:10:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5d180afae0 msgr2=0x7f5d180b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d358a8640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5d180afae0 0x7f5d180b1ff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d28009260 comp rx=0 tx=0).stop 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30103410 msgr2=0x7f5d3019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.456+0000 7f5d358a8640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30103410 0x7f5d3019a730 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400d770 tx=0x7f5d2400dc30 comp rx=0 tx=0).stop 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 shutdown_connections 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d30108110 0x7f5d301a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f5d180afae0 0x7f5d180b1ff0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d30103410 0x7f5d3019a730 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 --2- 172.21.15.78:0/3016072635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d301021d0 0x7f5d3019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:10:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 >> 172.21.15.78:0/3016072635 conn(0x7f5d300fd3c0 msgr2=0x7f5d3006aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:10:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 shutdown_connections 2024-04-25T14:10:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-04-25T14:10:41.457+0000 7f5d358a8640 1 -- 172.21.15.78:0/3016072635 wait complete. 2024-04-25T14:10:41.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:41 smithi078 ceph-mon[29163]: scrub summary: idle 2024-04-25T14:10:41.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:41 smithi112 ceph-mon[32617]: scrub summary: idle 2024-04-25T14:10:42.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:42 smithi129 ceph-mon[32569]: pgmap v683: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:42 smithi078 ceph-mon[29163]: pgmap v683: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:42.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:42 smithi112 ceph-mon[32617]: pgmap v683: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:44.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:44 smithi129 ceph-mon[32569]: pgmap v684: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:44 smithi078 ceph-mon[29163]: pgmap v684: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:44.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:44 smithi112 ceph-mon[32617]: pgmap v684: 97 pgs: 97 active+clean; 3.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:45.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:45 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:45 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:45.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:45 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:10:46.434 INFO:tasks.workunit.client.0.smithi078.stdout:561/600: open fd = 3 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:562/600: open fd = 3 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:563/600: open fd = 3 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:564/600: open fd = 3 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.435 INFO:tasks.workunit.client.0.smithi078.stdout:565/600: open fd = 3 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:566/600: open fd = 3 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:567/600: open fd = 3 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:568/600: open fd = 3 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.436 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:569/600: open fd = 3 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:570/600: open fd = 3 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:571/600: open fd = 3 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:572/600: open fd = 3 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.437 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:573/600: open fd = 3 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:574/600: open fd = 3 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:575/600: open fd = 3 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.438 INFO:tasks.workunit.client.0.smithi078.stdout:576/600: open fd = 3 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:577/600: open fd = 3 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:578/600: open fd = 3 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:579/600: open fd = 3 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.439 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:580/600: open fd = 3 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:581/600: open fd = 3 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:582/600: open fd = 3 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:583/600: open fd = 3 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.440 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:584/600: open fd = 3 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:585/600: open fd = 3 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:586/600: open fd = 3 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:587/600: open fd = 3 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.441 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:588/600: open fd = 3 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:589/600: open fd = 3 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:590/600: open fd = 3 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.442 INFO:tasks.workunit.client.0.smithi078.stdout:591/600: open fd = 3 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:592/600: open fd = 3 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:593/600: open fd = 3 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:594/600: open fd = 3 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.443 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:595/600: open fd = 3 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:596/600: open fd = 3 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:597/600: open fd = 3 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:598/600: open fd = 3 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.444 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:599/600: open fd = 3 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:600/600: open fd = 3 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:write ret = 32 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stdout:pread ret = 64 2024-04-25T14:10:46.445 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo OK 2024-04-25T14:10:46.446 INFO:tasks.workunit.client.0.smithi078.stdout:OK 2024-04-25T14:10:46.446 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T14:10:46.446 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T14:10:46.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: pgmap v685: 97 pgs: 97 active+clean; 4.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:10:46.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.466 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.467 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:46.467 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:46.467 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.479 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-25T14:10:46.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T14:10:46.515 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: pgmap v685: 97 pgs: 97 active+clean; 4.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:10:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:46.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: pgmap v685: 97 pgs: 97 active+clean; 4.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-25T14:10:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:10:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:10:46.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:10:46.785 DEBUG:teuthology.parallel:result is None 2024-04-25T14:10:46.785 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T14:10:46.817 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T14:10:46.817 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T14:10:46.827 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T14:10:46.864 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-25T14:10:46.874 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-25T14:10:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:48 smithi078 ceph-mon[29163]: pgmap v686: 97 pgs: 97 active+clean; 4.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:48.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:48 smithi112 ceph-mon[32617]: pgmap v686: 97 pgs: 97 active+clean; 4.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:48 smithi129 ceph-mon[32569]: pgmap v686: 97 pgs: 97 active+clean; 4.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T14:10:50.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:50 smithi112 ceph-mon[32617]: pgmap v687: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:10:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:50 smithi129 ceph-mon[32569]: pgmap v687: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:10:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:50 smithi078 ceph-mon[29163]: pgmap v687: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-25T14:10:52.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:52 smithi112 ceph-mon[32617]: pgmap v688: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:52.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:52 smithi129 ceph-mon[32569]: pgmap v688: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:52 smithi078 ceph-mon[29163]: pgmap v688: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:54.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:54 smithi112 ceph-mon[32617]: pgmap v689: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:54.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:54 smithi129 ceph-mon[32569]: pgmap v689: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:54 smithi078 ceph-mon[29163]: pgmap v689: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-25T14:10:56.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:56 smithi112 ceph-mon[32617]: pgmap v690: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:56.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:56 smithi129 ceph-mon[32569]: pgmap v690: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:56.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:56 smithi078 ceph-mon[29163]: pgmap v690: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-25T14:10:58.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:10:58 smithi112 ceph-mon[32617]: pgmap v691: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-25T14:10:58.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:10:58 smithi129 ceph-mon[32569]: pgmap v691: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-25T14:10:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:10:58 smithi078 ceph-mon[29163]: pgmap v691: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-25T14:11:00.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:00 smithi112 ceph-mon[32617]: pgmap v692: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-25T14:11:00.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:00 smithi129 ceph-mon[32569]: pgmap v692: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-25T14:11:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:00 smithi078 ceph-mon[29163]: pgmap v692: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-04-25T14:11:02.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:02 smithi112 ceph-mon[32617]: pgmap v693: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-04-25T14:11:02.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:02 smithi078 ceph-mon[29163]: pgmap v693: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-04-25T14:11:02.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:02 smithi129 ceph-mon[32569]: pgmap v693: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-04-25T14:11:04.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:04 smithi112 ceph-mon[32617]: pgmap v694: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T14:11:04.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:04 smithi129 ceph-mon[32569]: pgmap v694: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T14:11:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:04 smithi078 ceph-mon[29163]: pgmap v694: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T14:11:06.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:06 smithi112 ceph-mon[32617]: pgmap v695: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-25T14:11:06.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:06 smithi129 ceph-mon[32569]: pgmap v695: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-25T14:11:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:06 smithi078 ceph-mon[29163]: pgmap v695: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-25T14:11:08.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:08 smithi112 ceph-mon[32617]: pgmap v696: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T14:11:08.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:08 smithi129 ceph-mon[32569]: pgmap v696: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T14:11:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:08 smithi078 ceph-mon[29163]: pgmap v696: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T14:11:08.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T14:11:08.802 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 -- 172.21.15.78:0/1796300337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 msgr2=0x7f2e8c10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/1796300337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c10ac10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009ca0 tx=0x7f2e8402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 -- 172.21.15.78:0/1796300337 shutdown_connections 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/1796300337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c10ac10 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/1796300337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e8c103af0 0x7f2e8c103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/1796300337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e8c1028b0 0x7f2e8c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 -- 172.21.15.78:0/1796300337 >> 172.21.15.78:0/1796300337 conn(0x7f2e8c0fdaa0 msgr2=0x7f2e8c0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 -- 172.21.15.78:0/1796300337 shutdown_connections 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 -- 172.21.15.78:0/1796300337 wait complete. 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 Processor -- start 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.925+0000 7f2e91f9d640 1 -- start start 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e91f9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e8c1028b0 0x7f2e8c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e91f9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e8c103af0 0x7f2e8c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e91f9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e91f9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10e350 con 0x7f2e8c1087f0 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e91f9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10e1d0 con 0x7f2e8c1028b0 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e91f9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10e4d0 con 0x7f2e8c103af0 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c1a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40202/0 (socket says 172.21.15.78:40202) 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 -- 172.21.15.78:0/2674903429 learned_addr learned my addr 172.21.15.78:0/2674903429 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e8c1028b0 0x7f2e8c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e8c103af0 0x7f2e8c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 -- 172.21.15.78:0/2674903429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e8c103af0 msgr2=0x7f2e8c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e8c103af0 0x7f2e8c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 -- 172.21.15.78:0/2674903429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e8c1028b0 msgr2=0x7f2e8c19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e8c1028b0 0x7f2e8c19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 -- 172.21.15.78:0/2674903429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c1a1ad0 con 0x7f2e8c1087f0 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e8c1028b0 0x7f2e8c19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e8c103af0 0x7f2e8c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e9179c640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c1a1510 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f2e84006010 tx=0x7f2e8402f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e89ffb640 1 -- 172.21.15.78:0/2674903429 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84003c70 con 0x7f2e8c1087f0 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e89ffb640 1 -- 172.21.15.78:0/2674903429 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e84003e10 con 0x7f2e8c1087f0 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c1a1d60 con 0x7f2e8c1087f0 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.926+0000 7f2e89ffb640 1 -- 172.21.15.78:0/2674903429 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8402fcc0 con 0x7f2e8c1087f0 2024-04-25T14:11:08.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.927+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c1a21f0 con 0x7f2e8c1087f0 2024-04-25T14:11:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.929+0000 7f2e89ffb640 1 -- 172.21.15.78:0/2674903429 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2e84004040 con 0x7f2e8c1087f0 2024-04-25T14:11:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.929+0000 7f2e9a18e740 1 Processor -- start 2024-04-25T14:11:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.929+0000 7f2e9a18e740 1 -- start start 2024-04-25T14:11:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daaaa390 0x5609dae4bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daa814d0 0x5609dae4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa896b0 0x5609dae4ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609dadb0d80 con 0x5609daa896b0 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609dad6a8b0 con 0x5609daa814d0 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5609dadacd40 con 0x5609daaaa390 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa896b0 0x5609dae4ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa896b0 0x5609dae4ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40218/0 (socket says 172.21.15.78:40218) 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 -- 172.21.15.78:0/538311066 learned_addr learned my addr 172.21.15.78:0/538311066 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daa814d0 0x5609dae4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daaaa390 0x5609dae4bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 -- 172.21.15.78:0/538311066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daaaa390 msgr2=0x5609dae4bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daaaa390 0x5609dae4bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 -- 172.21.15.78:0/538311066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daa814d0 msgr2=0x5609dae4c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daa814d0 0x5609dae4c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 -- 172.21.15.78:0/538311066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dad3ba40 con 0x5609daa896b0 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daa814d0 0x5609dae4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:08.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e89ffb640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2e740afae0 0x7f2e740b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa896b0 0x5609dae4ce60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000de90 tx=0x7f2e8000d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c03f09f1ffd85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2e740afae0 0x7f2e740b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e89ffb640 1 -- 172.21.15.78:0/2674903429 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2e84100470 con 0x7f2e8c1087f0 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e72ffd640 1 -- 172.21.15.78:0/538311066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80017070 con 0x5609daa896b0 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e72ffd640 1 -- 172.21.15.78:0/538311066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e800047a0 con 0x5609daa896b0 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.930+0000 7f2e72ffd640 1 -- 172.21.15.78:0/538311066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800127e0 con 0x5609daa896b0 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 -- 172.21.15.78:0/538311066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa896b0 msgr2=0x5609dae4ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2e740afae0 0x7f2e740b1ff0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0058c0 tx=0x7f2e7c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa896b0 0x5609dae4ce60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000de90 tx=0x7f2e8000d5e0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 -- 172.21.15.78:0/538311066 shutdown_connections 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa896b0 0x5609dae4ce60 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daa814d0 0x5609dae4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/538311066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daaaa390 0x5609dae4bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 -- 172.21.15.78:0/538311066 >> 172.21.15.78:0/538311066 conn(0x5609daa9e4e0 msgr2=0x5609dae47460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 -- 172.21.15.78:0/538311066 shutdown_connections 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.931+0000 7f2e9a18e740 1 -- 172.21.15.78:0/538311066 wait complete. 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 -- start start 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 Processor -- start 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae55be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae56120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609dae4d970 con 0x5609daa814d0 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609dad37d10 con 0x5609daaaa390 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5609dae4daf0 con 0x5609daa896b0 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e9a18e740 10 client.0 _ll_register_callbacks cb 0x5609dad64210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e90f9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae55be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e90f9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae55be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40220/0 (socket says 172.21.15.78:40220) 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e90f9b640 1 -- 172.21.15.78:0/2520174120 learned_addr learned my addr 172.21.15.78:0/2520174120 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.932+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae56120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e9179c640 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e90f9b640 1 -- 172.21.15.78:0/2520174120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 msgr2=0x5609dae56120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae56120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e90f9b640 1 -- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 msgr2=0x5609dae5d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e90f9b640 1 -- 172.21.15.78:0/2520174120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dab8c7d0 con 0x5609daa814d0 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae55be0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0126d0 tx=0x7f2e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796e4c3b0ae5fedb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e6bfff640 1 -- 172.21.15.78:0/2520174120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c005220 con 0x5609daa814d0 2024-04-25T14:11:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e6bfff640 1 -- 172.21.15.78:0/2520174120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2e7c0053c0 con 0x5609daa814d0 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e800047a0 con 0x5609daa814d0 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e6bfff640 1 -- 172.21.15.78:0/2520174120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c005600 con 0x5609daa814d0 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e9a18e740 10 client.18870 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5609daac23f0 con 0x5609daa814d0 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e6bfff640 1 -- 172.21.15.78:0/2520174120 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e800047a0 con 0x5609daa814d0 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.933+0000 7f2e6bfff640 1 -- 172.21.15.78:0/2520174120 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2e7c01fc20 con 0x5609daa814d0 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e9a18e740 10 client.18870 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e9a18e740 10 client.18870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e9a18e740 20 client.18870 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa68f60 0x5609dae63620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e9a18e740 4 client.18870 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:11:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609dadbe990 con 0x5609daa68f60 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa68f60 0x5609dae63620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa68f60 0x5609dae63620 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaaa2cd66a267c59 server_cookie=2350db34158ba08e in_seq=0 out_seq=0 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.934+0000 7f2e6bfff640 10 client.18870 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e6bfff640 1 -- 172.21.15.78:0/2520174120 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609dadbe990 con 0x5609daa68f60 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e6bfff640 10 client.18870 handle_command_reply: tid=0 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 client.18870 shutdown 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 msgr2=0x5609dae55be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae55be0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0126d0 tx=0x7f2e7c0046c0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 shutdown_connections 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa68f60 0x5609dae63620 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae56120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/2520174120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae55be0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e6bfff640 0 client.18870 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.935+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 >> 172.21.15.78:0/2520174120 conn(0x5609daa9e4e0 msgr2=0x5609dae4a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.936+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 shutdown_connections 2024-04-25T14:11:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.936+0000 7f2e9a18e740 1 -- 172.21.15.78:0/2520174120 wait complete. 2024-04-25T14:11:08.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.936+0000 7f2e9a18e740 20 client.18870 trim_cache size 0 max 16384 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 Processor -- start 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 -- start start 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae47500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae4a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1ad8e0 con 0x5609daa814d0 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1ab1b0 con 0x5609daaaa390 2024-04-25T14:11:08.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1a9000 con 0x5609daa896b0 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e90f9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae47500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e90f9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae47500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40234/0 (socket says 172.21.15.78:40234) 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.953+0000 7f2e90f9b640 1 -- 172.21.15.78:0/1328274028 learned_addr learned my addr 172.21.15.78:0/1328274028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e90f9b640 1 -- 172.21.15.78:0/1328274028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 msgr2=0x5609dae4a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/1328274028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae4a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e90f9b640 1 -- 172.21.15.78:0/1328274028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 msgr2=0x5609dae5d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/1328274028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e90f9b640 1 -- 172.21.15.78:0/1328274028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609daa820f0 con 0x5609daa814d0 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/1328274028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae47500 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c021040 tx=0x7f2e7c012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91f906b8a0a82d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e727fc640 1 -- 172.21.15.78:0/1328274028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c01f8f0 con 0x5609daa814d0 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e727fc640 1 -- 172.21.15.78:0/1328274028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e7c00f7d0 con 0x5609daa814d0 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.954+0000 7f2e727fc640 1 -- 172.21.15.78:0/1328274028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c032870 con 0x5609daa814d0 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 -- 172.21.15.78:0/1328274028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 msgr2=0x5609dae47500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/1328274028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae47500 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c021040 tx=0x7f2e7c012f60 comp rx=0 tx=0).stop 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 -- 172.21.15.78:0/1328274028 shutdown_connections 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/1328274028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/1328274028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae4a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/1328274028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae47500 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 -- 172.21.15.78:0/1328274028 >> 172.21.15.78:0/1328274028 conn(0x5609daa9e4e0 msgr2=0x5609dae4a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 -- 172.21.15.78:0/1328274028 shutdown_connections 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 -- 172.21.15.78:0/1328274028 wait complete. 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 -- start start 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 Processor -- start 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae5ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae5b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.955+0000 7f2e9a18e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1a9670 con 0x5609daa814d0 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1aaae0 con 0x5609daaaa390 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9a18e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1a9d40 con 0x5609daa896b0 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9a18e740 10 client.0 _ll_register_callbacks cb 0x7f2e7c00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae5ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae5ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40236/0 (socket says 172.21.15.78:40236) 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 -- 172.21.15.78:0/669499674 learned_addr learned my addr 172.21.15.78:0/669499674 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9179c640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae5b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 -- 172.21.15.78:0/669499674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 msgr2=0x5609dae5b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae5b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 -- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 msgr2=0x5609dae5c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 -- 172.21.15.78:0/669499674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dad714b0 con 0x5609daa814d0 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9179c640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae5b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e90f9b640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae5ac00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c01f930 tx=0x7f2e7c01f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735a75cea27077c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e6bfff640 1 -- 172.21.15.78:0/669499674 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c004220 con 0x5609daa814d0 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e6bfff640 1 -- 172.21.15.78:0/669499674 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f2e7c0043c0 con 0x5609daa814d0 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c00f7d0 con 0x5609daa814d0 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e6bfff640 1 -- 172.21.15.78:0/669499674 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c01fd90 con 0x5609daa814d0 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9a18e740 10 client.18876 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:08.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.956+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5609dae5d6f0 con 0x5609daa814d0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e6bfff640 1 -- 172.21.15.78:0/669499674 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e7c00f7d0 con 0x5609daa814d0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e6bfff640 1 -- 172.21.15.78:0/669499674 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2e7c04d050 con 0x5609daa814d0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e9a18e740 10 client.18876 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e9a18e740 10 client.18876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e9a18e740 20 client.18876 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa3e3a0 0x5609daecdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e9a18e740 4 client.18876 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609dadb8800 con 0x5609daa3e3a0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.957+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa3e3a0 0x5609daecdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e6bfff640 10 client.18876 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e8bfff640 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa3e3a0 0x5609daecdcd0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f8344ede84957b server_cookie=b063bd9986dd1cfe in_seq=0 out_seq=0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e6bfff640 1 -- 172.21.15.78:0/669499674 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5609dadb8800 con 0x5609daa3e3a0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e6bfff640 10 client.18876 handle_command_reply: tid=0 2024-04-25T14:11:08.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 client.18876 shutdown 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 msgr2=0x5609dae5ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae5ac00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c01f930 tx=0x7f2e7c01f960 comp rx=0 tx=0).stop 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 shutdown_connections 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5609daaaa390 0x5609dae5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x5609daa3e3a0 0x5609daecdcd0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609daa896b0 0x5609dae5b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e6bfff640 0 client.18876 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 --2- 172.21.15.78:0/669499674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609daa814d0 0x5609dae5ac00 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 >> 172.21.15.78:0/669499674 conn(0x5609daa9e4e0 msgr2=0x5609dae4ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 shutdown_connections 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 1 -- 172.21.15.78:0/669499674 wait complete. 2024-04-25T14:11:08.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.958+0000 7f2e9a18e740 20 client.18876 trim_cache size 0 max 16384 2024-04-25T14:11:08.966 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:11:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-04-25T14:11:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-04-25T14:11:08.966 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:11:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2e740afae0 msgr2=0x7f2e740b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2e740afae0 0x7f2e740b1ff0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0058c0 tx=0x7f2e7c005850 comp rx=0 tx=0).stop 2024-04-25T14:11:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 msgr2=0x7f2e8c1a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c1a1510 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f2e84006010 tx=0x7f2e8402f3e0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 shutdown_connections 2024-04-25T14:11:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f2e740afae0 0x7f2e740b1ff0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e8c1087f0 0x7f2e8c1a1510 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e8c103af0 0x7f2e8c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 --2- 172.21.15.78:0/2674903429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e8c1028b0 0x7f2e8c19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 >> 172.21.15.78:0/2674903429 conn(0x7f2e8c0fdaa0 msgr2=0x7f2e8c109cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 shutdown_connections 2024-04-25T14:11:08.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:08.959+0000 7f2e91f9d640 1 -- 172.21.15.78:0/2674903429 wait complete. 2024-04-25T14:11:08.970 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T14:11:08.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 -- 172.21.15.78:0/2040067273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c103c20 msgr2=0x7f7d6c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 --2- 172.21.15.78:0/2040067273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c103c20 0x7f7d6c104030 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d6002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:09.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 -- 172.21.15.78:0/2040067273 shutdown_connections 2024-04-25T14:11:09.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 --2- 172.21.15.78:0/2040067273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6c108920 0x7f7d6c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 --2- 172.21.15.78:0/2040067273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c103c20 0x7f7d6c104030 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 --2- 172.21.15.78:0/2040067273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6c1029e0 0x7f7d6c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 -- 172.21.15.78:0/2040067273 >> 172.21.15.78:0/2040067273 conn(0x7f7d6c0fdbf0 msgr2=0x7f7d6c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 -- 172.21.15.78:0/2040067273 shutdown_connections 2024-04-25T14:11:09.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.077+0000 7f7d72396640 1 -- 172.21.15.78:0/2040067273 wait complete. 2024-04-25T14:11:09.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 Processor -- start 2024-04-25T14:11:09.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 -- start start 2024-04-25T14:11:09.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6c1029e0 0x7f7d6c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6c103c20 0x7f7d6c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c108920 0x7f7d6c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c10f670 con 0x7f7d6c108920 2024-04-25T14:11:09.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c10f4f0 con 0x7f7d6c1029e0 2024-04-25T14:11:09.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d72396640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c10f7f0 con 0x7f7d6c103c20 2024-04-25T14:11:09.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d70933640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c108920 0x7f7d6c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d70933640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c108920 0x7f7d6c1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40258/0 (socket says 172.21.15.78:40258) 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.078+0000 7f7d70933640 1 -- 172.21.15.78:0/1820833883 learned_addr learned my addr 172.21.15.78:0/1820833883 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d6b7fe640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6c103c20 0x7f7d6c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d6bfff640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6c1029e0 0x7f7d6c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d70933640 1 -- 172.21.15.78:0/1820833883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6c103c20 msgr2=0x7f7d6c19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d70933640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6c103c20 0x7f7d6c19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d70933640 1 -- 172.21.15.78:0/1820833883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6c1029e0 msgr2=0x7f7d6c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d70933640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6c1029e0 0x7f7d6c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d70933640 1 -- 172.21.15.78:0/1820833883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c1a1df0 con 0x7f7d6c108920 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d6bfff640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6c1029e0 0x7f7d6c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d70933640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c108920 0x7f7d6c1a1780 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400eaa0 tx=0x7f7d6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d697fa640 1 -- 172.21.15.78:0/1820833883 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400cdb0 con 0x7f7d6c108920 2024-04-25T14:11:09.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c1a20e0 con 0x7f7d6c108920 2024-04-25T14:11:09.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d697fa640 1 -- 172.21.15.78:0/1820833883 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d64005020 con 0x7f7d6c108920 2024-04-25T14:11:09.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d697fa640 1 -- 172.21.15.78:0/1820833883 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d640116d0 con 0x7f7d6c108920 2024-04-25T14:11:09.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.079+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c1a2600 con 0x7f7d6c108920 2024-04-25T14:11:09.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.080+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c107730 con 0x7f7d6c108920 2024-04-25T14:11:09.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.082+0000 7f7d697fa640 1 -- 172.21.15.78:0/1820833883 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7d640051d0 con 0x7f7d6c108920 2024-04-25T14:11:09.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.083+0000 7f7d697fa640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d540afaf0 0x7f7d540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.083+0000 7f7d6bfff640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d540afaf0 0x7f7d540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.083+0000 7f7d697fa640 1 -- 172.21.15.78:0/1820833883 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7d640df1c0 con 0x7f7d6c108920 2024-04-25T14:11:09.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.084+0000 7f7d6bfff640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d540afaf0 0x7f7d540b2000 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0058c0 tx=0x7f7d5c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.087+0000 7f7d697fa640 1 -- 172.21.15.78:0/1820833883 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d64016030 con 0x7f7d6c108920 2024-04-25T14:11:09.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.304+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6c02db50 con 0x7f7d6c108920 2024-04-25T14:11:09.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.305+0000 7f7d697fa640 1 -- 172.21.15.78:0/1820833883 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f7d640a1e20 con 0x7f7d6c108920 2024-04-25T14:11:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:11:09.305 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:11:09.305 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:11:09.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d540afaf0 msgr2=0x7f7d540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d540afaf0 0x7f7d540b2000 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0058c0 tx=0x7f7d5c005ec0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c108920 msgr2=0x7f7d6c1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c108920 0x7f7d6c1a1780 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400eaa0 tx=0x7f7d6400ef60 comp rx=0 tx=0).stop 2024-04-25T14:11:09.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 shutdown_connections 2024-04-25T14:11:09.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f7d540afaf0 0x7f7d540b2000 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d6c108920 0x7f7d6c1a1780 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d6c103c20 0x7f7d6c19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 --2- 172.21.15.78:0/1820833883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d6c1029e0 0x7f7d6c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 >> 172.21.15.78:0/1820833883 conn(0x7f7d6c0fdbf0 msgr2=0x7f7d6c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 shutdown_connections 2024-04-25T14:11:09.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.307+0000 7f7d72396640 1 -- 172.21.15.78:0/1820833883 wait complete. 2024-04-25T14:11:09.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 -- 172.21.15.78:0/631027689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4634103c30 msgr2=0x7f4634104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 --2- 172.21.15.78:0/631027689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4634103c30 0x7f4634104040 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f4620009900 tx=0x7f462002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 -- 172.21.15.78:0/631027689 shutdown_connections 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 --2- 172.21.15.78:0/631027689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4634108930 0x7f463410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 --2- 172.21.15.78:0/631027689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4634103c30 0x7f4634104040 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 --2- 172.21.15.78:0/631027689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46341029f0 0x7f46341036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 -- 172.21.15.78:0/631027689 >> 172.21.15.78:0/631027689 conn(0x7f46340fdbc0 msgr2=0x7f4634100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 -- 172.21.15.78:0/631027689 shutdown_connections 2024-04-25T14:11:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.429+0000 7f463ac15640 1 -- 172.21.15.78:0/631027689 wait complete. 2024-04-25T14:11:09.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 Processor -- start 2024-04-25T14:11:09.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 -- start start 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46341029f0 0x7f463419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4634103c30 0x7f463419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4634108930 0x7f46341a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f463410e510 con 0x7f46341029f0 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f463410e390 con 0x7f4634103c30 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f463ac15640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f463410e690 con 0x7f4634108930 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46341029f0 0x7f463419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46341029f0 0x7f463419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40276/0 (socket says 172.21.15.78:40276) 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 -- 172.21.15.78:0/3011871727 learned_addr learned my addr 172.21.15.78:0/3011871727 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f4633fff640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4634103c30 0x7f463419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46391b2640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4634108930 0x7f46341a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 -- 172.21.15.78:0/3011871727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4634108930 msgr2=0x7f46341a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4634108930 0x7f46341a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 -- 172.21.15.78:0/3011871727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4634103c30 msgr2=0x7f463419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4634103c30 0x7f463419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 -- 172.21.15.78:0/3011871727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341a1d80 con 0x7f46341029f0 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46391b2640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4634108930 0x7f46341a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.430+0000 7f46389b1640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46341029f0 0x7f463419a240 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f462800c970 tx=0x7f462800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.431+0000 7f4631ffb640 1 -- 172.21.15.78:0/3011871727 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628004220 con 0x7f46341029f0 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.431+0000 7f4631ffb640 1 -- 172.21.15.78:0/3011871727 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46280043c0 con 0x7f46341029f0 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.431+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341a2070 con 0x7f46341029f0 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.431+0000 7f4631ffb640 1 -- 172.21.15.78:0/3011871727 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628004a00 con 0x7f46341029f0 2024-04-25T14:11:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.431+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341a2570 con 0x7f46341029f0 2024-04-25T14:11:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.433+0000 7f4631ffb640 1 -- 172.21.15.78:0/3011871727 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f46280079b0 con 0x7f46341029f0 2024-04-25T14:11:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.433+0000 7f4641ba4740 1 Processor -- start 2024-04-25T14:11:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.433+0000 7f4641ba4740 1 -- start start 2024-04-25T14:11:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4631ffb640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f46240afaf0 0x7f46240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfe5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa904d0 0x555dcfe5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa986b0 0x555dcfe5ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4641ba4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555dcfdbfd80 con 0x555dcfab9390 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4641ba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555dcfd79a20 con 0x555dcfa904d0 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4641ba4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555dcfdbbd40 con 0x555dcfa986b0 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfe5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46391b2640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f46240afaf0 0x7f46240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfe5b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40286/0 (socket says 172.21.15.78:40286) 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 -- 172.21.15.78:0/28277889 learned_addr learned my addr 172.21.15.78:0/28277889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4631ffb640 1 -- 172.21.15.78:0/3011871727 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f46280deac0 con 0x7f46341029f0 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4633fff640 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa904d0 0x555dcfe5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46391b2640 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa986b0 0x555dcfe5ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 -- 172.21.15.78:0/28277889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa986b0 msgr2=0x555dcfe5ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa986b0 0x555dcfe5ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 -- 172.21.15.78:0/28277889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa904d0 msgr2=0x555dcfe5bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa904d0 0x555dcfe5bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 -- 172.21.15.78:0/28277889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcfd84c30 con 0x555dcfab9390 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f4633fff640 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa904d0 0x555dcfe5bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46391b2640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f46240afaf0 0x7f46240b2000 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f462c009fd0 tx=0x7f462c013ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f46389b1640 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfe5b810 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f46280dc5b0 tx=0x7f4628013840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9396b62b48cb4e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f461effd640 1 -- 172.21.15.78:0/28277889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462800d340 con 0x555dcfab9390 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f461effd640 1 -- 172.21.15.78:0/28277889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46280a6aa0 con 0x555dcfab9390 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.434+0000 7f461effd640 1 -- 172.21.15.78:0/28277889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280056b0 con 0x555dcfab9390 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 -- 172.21.15.78:0/28277889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 msgr2=0x555dcfe5b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfe5b810 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f46280dc5b0 tx=0x7f4628013840 comp rx=0 tx=0).stop 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 -- 172.21.15.78:0/28277889 shutdown_connections 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa986b0 0x555dcfe5ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa904d0 0x555dcfe5bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 --2- 172.21.15.78:0/28277889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfe5b810 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 -- 172.21.15.78:0/28277889 >> 172.21.15.78:0/28277889 conn(0x555dcfaad4e0 msgr2=0x555dcfe57020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 -- 172.21.15.78:0/28277889 shutdown_connections 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.435+0000 7f4641ba4740 1 -- 172.21.15.78:0/28277889 wait complete. 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 -- start start 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 Processor -- start 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa986b0 0x555dcfe65b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfab9390 0x555dcfe6ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555dcfe5d530 con 0x555dcfa986b0 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555dcfd46dc0 con 0x555dcfab9390 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555dcfe5d6b0 con 0x555dcfa904d0 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4641ba4740 10 client.0 _ll_register_callbacks cb 0x555dcfd62600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4633fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa986b0 0x555dcfe65b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4633fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa986b0 0x555dcfe65b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40292/0 (socket says 172.21.15.78:40292) 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f4633fff640 1 -- 172.21.15.78:0/4118078527 learned_addr learned my addr 172.21.15.78:0/4118078527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.436+0000 7f46391b2640 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfab9390 0x555dcfe6ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f46389b1640 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe65620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4633fff640 1 -- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 msgr2=0x555dcfe65620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4633fff640 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe65620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4633fff640 1 -- 172.21.15.78:0/4118078527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfab9390 msgr2=0x555dcfe6ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4633fff640 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfab9390 0x555dcfe6ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4633fff640 1 -- 172.21.15.78:0/4118078527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcfad13f0 con 0x555dcfa986b0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4633fff640 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa986b0 0x555dcfe65b60 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f4620031500 tx=0x7f46200317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d329ff9807f21307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4613fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4613fff640 1 -- 172.21.15.78:0/4118078527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620035070 con 0x555dcfa986b0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4613fff640 1 -- 172.21.15.78:0/4118078527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f4620004240 con 0x555dcfa986b0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46280a6aa0 con 0x555dcfa986b0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4613fff640 1 -- 172.21.15.78:0/4118078527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4620004440 con 0x555dcfa986b0 2024-04-25T14:11:09.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4641ba4740 10 client.18894 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555dcfb9b7d0 con 0x555dcfa986b0 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.437+0000 7f4613fff640 1 -- 172.21.15.78:0/4118078527 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46280a6aa0 con 0x555dcfa986b0 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4613fff640 1 -- 172.21.15.78:0/4118078527 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4620004c60 con 0x555dcfa986b0 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4641ba4740 10 client.18894 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4641ba4740 10 client.18894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4641ba4740 20 client.18894 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4641ba4740 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa77f60 0x555dcfe73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4641ba4740 4 client.18894 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dcfdcd990 con 0x555dcfa77f60 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f46389b1640 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa77f60 0x555dcfe73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f4613fff640 10 client.18894 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.438+0000 7f46389b1640 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa77f60 0x555dcfe73300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a845d33adf48f9da server_cookie=f0bd5d9afda728e1 in_seq=0 out_seq=0 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4613fff640 1 -- 172.21.15.78:0/4118078527 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555dcfdcd990 con 0x555dcfa77f60 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4613fff640 10 client.18894 handle_command_reply: tid=0 2024-04-25T14:11:09.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 client.18894 shutdown 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa986b0 msgr2=0x555dcfe65b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa986b0 0x555dcfe65b60 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f4620031500 tx=0x7f46200317d0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 shutdown_connections 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa77f60 0x555dcfe73300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfab9390 0x555dcfe6ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa986b0 0x555dcfe65b60 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 --2- 172.21.15.78:0/4118078527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4613fff640 0 client.18894 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 >> 172.21.15.78:0/4118078527 conn(0x555dcfaad4e0 msgr2=0x555dcfe59f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 shutdown_connections 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.439+0000 7f4641ba4740 1 -- 172.21.15.78:0/4118078527 wait complete. 2024-04-25T14:11:09.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.440+0000 7f4641ba4740 20 client.18894 trim_cache size 0 max 16384 2024-04-25T14:11:09.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 Processor -- start 2024-04-25T14:11:09.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 -- start start 2024-04-25T14:11:09.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa904d0 0x555dcfe63cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 0x555dcfe641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfab9390 0x555dcfe77ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46341adb50 con 0x555dcfa904d0 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46341ab420 con 0x555dcfa986b0 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f4641ba4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46341a9270 con 0x555dcfab9390 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f46389b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa904d0 0x555dcfe63cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f46389b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa904d0 0x555dcfe63cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40298/0 (socket says 172.21.15.78:40298) 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.453+0000 7f46389b1640 1 -- 172.21.15.78:0/2423645076 learned_addr learned my addr 172.21.15.78:0/2423645076 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46391b2640 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfab9390 0x555dcfe77ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46389b1640 1 -- 172.21.15.78:0/2423645076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfab9390 msgr2=0x555dcfe77ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46389b1640 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfab9390 0x555dcfe77ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46389b1640 1 -- 172.21.15.78:0/2423645076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 msgr2=0x555dcfe641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46389b1640 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 0x555dcfe641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46389b1640 1 -- 172.21.15.78:0/2423645076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcfe523a0 con 0x555dcfa904d0 2024-04-25T14:11:09.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46391b2640 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfab9390 0x555dcfe77ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f46389b1640 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa904d0 0x555dcfe63cb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f462800c940 tx=0x7f462801cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c2f84c8d6da5e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f461e7fc640 1 -- 172.21.15.78:0/2423645076 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280a6aa0 con 0x555dcfa904d0 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f461e7fc640 1 -- 172.21.15.78:0/2423645076 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46280a7690 con 0x555dcfa904d0 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.454+0000 7f461e7fc640 1 -- 172.21.15.78:0/2423645076 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280c9bb0 con 0x555dcfa904d0 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- 172.21.15.78:0/2423645076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa904d0 msgr2=0x555dcfe63cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa904d0 0x555dcfe63cb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f462800c940 tx=0x7f462801cd30 comp rx=0 tx=0).stop 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- 172.21.15.78:0/2423645076 shutdown_connections 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfab9390 0x555dcfe77ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 0x555dcfe641f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2423645076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfa904d0 0x555dcfe63cb0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- 172.21.15.78:0/2423645076 >> 172.21.15.78:0/2423645076 conn(0x555dcfaad4e0 msgr2=0x555dcfe60370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- 172.21.15.78:0/2423645076 shutdown_connections 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- 172.21.15.78:0/2423645076 wait complete. 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- start start 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 Processor -- start 2024-04-25T14:11:09.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe6b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 0x555dcfe6b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfedda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46341a98e0 con 0x555dcfab9390 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46341aad50 con 0x555dcfa986b0 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.455+0000 7f4641ba4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46341a9fb0 con 0x555dcfa904d0 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4641ba4740 10 client.0 _ll_register_callbacks cb 0x7f462c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfedda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfedda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40310/0 (socket says 172.21.15.78:40310) 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 -- 172.21.15.78:0/2155545005 learned_addr learned my addr 172.21.15.78:0/2155545005 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4633fff640 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 0x555dcfe6b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46389b1640 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe6b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 -- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 msgr2=0x555dcfe6b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe6b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 -- 172.21.15.78:0/2155545005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 msgr2=0x555dcfe6b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 0x555dcfe6b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 -- 172.21.15.78:0/2155545005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcfede030 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f46391b2640 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfedda70 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f462c011c60 tx=0x7f462c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd785af2db587960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4613fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4613fff640 1 -- 172.21.15.78:0/2155545005 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c003b70 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4613fff640 1 -- 172.21.15.78:0/2155545005 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f462c003d10 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46280a7690 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4613fff640 1 -- 172.21.15.78:0/2155545005 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f462c0218a0 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4641ba4740 10 client.18900 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.456+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555dcfedf270 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4613fff640 1 -- 172.21.15.78:0/2155545005 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46280a7690 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4613fff640 1 -- 172.21.15.78:0/2155545005 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f462c004590 con 0x555dcfab9390 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4641ba4740 10 client.18900 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4641ba4740 10 client.18900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-25T14:11:09.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4641ba4740 20 client.18900 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa4d3a0 0x555dcfee1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4641ba4740 4 client.18900 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 --> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555dcfdc7800 con 0x555dcfa4d3a0 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f46389b1640 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa4d3a0 0x555dcfee1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f4613fff640 10 client.18900 ms_handle_connect on v2:172.21.15.129:6834/2365161980 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.457+0000 7f46389b1640 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa4d3a0 0x555dcfee1c70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5d544daf1e56422 server_cookie=ac11e25ae402bcff in_seq=0 out_seq=0 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4613fff640 1 -- 172.21.15.78:0/2155545005 <== mds.2 v2:172.21.15.129:6834/2365161980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555dcfdc7800 con 0x555dcfa4d3a0 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4613fff640 10 client.18900 handle_command_reply: tid=0 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 client.18900 shutdown 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 msgr2=0x555dcfedda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfedda70 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f462c011c60 tx=0x7f462c019000 comp rx=0 tx=0).stop 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 shutdown_connections 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:6834/2365161980,v1:172.21.15.129:6835/2365161980] conn(0x555dcfa4d3a0 0x555dcfee1c70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555dcfab9390 0x555dcfedda70 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dcfa986b0 0x555dcfe6b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4613fff640 0 client.18900 ms_handle_reset on v2:172.21.15.129:6834/2365161980 2024-04-25T14:11:09.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 --2- 172.21.15.78:0/2155545005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555dcfa904d0 0x555dcfe6b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 >> 172.21.15.78:0/2155545005 conn(0x555dcfaad4e0 msgr2=0x555dcfe63cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 shutdown_connections 2024-04-25T14:11:09.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 1 -- 172.21.15.78:0/2155545005 wait complete. 2024-04-25T14:11:09.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.458+0000 7f4641ba4740 20 client.18900 trim_cache size 0 max 16384 2024-04-25T14:11:09.464 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f46240afaf0 msgr2=0x7f46240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f46240afaf0 0x7f46240b2000 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f462c009fd0 tx=0x7f462c013ba0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46341029f0 msgr2=0x7f463419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46341029f0 0x7f463419a240 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f462800c970 tx=0x7f462800ce30 comp rx=0 tx=0).stop 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 shutdown_connections 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4634108930 0x7f46341a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4634103c30 0x7f463419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f46240afaf0 0x7f46240b2000 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 --2- 172.21.15.78:0/3011871727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46341029f0 0x7f463419a240 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 >> 172.21.15.78:0/3011871727 conn(0x7f46340fdbc0 msgr2=0x7f4634074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 shutdown_connections 2024-04-25T14:11:09.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.459+0000 7f463ac15640 1 -- 172.21.15.78:0/3011871727 wait complete. 2024-04-25T14:11:09.470 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T14:11:09.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 -- 172.21.15.78:0/2671837656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0102f90 msgr2=0x7fade01033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 --2- 172.21.15.78:0/2671837656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0102f90 0x7fade01033a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009900 tx=0x7fadd002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 -- 172.21.15.78:0/2671837656 shutdown_connections 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 --2- 172.21.15.78:0/2671837656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0104470 0x7fade010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 --2- 172.21.15.78:0/2671837656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade0103970 0x7fade0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 --2- 172.21.15.78:0/2671837656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0102f90 0x7fade01033a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 -- 172.21.15.78:0/2671837656 >> 172.21.15.78:0/2671837656 conn(0x7fade00fd350 msgr2=0x7fade00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 -- 172.21.15.78:0/2671837656 shutdown_connections 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.578+0000 7fade514d640 1 -- 172.21.15.78:0/2671837656 wait complete. 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.579+0000 7fade514d640 1 Processor -- start 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.579+0000 7fade514d640 1 -- start start 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade0102f90 0x7fade019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0103970 0x7fade019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0104470 0x7fade01a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fade010e530 con 0x7fade0104470 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade010e3b0 con 0x7fade0103970 2024-04-25T14:11:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fade010e6b0 con 0x7fade0102f90 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0104470 0x7fade01a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0104470 0x7fade01a3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40334/0 (socket says 172.21.15.78:40334) 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 -- 172.21.15.78:0/1819860293 learned_addr learned my addr 172.21.15.78:0/1819860293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fadded9d640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade0102f90 0x7fade019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fadde59c640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0103970 0x7fade019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 -- 172.21.15.78:0/1819860293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade0102f90 msgr2=0x7fade019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade0102f90 0x7fade019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 -- 172.21.15.78:0/1819860293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0103970 msgr2=0x7fade019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0103970 0x7fade019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 -- 172.21.15.78:0/1819860293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade01a3f70 con 0x7fade0104470 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fadde59c640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0103970 0x7fade019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7faddf59e640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0104470 0x7fade01a3900 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fadd800eb10 tx=0x7fadd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fadc7fff640 1 -- 172.21.15.78:0/1819860293 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8004b20 con 0x7fade0104470 2024-04-25T14:11:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fadc7fff640 1 -- 172.21.15.78:0/1819860293 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd8004cc0 con 0x7fade0104470 2024-04-25T14:11:09.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade01a4260 con 0x7fade0104470 2024-04-25T14:11:09.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fadc7fff640 1 -- 172.21.15.78:0/1819860293 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8010670 con 0x7fade0104470 2024-04-25T14:11:09.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.580+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01a47b0 con 0x7fade0104470 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 Processor -- start 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadc7fff640 1 -- 172.21.15.78:0/1819860293 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fadd8005020 con 0x7fade0104470 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 -- start start 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 0x55acb2957810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb258c4d0 0x55acb2957d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb2958a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55acb28bbd80 con 0x55acb258c4d0 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55acb2875a20 con 0x55acb25b5390 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.583+0000 7fadebfff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55acb28b7d40 con 0x55acb25946b0 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb258c4d0 0x55acb2957d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb258c4d0 0x55acb2957d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40346/0 (socket says 172.21.15.78:40346) 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 -- 172.21.15.78:0/634409736 learned_addr learned my addr 172.21.15.78:0/634409736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadde59c640 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 0x55acb2957810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadde59c640 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb2958a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 -- 172.21.15.78:0/634409736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 msgr2=0x55acb2958a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb2958a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 -- 172.21.15.78:0/634409736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 msgr2=0x55acb2957810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 0x55acb2957810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 -- 172.21.15.78:0/634409736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb2880c30 con 0x55acb258c4d0 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadc7fff640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadc80afaf0 0x7fadc80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadded9d640 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb258c4d0 0x55acb2957d90 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009230 tx=0x7fadd0031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629418bec77a4fed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadde59c640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadc80afaf0 0x7fadc80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadc7fff640 1 -- 172.21.15.78:0/1819860293 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fadd80df250 con 0x7fade0104470 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadc57fa640 1 -- 172.21.15.78:0/634409736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0035030 con 0x55acb258c4d0 2024-04-25T14:11:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadc57fa640 1 -- 172.21.15.78:0/634409736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd0040070 con 0x55acb258c4d0 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.584+0000 7fadc57fa640 1 -- 172.21.15.78:0/634409736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003970 con 0x55acb258c4d0 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 -- 172.21.15.78:0/634409736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb258c4d0 msgr2=0x55acb2957d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadde59c640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadc80afaf0 0x7fadc80b2000 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005d00 tx=0x7fadd4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb258c4d0 0x55acb2957d90 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009230 tx=0x7fadd0031b10 comp rx=0 tx=0).stop 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 -- 172.21.15.78:0/634409736 shutdown_connections 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb2958a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb258c4d0 0x55acb2957d90 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 --2- 172.21.15.78:0/634409736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 0x55acb2957810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 -- 172.21.15.78:0/634409736 >> 172.21.15.78:0/634409736 conn(0x55acb25a94e0 msgr2=0x55acb2953020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 -- 172.21.15.78:0/634409736 shutdown_connections 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 -- 172.21.15.78:0/634409736 wait complete. 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.585+0000 7fadebfff740 1 -- start start 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 Processor -- start 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb258c4d0 0x55acb2961560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2961aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 0x55acb2968a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55acb29594d0 con 0x55acb25946b0 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55acb2842dc0 con 0x55acb25b5390 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55acb2959650 con 0x55acb258c4d0 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 10 client.0 _ll_register_callbacks cb 0x55acb285e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2961aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2961aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40354/0 (socket says 172.21.15.78:40354) 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 -- 172.21.15.78:0/2471918569 learned_addr learned my addr 172.21.15.78:0/2471918569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadded9d640 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb258c4d0 0x55acb2961560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 -- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb258c4d0 msgr2=0x55acb2961560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb258c4d0 0x55acb2961560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 -- 172.21.15.78:0/2471918569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 msgr2=0x55acb2968a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 0x55acb2968a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 -- 172.21.15.78:0/2471918569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb25cd3f0 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadde59c640 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2961aa0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fadd4010800 tx=0x7fadd4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5596f5a8fd83675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadc2ff5640 1 -- 172.21.15.78:0/2471918569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd400fb40 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadc2ff5640 1 -- 172.21.15.78:0/2471918569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fadd400fce0 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.586+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd0040070 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadc2ff5640 1 -- 172.21.15.78:0/2471918569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fadd4020420 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 10 client.18912 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55acb26977d0 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadc2ff5640 1 -- 172.21.15.78:0/2471918569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd0040070 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadc2ff5640 1 -- 172.21.15.78:0/2471918569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fadd4028630 con 0x55acb25946b0 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 10 client.18912 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 10 client.18912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 20 client.18912 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb2573f60 0x55acb296f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 4 client.18912 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:11:09.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb28c9990 con 0x55acb2573f60 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.587+0000 7fadded9d640 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb2573f60 0x55acb296f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.588+0000 7fadc2ff5640 10 client.18912 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.588+0000 7fadded9d640 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb2573f60 0x55acb296f3e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec9e2efc57c88f56 server_cookie=6f68e800e30d4dd0 in_seq=0 out_seq=0 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadc2ff5640 1 -- 172.21.15.78:0/2471918569 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55acb28c9990 con 0x55acb2573f60 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadc2ff5640 10 client.18912 handle_command_reply: tid=0 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 client.18912 shutdown 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 msgr2=0x55acb2961aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2961aa0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fadd4010800 tx=0x7fadd4005c90 comp rx=0 tx=0).stop 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 shutdown_connections 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb25b5390 0x55acb2968a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2961aa0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb2573f60 0x55acb296f3e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 --2- 172.21.15.78:0/2471918569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb258c4d0 0x55acb2961560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadc2ff5640 0 client.18912 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:11:09.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 >> 172.21.15.78:0/2471918569 conn(0x55acb25a94e0 msgr2=0x55acb2955f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 shutdown_connections 2024-04-25T14:11:09.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 1 -- 172.21.15.78:0/2471918569 wait complete. 2024-04-25T14:11:09.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.589+0000 7fadebfff740 20 client.18912 trim_cache size 0 max 16384 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.602+0000 7fadebfff740 1 Processor -- start 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.602+0000 7fadebfff740 1 -- start start 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadebfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb29530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadebfff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2968920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadebfff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25b5390 0x55acb295fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadebfff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fade01afdc0 con 0x55acb25946b0 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadebfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade01ad690 con 0x55acb258c4d0 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadebfff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fade01ab4e0 con 0x55acb25b5390 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2968920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2968920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40370/0 (socket says 172.21.15.78:40370) 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 -- 172.21.15.78:0/3751029417 learned_addr learned my addr 172.21.15.78:0/3751029417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7faddf59e640 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25b5390 0x55acb295fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 -- 172.21.15.78:0/3751029417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25b5390 msgr2=0x55acb295fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25b5390 0x55acb295fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 -- 172.21.15.78:0/3751029417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 msgr2=0x55acb29530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadded9d640 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb29530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb29530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 -- 172.21.15.78:0/3751029417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb295cce0 con 0x55acb25946b0 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadded9d640 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb29530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadde59c640 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2968920 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fadd4013d30 tx=0x7fadd4029cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed20e5f96134ebfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadc4ff9640 1 -- 172.21.15.78:0/3751029417 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4028400 con 0x55acb25946b0 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadc4ff9640 1 -- 172.21.15.78:0/3751029417 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40054f0 con 0x55acb25946b0 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.603+0000 7fadc4ff9640 1 -- 172.21.15.78:0/3751029417 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4004730 con 0x55acb25946b0 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 -- 172.21.15.78:0/3751029417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 msgr2=0x55acb2968920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2968920 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fadd4013d30 tx=0x7fadd4029cd0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 -- 172.21.15.78:0/3751029417 shutdown_connections 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25b5390 0x55acb295fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25946b0 0x55acb2968920 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 --2- 172.21.15.78:0/3751029417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb29530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 -- 172.21.15.78:0/3751029417 >> 172.21.15.78:0/3751029417 conn(0x55acb25a94e0 msgr2=0x55acb29529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 -- 172.21.15.78:0/3751029417 shutdown_connections 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 -- 172.21.15.78:0/3751029417 wait complete. 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.604+0000 7fadebfff740 1 -- start start 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 1 Processor -- start 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb2965cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb29661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25b5390 0x55acb2967120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fade01abb50 con 0x55acb25b5390 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fade01acfc0 con 0x55acb258c4d0 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fade01ac220 con 0x55acb25946b0 2024-04-25T14:11:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadebfff740 10 client.0 _ll_register_callbacks cb 0x7fadd400d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25b5390 0x55acb2967120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25b5390 0x55acb2967120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40374/0 (socket says 172.21.15.78:40374) 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 -- 172.21.15.78:0/1420991475 learned_addr learned my addr 172.21.15.78:0/1420991475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadded9d640 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb2965cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadde59c640 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb29661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 -- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 msgr2=0x55acb29661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb29661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 -- 172.21.15.78:0/1420991475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 msgr2=0x55acb2965cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb2965cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 -- 172.21.15.78:0/1420991475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb2968e60 con 0x55acb25b5390 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7faddf59e640 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25b5390 0x55acb2967120 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fadd80049d0 tx=0x7fadd8005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bfef4502ecc403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.605+0000 7fadc2ff5640 1 -- 172.21.15.78:0/1420991475 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd80a7e20 con 0x55acb25b5390 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadc2ff5640 1 -- 172.21.15.78:0/1420991475 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7fadd80b00f0 con 0x55acb25b5390 2024-04-25T14:11:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd40054f0 con 0x55acb25b5390 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadc2ff5640 1 -- 172.21.15.78:0/1420991475 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fadd80a7720 con 0x55acb25b5390 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 10 client.18918 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55acb29687d0 con 0x55acb25b5390 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadc2ff5640 1 -- 172.21.15.78:0/1420991475 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd40054f0 con 0x55acb25b5390 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadc2ff5640 1 -- 172.21.15.78:0/1420991475 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fadd808ebc0 con 0x55acb25b5390 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 10 client.18918 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 10 client.18918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 20 client.18918 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb25493a0 0x55acb29d9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 4 client.18918 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 --> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55acb28c3800 con 0x55acb25493a0 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.606+0000 7fadded9d640 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb25493a0 0x55acb29d9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadc2ff5640 10 client.18918 ms_handle_connect on v2:172.21.15.129:6838/1046146051 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadded9d640 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb25493a0 0x55acb29d9b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1577b3b1758e1d7b server_cookie=aa7e11634b596bb0 in_seq=0 out_seq=0 2024-04-25T14:11:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadc2ff5640 1 -- 172.21.15.78:0/1420991475 <== mds.1 v2:172.21.15.129:6838/1046146051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55acb28c3800 con 0x55acb25493a0 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadc2ff5640 10 client.18918 handle_command_reply: tid=0 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 client.18918 shutdown 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25b5390 msgr2=0x55acb2967120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25b5390 0x55acb2967120 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fadd80049d0 tx=0x7fadd8005690 comp rx=0 tx=0).stop 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 shutdown_connections 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acb25b5390 0x55acb2967120 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:6838/1046146051,v1:172.21.15.129:6839/1046146051] conn(0x55acb25493a0 0x55acb29d9b00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55acb25946b0 0x55acb29661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadc2ff5640 0 client.18918 ms_handle_reset on v2:172.21.15.129:6838/1046146051 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 --2- 172.21.15.78:0/1420991475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55acb258c4d0 0x55acb2965cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 >> 172.21.15.78:0/1420991475 conn(0x55acb25a94e0 msgr2=0x55acb295fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 shutdown_connections 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 1 -- 172.21.15.78:0/1420991475 wait complete. 2024-04-25T14:11:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.607+0000 7fadebfff740 20 client.18918 trim_cache size 0 max 16384 2024-04-25T14:11:09.614 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:11:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadc80afaf0 msgr2=0x7fadc80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadc80afaf0 0x7fadc80b2000 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fadd4005d00 tx=0x7fadd4005e50 comp rx=0 tx=0).stop 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0104470 msgr2=0x7fade01a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0104470 0x7fade01a3900 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fadd800eb10 tx=0x7fadd800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 shutdown_connections 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7fadc80afaf0 0x7fadc80b2000 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade0104470 0x7fade01a3900 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fade0103970 0x7fade019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 --2- 172.21.15.78:0/1819860293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fade0102f90 0x7fade019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 >> 172.21.15.78:0/1819860293 conn(0x7fade00fd350 msgr2=0x7fade00ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 shutdown_connections 2024-04-25T14:11:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.608+0000 7fade514d640 1 -- 172.21.15.78:0/1819860293 wait complete. 2024-04-25T14:11:09.619 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.728+0000 7f3c232e2640 1 -- 172.21.15.78:0/2302052702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c06b0e0 msgr2=0x7f3c1c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.728+0000 7f3c232e2640 1 --2- 172.21.15.78:0/2302052702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c06b0e0 0x7f3c1c101d60 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009900 tx=0x7f3c1802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 -- 172.21.15.78:0/2302052702 shutdown_connections 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 --2- 172.21.15.78:0/2302052702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c1033c0 0x7f3c1c103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 --2- 172.21.15.78:0/2302052702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c1c102330 0x7f3c1c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 --2- 172.21.15.78:0/2302052702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c06b0e0 0x7f3c1c101d60 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 -- 172.21.15.78:0/2302052702 >> 172.21.15.78:0/2302052702 conn(0x7f3c1c0fd350 msgr2=0x7f3c1c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 -- 172.21.15.78:0/2302052702 shutdown_connections 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 -- 172.21.15.78:0/2302052702 wait complete. 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 Processor -- start 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 -- start start 2024-04-25T14:11:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.729+0000 7f3c232e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c06b0e0 0x7f3c1c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c232e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c102330 0x7f3c1c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c232e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c1c1033c0 0x7f3c1c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c232e2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10e500 con 0x7f3c1c102330 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c232e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10e380 con 0x7f3c1c1033c0 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c232e2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c10e680 con 0x7f3c1c06b0e0 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c102330 0x7f3c1c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c102330 0x7f3c1c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40404/0 (socket says 172.21.15.78:40404) 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 -- 172.21.15.78:0/1768362444 learned_addr learned my addr 172.21.15.78:0/1768362444 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2187f640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c1c1033c0 0x7f3c1c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c06b0e0 0x7f3c1c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 -- 172.21.15.78:0/1768362444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c06b0e0 msgr2=0x7f3c1c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c06b0e0 0x7f3c1c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 -- 172.21.15.78:0/1768362444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c1c1033c0 msgr2=0x7f3c1c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c1c1033c0 0x7f3c1c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 -- 172.21.15.78:0/1768362444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c1a0d10 con 0x7f3c1c102330 2024-04-25T14:11:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c102330 0x7f3c1c199710 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009970 tx=0x7f3c0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c127fc640 1 -- 172.21.15.78:0/1768362444 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c00d020 con 0x7f3c1c102330 2024-04-25T14:11:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c127fc640 1 -- 172.21.15.78:0/1768362444 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0c00d1c0 con 0x7f3c1c102330 2024-04-25T14:11:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c104100 con 0x7f3c1c102330 2024-04-25T14:11:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c127fc640 1 -- 172.21.15.78:0/1768362444 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c00cc50 con 0x7f3c1c102330 2024-04-25T14:11:09.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.730+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c104650 con 0x7f3c1c102330 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 Processor -- start 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 -- start start 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c127fc640 1 -- 172.21.15.78:0/1768362444 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3c0c007860 con 0x7f3c1c102330 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e70ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e33f4d0 0x56279e70f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e3476b0 0x56279e70fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56279e66ed80 con 0x56279e33f4d0 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56279e628a20 con 0x56279e368390 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56279e66ad40 con 0x56279e3476b0 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2107e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e33f4d0 0x56279e70f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2107e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e33f4d0 0x56279e70f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40416/0 (socket says 172.21.15.78:40416) 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2107e640 1 -- 172.21.15.78:0/3067740004 learned_addr learned my addr 172.21.15.78:0/3067740004 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.733+0000 7f3c2187f640 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e70ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2087d640 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e3476b0 0x56279e70fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2107e640 1 -- 172.21.15.78:0/3067740004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e3476b0 msgr2=0x56279e70fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2107e640 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e3476b0 0x56279e70fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2107e640 1 -- 172.21.15.78:0/3067740004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 msgr2=0x56279e70ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2107e640 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e70ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2107e640 1 -- 172.21.15.78:0/3067740004 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56279e633c30 con 0x56279e33f4d0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2107e640 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e33f4d0 0x56279e70f1a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c18032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c589a8c20b296931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c127fc640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3c080afaf0 0x7f3c080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c077fe640 1 -- 172.21.15.78:0/3067740004 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1803f070 con 0x56279e33f4d0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c2187f640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3c080afaf0 0x7f3c080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c077fe640 1 -- 172.21.15.78:0/3067740004 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c18004040 con 0x56279e33f4d0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c077fe640 1 -- 172.21.15.78:0/3067740004 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18035070 con 0x56279e33f4d0 2024-04-25T14:11:09.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.734+0000 7f3c127fc640 1 -- 172.21.15.78:0/1768362444 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3c0c0ddee0 con 0x7f3c1c102330 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 -- 172.21.15.78:0/3067740004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e33f4d0 msgr2=0x56279e70f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2187f640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3c080afaf0 0x7f3c080b2000 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x56279e70fc70 tx=0x7f3c140097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e33f4d0 0x56279e70f1a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c18032ba0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 -- 172.21.15.78:0/3067740004 shutdown_connections 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e3476b0 0x56279e70fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e33f4d0 0x56279e70f1a0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/3067740004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e70ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 -- 172.21.15.78:0/3067740004 >> 172.21.15.78:0/3067740004 conn(0x56279e35c4e0 msgr2=0x56279e706020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 -- 172.21.15.78:0/3067740004 shutdown_connections 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 -- 172.21.15.78:0/3067740004 wait complete. 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 -- start start 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.735+0000 7f3c2a18d740 1 Processor -- start 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e718910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e718e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56279e7108e0 con 0x56279e3476b0 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56279e5f5dc0 con 0x56279e368390 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56279e710a60 con 0x56279e33f4d0 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2a18d740 10 client.0 _ll_register_callbacks cb 0x56279e611600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e718e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e718e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40430/0 (socket says 172.21.15.78:40430) 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 -- 172.21.15.78:0/1190861320 learned_addr learned my addr 172.21.15.78:0/1190861320 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2187f640 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e718910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 -- 172.21.15.78:0/1190861320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 msgr2=0x56279e718910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e718910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 -- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 msgr2=0x56279e71fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 -- 172.21.15.78:0/1190861320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279e3803f0 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e718e50 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0a82b0 tx=0x7f3c0c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d8603ea7ceda02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.736+0000 7f3c04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1190861320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0a7d20 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1190861320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3c0c0a7810 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c18004040 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1190861320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3c0c007b50 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 10 client.18930 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56279e44a7d0 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1190861320 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c18004040 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1190861320 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3c0c00c070 con 0x56279e3476b0 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 10 client.18930 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 10 client.18930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 20 client.18930 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e326f60 0x56279e7266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 4 client.18930 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T14:11:09.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56279e67c990 con 0x56279e326f60 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.737+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e326f60 0x56279e7266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.738+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e326f60 0x56279e7266a0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9978d41931b5ce server_cookie=68cff77fb7c57d1c in_seq=0 out_seq=0 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.738+0000 7f3c04ff9640 10 client.18930 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1190861320 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56279e67c990 con 0x56279e326f60 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c04ff9640 10 client.18930 handle_command_reply: tid=0 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 client.18930 shutdown 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 msgr2=0x56279e718e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e718e50 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0a82b0 tx=0x7f3c0c009f60 comp rx=0 tx=0).stop 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 shutdown_connections 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e718e50 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e326f60 0x56279e7266a0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1190861320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e718910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c04ff9640 0 client.18930 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 >> 172.21.15.78:0/1190861320 conn(0x56279e35c4e0 msgr2=0x56279e708f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 shutdown_connections 2024-04-25T14:11:09.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1190861320 wait complete. 2024-04-25T14:11:09.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.739+0000 7f3c2a18d740 20 client.18930 trim_cache size 0 max 16384 2024-04-25T14:11:09.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.752+0000 7f3c2a18d740 1 Processor -- start 2024-04-25T14:11:09.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.752+0000 7f3c2a18d740 1 -- start start 2024-04-25T14:11:09.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.752+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e717520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.752+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e3476b0 0x56279e71fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.752+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e368390 0x56279e705a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1ab9f0 con 0x56279e368390 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1a92c0 con 0x56279e3476b0 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1a7110 con 0x56279e33f4d0 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e368390 0x56279e705a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e368390 0x56279e705a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40432/0 (socket says 172.21.15.78:40432) 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 -- 172.21.15.78:0/2677022142 learned_addr learned my addr 172.21.15.78:0/2677022142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2107e640 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e717520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2087d640 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e3476b0 0x56279e71fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 -- 172.21.15.78:0/2677022142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 msgr2=0x56279e717520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e717520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 -- 172.21.15.78:0/2677022142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e3476b0 msgr2=0x56279e71fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e3476b0 0x56279e71fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 -- 172.21.15.78:0/2677022142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56279e710820 con 0x56279e368390 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2087d640 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e3476b0 0x56279e71fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:09.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2107e640 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e717520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:09.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c2187f640 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e368390 0x56279e705a50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097b0 tx=0x7f3c14015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60de4aadc0ab46d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c06ffd640 1 -- 172.21.15.78:0/2677022142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14029470 con 0x56279e368390 2024-04-25T14:11:09.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c06ffd640 1 -- 172.21.15.78:0/2677022142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c14024070 con 0x56279e368390 2024-04-25T14:11:09.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.753+0000 7f3c06ffd640 1 -- 172.21.15.78:0/2677022142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14010a60 con 0x56279e368390 2024-04-25T14:11:09.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 -- 172.21.15.78:0/2677022142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e368390 msgr2=0x56279e705a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e368390 0x56279e705a50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097b0 tx=0x7f3c14015000 comp rx=0 tx=0).stop 2024-04-25T14:11:09.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 -- 172.21.15.78:0/2677022142 shutdown_connections 2024-04-25T14:11:09.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e368390 0x56279e705a50 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e3476b0 0x56279e71fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/2677022142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e717520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 -- 172.21.15.78:0/2677022142 >> 172.21.15.78:0/2677022142 conn(0x56279e35c4e0 msgr2=0x56279e7136e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 -- 172.21.15.78:0/2677022142 shutdown_connections 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 -- 172.21.15.78:0/2677022142 wait complete. 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.754+0000 7f3c2a18d740 1 -- start start 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 1 Processor -- start 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e71d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e71d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1a7780 con 0x56279e3476b0 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1a8bf0 con 0x56279e368390 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c1a7e50 con 0x56279e33f4d0 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2a18d740 10 client.0 _ll_register_callbacks cb 0x7f3c0c0e14d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e71d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e71d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40436/0 (socket says 172.21.15.78:40436) 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 -- 172.21.15.78:0/1230711648 learned_addr learned my addr 172.21.15.78:0/1230711648 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e71d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2187f640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 -- 172.21.15.78:0/1230711648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 msgr2=0x56279e71d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e71d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 -- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 msgr2=0x56279e71e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 -- 172.21.15.78:0/1230711648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279e7131d0 con 0x56279e3476b0 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2187f640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e71d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c2087d640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e71d6c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0a4b50 tx=0x7f3c0c0a7a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90f7d77cbb125a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.755+0000 7f3c04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T14:11:09.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1230711648 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0a7d20 con 0x56279e3476b0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1230711648 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3c0c0a7ae0 con 0x56279e3476b0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c14024070 con 0x56279e3476b0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1230711648 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3c0c0a8380 con 0x56279e3476b0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 10 client.18936 fetch_fsmap learned FSMap version 22 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56279e71fbd0 con 0x56279e3476b0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1230711648 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c14024070 con 0x56279e3476b0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1230711648 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3c0c00c040 con 0x56279e3476b0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 10 client.18936 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 10 client.18936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 20 client.18936 populate_metadata read hostname 'smithi078' 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e2fc3a0 0x56279e790bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 4 client.18936 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 --> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56279e676800 con 0x56279e2fc3a0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.756+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e2fc3a0 0x56279e790bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:09.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c04ff9640 10 client.18936 ms_handle_connect on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2107e640 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e2fc3a0 0x56279e790bc0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3130027949259960 server_cookie=8fb485ac6a984850 in_seq=0 out_seq=0 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c04ff9640 1 -- 172.21.15.78:0/1230711648 <== mds.0 v2:172.21.15.112:6838/67760277 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56279e676800 con 0x56279e2fc3a0 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c04ff9640 10 client.18936 handle_command_reply: tid=0 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 client.18936 shutdown 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 msgr2=0x56279e71d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e71d6c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0a4b50 tx=0x7f3c0c0a7a20 comp rx=0 tx=0).stop 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 shutdown_connections 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56279e368390 0x56279e71e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56279e3476b0 0x56279e71d6c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.112:6838/67760277,v1:172.21.15.112:6839/67760277] conn(0x56279e2fc3a0 0x56279e790bc0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 --2- 172.21.15.78:0/1230711648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56279e33f4d0 0x56279e71d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c04ff9640 0 client.18936 ms_handle_reset on v2:172.21.15.112:6838/67760277 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 >> 172.21.15.78:0/1230711648 conn(0x56279e35c4e0 msgr2=0x56279e708e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 shutdown_connections 2024-04-25T14:11:09.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 1 -- 172.21.15.78:0/1230711648 wait complete. 2024-04-25T14:11:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.757+0000 7f3c2a18d740 20 client.18936 trim_cache size 0 max 16384 2024-04-25T14:11:09.764 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-04-25T14:11:09.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3c080afaf0 msgr2=0x7f3c080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3c080afaf0 0x7f3c080b2000 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x56279e70fc70 tx=0x7f3c140097e0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c102330 msgr2=0x7f3c1c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c102330 0x7f3c1c199710 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009970 tx=0x7f3c0c009e30 comp rx=0 tx=0).stop 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 shutdown_connections 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3c1c1033c0 0x7f3c1c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3c080afaf0 0x7f3c080b2000 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c1c102330 0x7f3c1c199710 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 --2- 172.21.15.78:0/1768362444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c06b0e0 0x7f3c1c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 >> 172.21.15.78:0/1768362444 conn(0x7f3c1c0fd350 msgr2=0x7f3c1c1071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 shutdown_connections 2024-04-25T14:11:09.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:09.758+0000 7f3c232e2640 1 -- 172.21.15.78:0/1768362444 wait complete. 2024-04-25T14:11:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:10 smithi112 ceph-mon[32617]: pgmap v697: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T14:11:10.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:10 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1820833883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:10 smithi129 ceph-mon[32569]: pgmap v697: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T14:11:10.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:10 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1820833883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:10 smithi078 ceph-mon[29163]: pgmap v697: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T14:11:10.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:10 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1820833883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:10.769 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-25T14:11:10.770 INFO:tasks.fwd_scrub:done joining 2024-04-25T14:11:10.770 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T14:11:10.784 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.903+0000 7f8210c27640 1 -- 172.21.15.78:0/2351487964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c103c40 msgr2=0x7f820c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.903+0000 7f8210c27640 1 --2- 172.21.15.78:0/2351487964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c103c40 0x7f820c104050 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009900 tx=0x7f81f402a410 comp rx=0 tx=0).stop 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.903+0000 7f8210c27640 1 -- 172.21.15.78:0/2351487964 shutdown_connections 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.903+0000 7f8210c27640 1 --2- 172.21.15.78:0/2351487964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f820c108940 0x7f820c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.903+0000 7f8210c27640 1 --2- 172.21.15.78:0/2351487964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c103c40 0x7f820c104050 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.903+0000 7f8210c27640 1 --2- 172.21.15.78:0/2351487964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f820c102a00 0x7f820c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.903+0000 7f8210c27640 1 -- 172.21.15.78:0/2351487964 >> 172.21.15.78:0/2351487964 conn(0x7f820c0fdbf0 msgr2=0x7f820c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.904+0000 7f8210c27640 1 -- 172.21.15.78:0/2351487964 shutdown_connections 2024-04-25T14:11:10.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.904+0000 7f8210c27640 1 -- 172.21.15.78:0/2351487964 wait complete. 2024-04-25T14:11:10.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.904+0000 7f8210c27640 1 Processor -- start 2024-04-25T14:11:10.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.904+0000 7f8210c27640 1 -- start start 2024-04-25T14:11:10.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f8210c27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f820c102a00 0x7f820c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f8210c27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f820c103c40 0x7f820c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f8210c27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c108940 0x7f820c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f8210c27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f820c10e520 con 0x7f820c108940 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f8210c27640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f820c10e3a0 con 0x7f820c102a00 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f8210c27640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f820c10e6a0 con 0x7f820c103c40 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c108940 0x7f820c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c108940 0x7f820c1a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40452/0 (socket says 172.21.15.78:40452) 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 -- 172.21.15.78:0/1696393365 learned_addr learned my addr 172.21.15.78:0/1696393365 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820a59c640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f820c102a00 0x7f820c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f8203fff640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f820c103c40 0x7f820c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 -- 172.21.15.78:0/1696393365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f820c103c40 msgr2=0x7f820c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f820c103c40 0x7f820c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 -- 172.21.15.78:0/1696393365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f820c102a00 msgr2=0x7f820c075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f820c102a00 0x7f820c075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 -- 172.21.15.78:0/1696393365 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c1a1d20 con 0x7f820c108940 2024-04-25T14:11:10.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f820ad9d640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c108940 0x7f820c1a1500 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f820400eaa0 tx=0x7f820400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:10.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.905+0000 7f82037fe640 1 -- 172.21.15.78:0/1696393365 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820400cdb0 con 0x7f820c108940 2024-04-25T14:11:10.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.906+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c1a1fb0 con 0x7f820c108940 2024-04-25T14:11:10.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.906+0000 7f82037fe640 1 -- 172.21.15.78:0/1696393365 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204005020 con 0x7f820c108940 2024-04-25T14:11:10.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.906+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c1a2470 con 0x7f820c108940 2024-04-25T14:11:10.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.906+0000 7f82037fe640 1 -- 172.21.15.78:0/1696393365 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820401f060 con 0x7f820c108940 2024-04-25T14:11:10.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.907+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820c107750 con 0x7f820c108940 2024-04-25T14:11:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.908+0000 7f82037fe640 1 -- 172.21.15.78:0/1696393365 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f82040051d0 con 0x7f820c108940 2024-04-25T14:11:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.909+0000 7f82037fe640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f81f80afaf0 0x7f81f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.909+0000 7f820a59c640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f81f80afaf0 0x7f81f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.909+0000 7f82037fe640 1 -- 172.21.15.78:0/1696393365 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f82040df1b0 con 0x7f820c108940 2024-04-25T14:11:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.909+0000 7f820a59c640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f81f80afaf0 0x7f81f80b2000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0058c0 tx=0x7f81fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:10.913+0000 7f82037fe640 1 -- 172.21.15.78:0/1696393365 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8204016030 con 0x7f820c108940 2024-04-25T14:11:11.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.092+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f820c02dd20 con 0x7f820c108940 2024-04-25T14:11:11.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.094+0000 7f82037fe640 1 -- 172.21.15.78:0/1696393365 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f82040a7d80 con 0x7f820c108940 2024-04-25T14:11:11.094 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:11:11.094 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f81f80afaf0 msgr2=0x7f81f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f81f80afaf0 0x7f81f80b2000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0058c0 tx=0x7f81fc00a2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c108940 msgr2=0x7f820c1a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c108940 0x7f820c1a1500 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f820400eaa0 tx=0x7f820400ef60 comp rx=0 tx=0).stop 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 shutdown_connections 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f81f80afaf0 0x7f81f80b2000 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f820c108940 0x7f820c1a1500 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f820c103c40 0x7f820c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 --2- 172.21.15.78:0/1696393365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f820c102a00 0x7f820c075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 >> 172.21.15.78:0/1696393365 conn(0x7f820c0fdbf0 msgr2=0x7f820c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 shutdown_connections 2024-04-25T14:11:11.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.096+0000 7f8210c27640 1 -- 172.21.15.78:0/1696393365 wait complete. 2024-04-25T14:11:11.108 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:11.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 -- 172.21.15.78:0/3344981838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 msgr2=0x7f882010bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 --2- 172.21.15.78:0/3344981838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f882010bcb0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f8818009ca0 tx=0x7f881802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 -- 172.21.15.78:0/3344981838 shutdown_connections 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 --2- 172.21.15.78:0/3344981838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f882010bcb0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 --2- 172.21.15.78:0/3344981838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8820106c30 0x7f8820109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 --2- 172.21.15.78:0/3344981838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820101e50 0x7f8820102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 -- 172.21.15.78:0/3344981838 >> 172.21.15.78:0/3344981838 conn(0x7f88200fd380 msgr2=0x7f88200ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 -- 172.21.15.78:0/3344981838 shutdown_connections 2024-04-25T14:11:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.220+0000 7f8827feb640 1 -- 172.21.15.78:0/3344981838 wait complete. 2024-04-25T14:11:11.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 Processor -- start 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 -- start start 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8820101e50 0x7f882019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820106c30 0x7f882019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f88201a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f882010f630 con 0x7f88201097a0 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f882010f4b0 con 0x7f8820106c30 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8827feb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f882010f7b0 con 0x7f8820101e50 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f88201a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f88201a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40476/0 (socket says 172.21.15.78:40476) 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 -- 172.21.15.78:0/1497800682 learned_addr learned my addr 172.21.15.78:0/1497800682 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8825d87640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8820101e50 0x7f882019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8825586640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820106c30 0x7f882019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 -- 172.21.15.78:0/1497800682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8820101e50 msgr2=0x7f882019a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8820101e50 0x7f882019a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 -- 172.21.15.78:0/1497800682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820106c30 msgr2=0x7f882019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820106c30 0x7f882019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.221+0000 7f8826588640 1 -- 172.21.15.78:0/1497800682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88201a1c30 con 0x7f88201097a0 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.222+0000 7f8826588640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f88201a1670 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f88180097b0 tx=0x7f8818003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.222+0000 7f8816ffd640 1 -- 172.21.15.78:0/1497800682 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004240 con 0x7f88201097a0 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.222+0000 7f8816ffd640 1 -- 172.21.15.78:0/1497800682 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f881802f380 con 0x7f88201097a0 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.222+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88201a1ec0 con 0x7f88201097a0 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.222+0000 7f8816ffd640 1 -- 172.21.15.78:0/1497800682 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881802fc20 con 0x7f88201097a0 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.222+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88201a2320 con 0x7f88201097a0 2024-04-25T14:11:11.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.223+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882006a190 con 0x7f88201097a0 2024-04-25T14:11:11.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.224+0000 7f8816ffd640 1 -- 172.21.15.78:0/1497800682 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f88180043e0 con 0x7f88201097a0 2024-04-25T14:11:11.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.225+0000 7f8816ffd640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f880c0afb70 0x7f880c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.225+0000 7f8825d87640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f880c0afb70 0x7f880c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.225+0000 7f8816ffd640 1 -- 172.21.15.78:0/1497800682 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f88181006b0 con 0x7f88201097a0 2024-04-25T14:11:11.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.226+0000 7f8825d87640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f880c0afb70 0x7f880c0b2080 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8820104b20 tx=0x7f881c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.229+0000 7f8816ffd640 1 -- 172.21.15.78:0/1497800682 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f882006a190 con 0x7f88201097a0 2024-04-25T14:11:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.411+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f882002dc10 con 0x7f88201097a0 2024-04-25T14:11:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.411+0000 7f8816ffd640 1 -- 172.21.15.78:0/1497800682 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f88180c9220 con 0x7f88201097a0 2024-04-25T14:11:11.411 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f880c0afb70 msgr2=0x7f880c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f880c0afb70 0x7f880c0b2080 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8820104b20 tx=0x7f881c00a2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 msgr2=0x7f88201a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f88201a1670 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f88180097b0 tx=0x7f8818003c20 comp rx=0 tx=0).stop 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 shutdown_connections 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f880c0afb70 0x7f880c0b2080 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88201097a0 0x7f88201a1670 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8820106c30 0x7f882019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 --2- 172.21.15.78:0/1497800682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8820101e50 0x7f882019a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 >> 172.21.15.78:0/1497800682 conn(0x7f88200fd380 msgr2=0x7f8820107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 shutdown_connections 2024-04-25T14:11:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.413+0000 7f8827feb640 1 -- 172.21.15.78:0/1497800682 wait complete. 2024-04-25T14:11:11.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:11 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1696393365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:11.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:11:11.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:11 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1696393365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 -- 172.21.15.78:0/1100644679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c103c20 msgr2=0x7f204c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 --2- 172.21.15.78:0/1100644679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c103c20 0x7f204c104030 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2048009900 tx=0x7f204802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 -- 172.21.15.78:0/1100644679 shutdown_connections 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 --2- 172.21.15.78:0/1100644679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f204c108920 0x7f204c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 --2- 172.21.15.78:0/1100644679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c103c20 0x7f204c104030 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 --2- 172.21.15.78:0/1100644679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f204c1029e0 0x7f204c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 -- 172.21.15.78:0/1100644679 >> 172.21.15.78:0/1100644679 conn(0x7f204c0fdbd0 msgr2=0x7f204c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 -- 172.21.15.78:0/1100644679 shutdown_connections 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.536+0000 7f2053815640 1 -- 172.21.15.78:0/1100644679 wait complete. 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 Processor -- start 2024-04-25T14:11:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 -- start start 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c1029e0 0x7f204c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f204c103c20 0x7f204c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f204c108920 0x7f204c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10f610 con 0x7f204c1029e0 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10f490 con 0x7f204c108920 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2053815640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10f790 con 0x7f204c103c20 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c1029e0 0x7f204c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c1029e0 0x7f204c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40492/0 (socket says 172.21.15.78:40492) 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 -- 172.21.15.78:0/1020429980 learned_addr learned my addr 172.21.15.78:0/1020429980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f2050db0640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f204c103c20 0x7f204c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 -- 172.21.15.78:0/1020429980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f204c103c20 msgr2=0x7f204c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f204c103c20 0x7f204c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 -- 172.21.15.78:0/1020429980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f204c108920 msgr2=0x7f204c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f204c108920 0x7f204c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 -- 172.21.15.78:0/1020429980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c1a2010 con 0x7f204c1029e0 2024-04-25T14:11:11.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f20515b1640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c1029e0 0x7f204c19a580 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f204000c970 tx=0x7f204000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.537+0000 7f203e7fc640 1 -- 172.21.15.78:0/1020429980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040004220 con 0x7f204c1029e0 2024-04-25T14:11:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.538+0000 7f203e7fc640 1 -- 172.21.15.78:0/1020429980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20400043c0 con 0x7f204c1029e0 2024-04-25T14:11:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.538+0000 7f203e7fc640 1 -- 172.21.15.78:0/1020429980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040004a00 con 0x7f204c1029e0 2024-04-25T14:11:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.538+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c1a2300 con 0x7f204c1029e0 2024-04-25T14:11:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.538+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1a2850 con 0x7f204c1029e0 2024-04-25T14:11:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.538+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c107730 con 0x7f204c1029e0 2024-04-25T14:11:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.540+0000 7f203e7fc640 1 -- 172.21.15.78:0/1020429980 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f20400079b0 con 0x7f204c1029e0 2024-04-25T14:11:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.541+0000 7f203e7fc640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20380afb70 0x7f20380b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.541+0000 7f2050db0640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20380afb70 0x7f20380b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.541+0000 7f203e7fc640 1 -- 172.21.15.78:0/1020429980 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f20400dee20 con 0x7f204c1029e0 2024-04-25T14:11:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.541+0000 7f2050db0640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20380afb70 0x7f20380b2080 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2048000c00 tx=0x7f2048031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.544+0000 7f203e7fc640 1 -- 172.21.15.78:0/1020429980 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2040014030 con 0x7f204c1029e0 2024-04-25T14:11:11.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:11 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1696393365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:11.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.722+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f204c02de90 con 0x7f204c1029e0 2024-04-25T14:11:11.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.722+0000 7f203e7fc640 1 -- 172.21.15.78:0/1020429980 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f20400a7a70 con 0x7f204c1029e0 2024-04-25T14:11:11.722 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T14:11:11.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20380afb70 msgr2=0x7f20380b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20380afb70 0x7f20380b2080 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2048000c00 tx=0x7f2048031400 comp rx=0 tx=0).stop 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c1029e0 msgr2=0x7f204c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c1029e0 0x7f204c19a580 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f204000c970 tx=0x7f204000ce30 comp rx=0 tx=0).stop 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 shutdown_connections 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f204c108920 0x7f204c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f204c103c20 0x7f204c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f20380afb70 0x7f20380b2080 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 --2- 172.21.15.78:0/1020429980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f204c1029e0 0x7f204c19a580 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.724+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 >> 172.21.15.78:0/1020429980 conn(0x7f204c0fdbd0 msgr2=0x7f204c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.725+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 shutdown_connections 2024-04-25T14:11:11.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.725+0000 7f2053815640 1 -- 172.21.15.78:0/1020429980 wait complete. 2024-04-25T14:11:11.736 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:11:11.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 -- 172.21.15.78:0/90070176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0103c20 msgr2=0x7f43e0104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 --2- 172.21.15.78:0/90070176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0103c20 0x7f43e0104030 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009900 tx=0x7f43d002a410 comp rx=0 tx=0).stop 2024-04-25T14:11:11.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 -- 172.21.15.78:0/90070176 shutdown_connections 2024-04-25T14:11:11.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 --2- 172.21.15.78:0/90070176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e0108920 0x7f43e010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 --2- 172.21.15.78:0/90070176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0103c20 0x7f43e0104030 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 --2- 172.21.15.78:0/90070176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e01029e0 0x7f43e01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 -- 172.21.15.78:0/90070176 >> 172.21.15.78:0/90070176 conn(0x7f43e00fdbd0 msgr2=0x7f43e0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 -- 172.21.15.78:0/90070176 shutdown_connections 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 -- 172.21.15.78:0/90070176 wait complete. 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 Processor -- start 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.846+0000 7f43e7af2640 1 -- start start 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e01029e0 0x7f43e0199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e0103c20 0x7f43e01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0108920 0x7f43e01a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010e500 con 0x7f43e0108920 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010e380 con 0x7f43e0103c20 2024-04-25T14:11:11.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010e680 con 0x7f43e01029e0 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0108920 0x7f43e01a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0108920 0x7f43e01a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40520/0 (socket says 172.21.15.78:40520) 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 -- 172.21.15.78:0/1194420639 learned_addr learned my addr 172.21.15.78:0/1194420639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e588e640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e01029e0 0x7f43e0199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e508d640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e0103c20 0x7f43e01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 -- 172.21.15.78:0/1194420639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e01029e0 msgr2=0x7f43e0199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e01029e0 0x7f43e0199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 -- 172.21.15.78:0/1194420639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e0103c20 msgr2=0x7f43e01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e0103c20 0x7f43e01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 -- 172.21.15.78:0/1194420639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e01a0ef0 con 0x7f43e0108920 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e508d640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e0103c20 0x7f43e01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e608f640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0108920 0x7f43e01a0930 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f43d800cce0 tx=0x7f43d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43d6ffd640 1 -- 172.21.15.78:0/1194420639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d800f450 con 0x7f43e0108920 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43d6ffd640 1 -- 172.21.15.78:0/1194420639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d800faa0 con 0x7f43e0108920 2024-04-25T14:11:11.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e01a11e0 con 0x7f43e0108920 2024-04-25T14:11:11.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43d6ffd640 1 -- 172.21.15.78:0/1194420639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8018b70 con 0x7f43e0108920 2024-04-25T14:11:11.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.847+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e01a16e0 con 0x7f43e0108920 2024-04-25T14:11:11.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.848+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e0107730 con 0x7f43e0108920 2024-04-25T14:11:11.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.850+0000 7f43d6ffd640 1 -- 172.21.15.78:0/1194420639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f43d8018d10 con 0x7f43e0108920 2024-04-25T14:11:11.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.850+0000 7f43d6ffd640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43cc0afaf0 0x7f43cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:11.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.851+0000 7f43e588e640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43cc0afaf0 0x7f43cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:11.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.851+0000 7f43d6ffd640 1 -- 172.21.15.78:0/1194420639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f43d80df130 con 0x7f43e0108920 2024-04-25T14:11:11.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.851+0000 7f43e588e640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43cc0afaf0 0x7f43cc0b2000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005fd0 tx=0x7f43dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:11.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:11.854+0000 7f43d6ffd640 1 -- 172.21.15.78:0/1194420639 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43d8014030 con 0x7f43e0108920 2024-04-25T14:11:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.029+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43e002dd10 con 0x7f43e0108920 2024-04-25T14:11:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.030+0000 7f43d6ffd640 1 -- 172.21.15.78:0/1194420639 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f43d80a1e20 con 0x7f43e0108920 2024-04-25T14:11:12.030 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T14:11:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43cc0afaf0 msgr2=0x7f43cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43cc0afaf0 0x7f43cc0b2000 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005fd0 tx=0x7f43dc00b390 comp rx=0 tx=0).stop 2024-04-25T14:11:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0108920 msgr2=0x7f43e01a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0108920 0x7f43e01a0930 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f43d800cce0 tx=0x7f43d800a620 comp rx=0 tx=0).stop 2024-04-25T14:11:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 shutdown_connections 2024-04-25T14:11:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f43cc0afaf0 0x7f43cc0b2000 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43e0108920 0x7f43e01a0930 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e0103c20 0x7f43e01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 --2- 172.21.15.78:0/1194420639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e01029e0 0x7f43e0199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 >> 172.21.15.78:0/1194420639 conn(0x7f43e00fdbd0 msgr2=0x7f43e0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 shutdown_connections 2024-04-25T14:11:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.032+0000 7f43e7af2640 1 -- 172.21.15.78:0/1194420639 wait complete. 2024-04-25T14:11:12.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 -- 172.21.15.78:0/453179988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 msgr2=0x7f6a60105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 --2- 172.21.15.78:0/453179988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a60105140 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009900 tx=0x7f6a4802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 -- 172.21.15.78:0/453179988 shutdown_connections 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 --2- 172.21.15.78:0/453179988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60109a30 0x7f6a6010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 --2- 172.21.15.78:0/453179988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a60105140 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 --2- 172.21.15.78:0/453179988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a60103af0 0x7f6a601047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 -- 172.21.15.78:0/453179988 >> 172.21.15.78:0/453179988 conn(0x7f6a600fed40 msgr2=0x7f6a60101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 -- 172.21.15.78:0/453179988 shutdown_connections 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.151+0000 7f6a65242640 1 -- 172.21.15.78:0/453179988 wait complete. 2024-04-25T14:11:12.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.152+0000 7f6a65242640 1 Processor -- start 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.152+0000 7f6a65242640 1 -- start start 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a65242640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a60103af0 0x7f6a6019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a65242640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a6019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a65242640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60109a30 0x7f6a601a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a65242640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010f5f0 con 0x7f6a60104d30 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a65242640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010f470 con 0x7f6a60109a30 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a65242640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010f770 con 0x7f6a60103af0 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a6019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a6019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40534/0 (socket says 172.21.15.78:40534) 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 -- 172.21.15.78:0/668832419 learned_addr learned my addr 172.21.15.78:0/668832419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:12.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5f59e640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60109a30 0x7f6a601a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5ed9d640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a60103af0 0x7f6a6019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 -- 172.21.15.78:0/668832419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a60103af0 msgr2=0x7f6a6019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a60103af0 0x7f6a6019c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 -- 172.21.15.78:0/668832419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60109a30 msgr2=0x7f6a601a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60109a30 0x7f6a601a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 -- 172.21.15.78:0/668832419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a601a3f60 con 0x7f6a60104d30 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5ed9d640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a60103af0 0x7f6a6019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a5e59c640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a6019c960 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a57fff640 1 -- 172.21.15.78:0/668832419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4803f030 con 0x7f6a60104d30 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a601a41f0 con 0x7f6a60104d30 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.153+0000 7f6a57fff640 1 -- 172.21.15.78:0/668832419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a48003cb0 con 0x7f6a60104d30 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.154+0000 7f6a57fff640 1 -- 172.21.15.78:0/668832419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48035070 con 0x7f6a60104d30 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.154+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601a4730 con 0x7f6a60104d30 2024-04-25T14:11:12.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.154+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a601a4a50 con 0x7f6a60104d30 2024-04-25T14:11:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.156+0000 7f6a57fff640 1 -- 172.21.15.78:0/668832419 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6a48042030 con 0x7f6a60104d30 2024-04-25T14:11:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.157+0000 7f6a57fff640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a4c0afaf0 0x7f6a4c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.157+0000 7f6a5ed9d640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a4c0afaf0 0x7f6a4c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:12.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.157+0000 7f6a57fff640 1 -- 172.21.15.78:0/668832419 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f6a480ffa50 con 0x7f6a60104d30 2024-04-25T14:11:12.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.157+0000 7f6a5ed9d640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a4c0afaf0 0x7f6a4c0b2000 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6a601046f0 tx=0x7f6a50005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:12.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.161+0000 7f6a57fff640 1 -- 172.21.15.78:0/668832419 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a480c85c0 con 0x7f6a60104d30 2024-04-25T14:11:12.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.397+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f6a601a4c20 con 0x7f6a60104d30 2024-04-25T14:11:12.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.405+0000 7f6a57fff640 1 -- 172.21.15.78:0/668832419 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+803124 (secure 0 0 0) 0x7f6a48003e60 con 0x7f6a60104d30 2024-04-25T14:11:12.415 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 18, 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "active_gid": 14152, 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "active_name": "x", 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addrs": { 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6800", 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3547239980 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-04-25T14:11:12.416 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6801", 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3547239980 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addr": "172.21.15.78:6801/3547239980", 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "active_change": "2024-04-25T13:50:08.038043+0000", 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "available": true, 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "standbys": [ 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24109, 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "z", 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-04-25T14:11:12.417 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:11:12.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.419 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.425 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:11:12.428 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:11:12.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-04-25T14:11:12.430 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T14:11:12.434 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.435 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:11:12.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.437 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.438 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.440 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.442 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.446 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:12.447 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-04-25T14:11:12.451 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.453 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.456 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-04-25T14:11:12.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.458 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.459 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-04-25T14:11:12.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.461 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:11:12.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.468 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.472 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.473 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.474 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-04-25T14:11:12.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-04-25T14:11:12.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-04-25T14:11:12.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.481 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.483 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.484 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:11:12.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.487 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.488 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.489 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.491 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-04-25T14:11:12.497 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.498 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.502 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.506 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.509 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.510 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:11:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:11:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:11:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:11:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:11:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-04-25T14:11:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:11:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:11:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-04-25T14:11:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-04-25T14:11:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-04-25T14:11:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-04-25T14:11:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:12.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.615 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:11:12.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-04-25T14:11:12.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.621 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.624 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.626 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.641 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.643 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.650 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-04-25T14:11:12.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.655 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.656 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.658 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.660 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.664 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.665 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.670 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.673 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.675 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.677 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.681 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.682 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.684 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.690 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.696 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.698 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T14:11:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-04-25T14:11:12.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.703 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.704 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.705 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.711 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-04-25T14:11:12.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.714 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.723 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T14:11:12.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.727 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:12.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.744 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.745 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.746 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.747 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.753 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.754 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.757 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-04-25T14:11:12.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.764 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-04-25T14:11:12.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-04-25T14:11:12.773 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:11:12.774 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:11:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.779 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.780 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.781 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.782 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.784 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.788 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.790 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.792 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.796 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.797 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.798 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.799 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.802 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:11:12.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:11:12.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.808 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T14:11:12.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:12.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-04-25T14:11:12.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.815 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.818 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.826 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.832 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.837 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.843 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.846 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24113, 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "y", 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-04-25T14:11:12.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.853 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.854 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.857 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.861 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.862 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.863 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.864 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-04-25T14:11:12.869 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.871 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-04-25T14:11:12.882 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.883 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.888 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.889 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.891 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:11:12.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-04-25T14:11:12.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.895 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-04-25T14:11:12.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.897 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.898 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:11:12.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:11:12.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:11:12.902 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.903 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.904 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-04-25T14:11:12.905 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-04-25T14:11:12.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-04-25T14:11:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-04-25T14:11:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-04-25T14:11:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-04-25T14:11:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:11:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:11:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:11:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:11:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.993 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.994 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.995 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:12.998 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:12.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.000 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.005 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.008 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.010 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:13.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.013 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T14:11:13.014 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-04-25T14:11:13.016 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.018 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T14:11:13.019 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.021 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.022 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.023 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.025 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-04-25T14:11:13.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:11:13.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:11:13.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.032 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.033 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.036 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.037 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.038 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-04-25T14:11:13.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-04-25T14:11:13.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.047 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.049 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.051 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.057 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.058 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.061 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.064 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.065 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.067 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.068 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.070 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:11:13.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-04-25T14:11:13.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.074 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.076 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.077 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.079 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.080 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.081 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.082 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.083 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.084 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-04-25T14:11:13.086 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:11:13.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-04-25T14:11:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-04-25T14:11:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-04-25T14:11:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-04-25T14:11:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-04-25T14:11:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T14:11:13.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.177 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.178 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.182 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.183 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.185 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.188 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.189 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.191 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.192 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-04-25T14:11:13.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.196 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.199 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.203 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.204 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.209 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-04-25T14:11:13.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-04-25T14:11:13.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.213 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.216 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.217 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.219 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.220 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.222 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.223 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.226 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.229 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.230 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.231 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.233 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.234 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.237 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.238 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.240 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.241 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.242 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:11:13.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.246 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.248 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.250 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.251 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.254 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.255 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.258 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.264 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.265 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.268 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.271 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.272 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.273 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.274 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-04-25T14:11:13.275 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-04-25T14:11:13.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-04-25T14:11:13.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.285 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.286 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.287 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.289 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-04-25T14:11:13.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "modules": [ 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard", 2024-04-25T14:11:13.292 INFO:teuthology.orchestra.run.smithi078.stdout: "iostat", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "nfs", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "restful" 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:13.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.294 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.295 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.299 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-04-25T14:11:13.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.301 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.302 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-04-25T14:11:13.304 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.305 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.307 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.308 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.310 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.311 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-04-25T14:11:13.313 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.314 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T14:11:13.315 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.318 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.319 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.320 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T14:11:13.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-04-25T14:11:13.322 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.323 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:13.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.328 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.330 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.332 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-04-25T14:11:13.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.335 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.336 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:13.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.339 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-04-25T14:11:13.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.342 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.344 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.346 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.347 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-04-25T14:11:13.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-04-25T14:11:13.350 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-04-25T14:11:13.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-04-25T14:11:13.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-04-25T14:11:13.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T14:11:13.357 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.359 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-04-25T14:11:13.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.364 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.366 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-04-25T14:11:13.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T14:11:13.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T14:11:13.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-04-25T14:11:13.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-04-25T14:11:13.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.380 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.381 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T14:11:13.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.383 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.384 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.387 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.388 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.389 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-04-25T14:11:13.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-04-25T14:11:13.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.395 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.396 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.398 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T14:11:13.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T14:11:13.401 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T14:11:13.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.404 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T14:11:13.406 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.409 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.411 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T14:11:13.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.418 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.424 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.425 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.427 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T14:11:13.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.429 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.431 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T14:11:13.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.440 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.442 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.444 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.446 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.447 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.448 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T14:11:13.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.453 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.455 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.457 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.458 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.459 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.462 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-04-25T14:11:13.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-04-25T14:11:13.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-04-25T14:11:13.470 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.471 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.477 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T14:11:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.479 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-04-25T14:11:13.480 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-04-25T14:11:13.481 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-04-25T14:11:13.482 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.483 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.487 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.489 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.490 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.492 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.497 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.501 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.504 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.505 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.510 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.511 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.513 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.516 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.518 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T14:11:13.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.527 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.529 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.535 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.536 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.538 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.539 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.541 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.544 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.545 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.549 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.551 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.552 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.553 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.555 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T14:11:13.557 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.559 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.560 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.562 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-04-25T14:11:13.565 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.566 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.575 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.576 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:13.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.578 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-04-25T14:11:13.580 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.581 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.583 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.585 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.588 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.590 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.591 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.595 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T14:11:13.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.601 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T14:11:13.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.604 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.606 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-04-25T14:11:13.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.613 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.615 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.617 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.618 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.625 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.630 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.633 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.644 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-04-25T14:11:13.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-04-25T14:11:13.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-04-25T14:11:13.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-04-25T14:11:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-04-25T14:11:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-04-25T14:11:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-04-25T14:11:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-04-25T14:11:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-04-25T14:11:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-04-25T14:11:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-04-25T14:11:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-04-25T14:11:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-04-25T14:11:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-04-25T14:11:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-04-25T14:11:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T14:11:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "services": { 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard": "https://172.21.15.78:8443/" 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "always_on_modules": { 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "octopus": [ 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T14:11:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "pacific": [ 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T14:11:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "quincy": [ 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T14:11:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "reef": [ 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "squid": [ 2024-04-25T14:11:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "last_failure_osd_epoch": 3, 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "active_clients": [ 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 615323434 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "libcephsqlite", 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T14:11:13.742 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 489464145 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 220839409 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:13.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2596351593 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T14:11:13.744 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:11:13.745 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:12 smithi112 ceph-mon[32617]: pgmap v698: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T14:11:13.745 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1497800682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.745 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1020429980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.746 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:12 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1194420639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.746 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:12 smithi129 ceph-mon[32569]: pgmap v698: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T14:11:13.746 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1497800682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.746 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1020429980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.746 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:12 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1194420639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.746 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:13 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/668832419' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a4c0afaf0 msgr2=0x7f6a4c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a4c0afaf0 0x7f6a4c0b2000 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6a601046f0 tx=0x7f6a50005e70 comp rx=0 tx=0).stop 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 msgr2=0x7f6a6019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a6019c960 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48004340 comp rx=0 tx=0).stop 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 shutdown_connections 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60109a30 0x7f6a601a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f6a4c0afaf0 0x7f6a4c0b2000 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a60104d30 0x7f6a6019c960 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 --2- 172.21.15.78:0/668832419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a60103af0 0x7f6a6019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 >> 172.21.15.78:0/668832419 conn(0x7f6a600fed40 msgr2=0x7f6a600ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 shutdown_connections 2024-04-25T14:11:13.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:12.411+0000 7f6a65242640 1 -- 172.21.15.78:0/668832419 wait complete. 2024-04-25T14:11:13.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:12 smithi078 ceph-mon[29163]: pgmap v698: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T14:11:13.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1497800682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1020429980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:12 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1194420639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:11:13.755 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T14:11:13.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T14:11:13.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:13 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/668832419' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:11:13.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:13 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/668832419' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T14:11:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:14.378 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:14 smithi078 ceph-mon[29163]: pgmap v699: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T14:11:14.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:14 smithi129 ceph-mon[32569]: pgmap v699: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T14:11:14.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:14 smithi112 ceph-mon[32617]: pgmap v699: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T14:11:15.964 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:11:15.964 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:15.964 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 232, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 229, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 510784, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 140864, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.037987311, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.015473846, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.016888220, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.004889969, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 228, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.012123680, 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000053174 2024-04-25T14:11:15.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 510784, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 140864 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 174, 2024-04-25T14:11:15.966 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 109040, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.013592419, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.010461731, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001366185, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 172, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.009609810, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000055870 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:15.967 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 109040 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 2944, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 448, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000681859, 2024-04-25T14:11:15.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000035503, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000623117, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000816977, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000272325 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:11:15.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:15.970 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 149007, 2024-04-25T14:11:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-04-25T14:11:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 233, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 478000, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:15.974 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 233, 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 233, 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 478000, 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:15.975 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:11:16.663 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:16 smithi078 ceph-mon[29163]: pgmap v700: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T14:11:16.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:16 smithi129 ceph-mon[32569]: pgmap v700: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T14:11:16.694 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T14:11:16.694 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T14:11:16.694 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T14:11:16.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T14:11:16.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:16 smithi112 ceph-mon[32617]: pgmap v700: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T14:11:16.949 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:18.362 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:18 smithi078 ceph-mon[29163]: pgmap v701: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:11:18.622 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:11:18.622 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:18.622 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-04-25T14:11:18.622 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-04-25T14:11:18.622 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:11:18.622 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001167523, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000264364, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:18.623 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 179, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 109920, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:18.624 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.013683770, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.010718126, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001148055, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 177, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.009983962, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000056406 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.625 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 109920 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 241, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 236, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 515088, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 144512, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.038779357, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.015969275, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.017526530, 2024-04-25T14:11:18.626 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.005092612, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 236, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.012790822, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000054198 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 515088, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 144512 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.627 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:18.628 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:18.629 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:18.630 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148945, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 29, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:18.631 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 238, 2024-04-25T14:11:18.632 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 478650, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 238, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 238, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 478650, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:18.633 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:11:18.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:18 smithi129 ceph-mon[32569]: pgmap v701: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:11:18.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:18 smithi112 ceph-mon[32617]: pgmap v701: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T14:11:19.319 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T14:11:19.319 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T14:11:19.319 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T14:11:19.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T14:11:19.559 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:20.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:20 smithi129 ceph-mon[32569]: pgmap v702: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:20.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:20 smithi078 ceph-mon[29163]: pgmap v702: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:20.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:20 smithi112 ceph-mon[32617]: pgmap v702: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:21.361 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 238, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 233, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 514272, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 142592, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.038043899, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.015839040, 2024-04-25T14:11:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.016192113, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.004920784, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 233, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.012491956, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000053613 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 514272, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 142592 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 176, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 109392, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.013952775, 2024-04-25T14:11:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.010693785, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001402356, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 174, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.009656373, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000055496 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 109392 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000248160, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000168023, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148984, 2024-04-25T14:11:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 235, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 478260, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 235, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 235, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 478260, 2024-04-25T14:11:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T14:11:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:21.375 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:11:22.008 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T14:11:22.008 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T14:11:22.008 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T14:11:22.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T14:11:22.248 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:11:22.623 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:22 smithi078 ceph-mon[29163]: pgmap v703: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:22.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:22 smithi129 ceph-mon[32569]: pgmap v703: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:22.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:22 smithi112 ceph-mon[32617]: pgmap v703: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001112549, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000273869, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:23.946 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 180, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 110096, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.014496904, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.011258952, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001363681, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:23.947 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 178, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.009453725, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000053110 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 110096 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 243, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 238, 2024-04-25T14:11:23.948 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 515632, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 145792, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.036948052, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.015476247, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.016819452, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.004752767, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 238, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.012515113, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000052584 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.949 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 515632, 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 145792 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T14:11:23.950 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148967, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 29, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T14:11:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-04-25T14:11:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 240, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 478910, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 240, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T14:11:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 240, 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 478910, 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T14:11:23.956 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T14:11:24.568 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:24 smithi112 ceph-mon[32617]: pgmap v704: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:24.582 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:24 smithi078 ceph-mon[29163]: pgmap v704: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:24.626 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T14:11:24.626 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T14:11:24.629 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T14:11:24.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T14:11:24.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:24 smithi129 ceph-mon[32569]: pgmap v704: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:24.877 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:11:26.536 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:26 smithi112 ceph-mon[32617]: pgmap v705: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:26.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:26 smithi129 ceph-mon[32569]: pgmap v705: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3922, 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 3819, 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 946750, 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 12467269, 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 295, 2024-04-25T14:11:26.678 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.465465879, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.136909882, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.182950804, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.098780109, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3820, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.229451289, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000060065 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1438, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000386798, 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000268 2024-04-25T14:11:26.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2902, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4493, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1546134, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1950941, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 13, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 5, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.417065383, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.157272164, 2024-04-25T14:11:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.201695783, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.070852066, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4493, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.305186283, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000067924 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4799, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.001226272, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000255 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1434368, 2024-04-25T14:11:26.681 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 1672688 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 2088, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2733, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 641789, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 1291146, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 9, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 7, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.215971043, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.073655212, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.076188377, 2024-04-25T14:11:26.682 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.054960379, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2728, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.227622180, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000083439 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2133, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000623983, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000292 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 398288 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T14:11:26.683 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 6237, 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.859213217, 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000298094 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T14:11:26.684 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 16, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000450407, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000028150 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 643, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 643, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 643, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 3.415913558, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005312462 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.685 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 347, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 25, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 5, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 20, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 17, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 1, 2024-04-25T14:11:26.686 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 1, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 2, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 684, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 675, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 26, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-25T14:11:26.687 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 316, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 316, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 204, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 23, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 181, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 112, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 1, 2024-04-25T14:11:26.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 611, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 613, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 599, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 721, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 704, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 2, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 622 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-25T14:11:26.689 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 4, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 1, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 3, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 2, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-25T14:11:26.690 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 23, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 2436, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 2405, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 2405, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 31, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-25T14:11:26.691 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 53, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 55, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 54, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 54, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 1, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 1, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 1, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 11973228, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 12066137, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 0, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2422, 2024-04-25T14:11:26.692 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 5.811484522, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002399456 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 0 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 24, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 49, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 25, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 19, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 20, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 1, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 25, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 29, 2024-04-25T14:11:26.693 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 4, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 1, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 15, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 14, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 128028, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 231824 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 1861, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 643, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 16, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-25T14:11:26.694 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003394480, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000678896 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 15, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002956577, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000197105 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.695 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-25T14:11:26.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.698 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 8, 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.013245766, 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001655720 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.699 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000427736, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000427736 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-25T14:11:26.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003747478, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001873739 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002026495, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001013247 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.701 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.008747315, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.004373657 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 600, 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 3.357963594, 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.005596605 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-25T14:11:26.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.704 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 7, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.021239029, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.003034147 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.705 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002165088, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.002165088 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.706 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 2, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 1, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 89, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 89, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 735, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:26.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:26.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:26.709 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 298812, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 116, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T14:11:26.710 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7632, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 161, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 98562, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 715, 2024-04-25T14:11:26.711 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 4559, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 8232045, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 26, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 4559, 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-25T14:11:26.712 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4559, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 30.149165399, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.006613109 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4559, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 5292 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 4559, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 363, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 4196, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-25T14:11:26.713 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 327, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 25, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 24, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 2423, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 169, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 516, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-25T14:11:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 339, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 50, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 599, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-25T14:11:26.715 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-25T14:11:26.716 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 77, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 3, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 12, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 12, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 810, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 0, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 10 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-25T14:11:26.717 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 1, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 2, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 2, 2024-04-25T14:11:26.718 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 2, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 2, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 8908, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 2213588, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 8908, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 8908, 2024-04-25T14:11:26.719 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 2213588, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:26.720 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 4559, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 8232247, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 4559, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 8232247, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:26.721 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 4559, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 4559, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 4559, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 4559, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 202, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 1, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 202, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.723 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55ff60346680": { 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2436, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 7871833, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2436, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:26.724 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2422, 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 7871833, 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:26.725 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T14:11:26.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:26 smithi078 ceph-mon[29163]: pgmap v705: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:27.426 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T14:11:27.426 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T14:11:27.426 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T14:11:27.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T14:11:27.685 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:11:28.648 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:28 smithi112 ceph-mon[32617]: pgmap v706: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:28.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:28 smithi129 ceph-mon[32569]: pgmap v706: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:28.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:28 smithi078 ceph-mon[29163]: pgmap v706: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:29.354 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 859, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 861, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 227061, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 273226, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.086342959, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.036379401, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.019616239, 2024-04-25T14:11:29.355 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.021572541, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 859, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.035478484, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000041302 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.356 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 777, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 954, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 193038, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 641661, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.087153144, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.032056094, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.018066322, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.028368403, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 953, 2024-04-25T14:11:29.357 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.043337400, 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000045474 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 396160 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1099, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1095, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 817408, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 417464, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.156067276, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.051780830, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.048699345, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.044585481, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1094, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.064243581, 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000058723 2024-04-25T14:11:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 531136, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 146528 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-25T14:11:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 4079, 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.133252064, 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000032667 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-25T14:11:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 1, 2024-04-25T14:11:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-25T14:11:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:11:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-25T14:11:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-25T14:11:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-25T14:11:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 5, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 2, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 4195732, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 4195424, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 5 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 3, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 3, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 0, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 1, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 1, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 0, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 0, 2024-04-25T14:11:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 119128, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215440 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-25T14:11:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-25T14:11:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-25T14:11:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-25T14:11:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.383 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 2, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 1, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1895, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.384 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:29.385 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:29.386 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 189423, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 51, 2024-04-25T14:11:29.387 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7632, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 161, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:29.388 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 8512, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 13, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 2483, 2024-04-25T14:11:29.389 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 21, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 3, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 2483, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2483, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.219966251, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000491327 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 2483, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 2484 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.390 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 2483, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 2483, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 1622, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 859, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-25T14:11:29.391 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:11:29.392 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-25T14:11:29.393 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 77, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 2, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 5, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 5, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-25T14:11:29.394 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-25T14:11:29.395 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 2731, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 976061, 2024-04-25T14:11:29.396 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 2731, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2731, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 976061, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:29.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 2483, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 1433, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2483, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1433, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.398 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 2483, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 2483, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 2483, 2024-04-25T14:11:29.399 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 2483, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:29.400 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55debcf288f0": { 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:29.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:29.402 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:29.403 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T14:11:30.081 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T14:11:30.081 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T14:11:30.081 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T14:11:30.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T14:11:30.317 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:11:30.439 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:30 smithi112 ceph-mon[32617]: pgmap v707: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:30.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:30 smithi129 ceph-mon[32569]: pgmap v707: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:30.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:30 smithi078 ceph-mon[29163]: pgmap v707: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1974, 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1967, 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 5246120, 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 694284, 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.204280719, 2024-04-25T14:11:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.079671582, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.058434573, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.050675657, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1966, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.085290988, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000043383 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 541696, 2024-04-25T14:11:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 146048 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 334, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 510, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 84710, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 502288, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.042888519, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.015643113, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.009962347, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.013244725, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 509, 2024-04-25T14:11:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.023608540, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000046382 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 397600 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1125, 2024-04-25T14:11:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 1126, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3959805, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 357439, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.100831867, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.027615161, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.030862482, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.033956158, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1124, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.030856522, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000027452 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T14:11:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 5431, 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.132864356, 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000024464 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-25T14:11:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 18, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 4, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 14, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 11, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 2, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-25T14:11:31.955 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 4, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 234, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:11:31.956 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:11:31.957 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-25T14:11:31.958 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-25T14:11:31.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 2436, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 2, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 54, 2024-04-25T14:11:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 11973228, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 12066137, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 12063304, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 2436 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 15, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 17, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 2, 2024-04-25T14:11:31.961 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 10, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 11, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 1, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 18, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 19, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 1, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 119840, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215440 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-04-25T14:11:31.962 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.963 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.964 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-25T14:11:31.965 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.966 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-25T14:11:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-25T14:11:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.967 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-25T14:11:31.968 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-25T14:11:31.969 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.973 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 2, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 1, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1898, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.974 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:31.975 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:31.976 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 189460, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 52, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:31.977 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 7632, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 161, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 59298, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 354, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:31.978 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 3177, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 26, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 4, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 3177, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3177, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 1.541957307, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000485350 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-25T14:11:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3177, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 3178 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 3177, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 3177, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 1689, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 1485, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-25T14:11:31.980 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 2, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-25T14:11:31.981 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-25T14:11:31.982 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 77, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 2, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 5, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 5, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-04-25T14:11:31.983 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:11:31.984 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 3429, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 8966236, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 3429, 2024-04-25T14:11:31.985 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 3429, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 8966236, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:31.986 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 3177, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 7871843, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 3177, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 7871843, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.987 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 3177, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 3177, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 3177, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 3177, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-25T14:11:31.989 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x55b24e90a8f0": { 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:31.990 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-25T14:11:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-25T14:11:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:31.994 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T14:11:32.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:32 smithi129 ceph-mon[32569]: pgmap v708: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:32.715 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:32 smithi112 ceph-mon[32617]: pgmap v708: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:32 smithi078 ceph-mon[29163]: pgmap v708: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:32.741 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T14:11:32.741 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T14:11:32.741 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T14:11:32.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T14:11:32.976 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:11:34.417 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:34 smithi112 ceph-mon[32617]: pgmap v709: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:34.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:34 smithi129 ceph-mon[32569]: pgmap v709: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 247, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 242, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 516720, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 148352, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.041390875, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.015343724, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.018418562, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.006198643, 2024-04-25T14:11:34.673 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 242, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.014225400, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000058782 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 516720, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 148352 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:34.674 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 183, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 160, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 110624, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.012738558, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.009188375, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001264113, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 181, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.011889235, 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000065686 2024-04-25T14:11:34.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 110624 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:34.676 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.000297976, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000222498, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:34.677 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:34.678 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:34.679 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:34.680 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148984, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:34.681 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T14:11:34.682 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 244, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 479430, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 244, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 244, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 479430, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T14:11:34.683 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T14:11:34.718 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:34 smithi078 ceph-mon[29163]: pgmap v709: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:35.319 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T14:11:35.320 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T14:11:35.320 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T14:11:35.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T14:11:35.568 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:11:36.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:36 smithi129 ceph-mon[32569]: pgmap v710: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:36.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:36 smithi078 ceph-mon[29163]: pgmap v710: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:36.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:36 smithi112 ceph-mon[32617]: pgmap v710: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:37.298 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1112, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 1105, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 831376, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 420966, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 5, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.162321755, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.051789994, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.050499310, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.047551275, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1104, 2024-04-25T14:11:37.299 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.065178934, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000059038 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 542080, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 147168 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 3, 2024-04-25T14:11:37.300 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 183, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 716, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 399818, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.010857552, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.007307120, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001714386, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000129959, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 181, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.012654442, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000069914 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:37.301 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 399120 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1650, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 1649, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 423573, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 523565, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-04-25T14:11:37.302 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.126338901, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.041714638, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.031135573, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.044324750, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1648, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.072678025, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000044100 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:11:37.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 4231, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.139833339, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000033049 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-04-25T14:11:37.304 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 0, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 0, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 0, 2024-04-25T14:11:37.305 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 0, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 1, 2024-04-25T14:11:37.306 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:11:37.307 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:11:37.308 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 0 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-04-25T14:11:37.309 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:11:37.310 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 4, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-04-25T14:11:37.311 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 0, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 2, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 4195424, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 4195283, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 4 2024-04-25T14:11:37.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 3, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 3, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 1, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 1, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 0, 2024-04-25T14:11:37.313 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 119052, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 215440 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 0, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 0, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 0, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.314 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.316 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.317 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-04-25T14:11:37.318 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.319 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-04-25T14:11:37.320 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-04-25T14:11:37.321 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.322 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.324 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 0, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 1, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 1, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 1, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 0, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 0, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 1091, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:37.326 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:37.327 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-25T14:11:37.328 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 181343, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 47, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 7632, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 161, 2024-04-25T14:11:37.329 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 8512, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 13, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-04-25T14:11:37.330 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 2508, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 21, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 3, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 2508, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2508, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 1.355968941, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000540657 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2508, 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 2509 2024-04-25T14:11:37.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 2508, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 2508, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 1639, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 867, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 0, 2024-04-25T14:11:37.332 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 1, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-04-25T14:11:37.333 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 77, 2024-04-25T14:11:37.334 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 2, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 4, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 4, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 0, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 2, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 0 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 0, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-04-25T14:11:37.335 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-25T14:11:37.336 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 2761, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 991328, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 2761, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 2761, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 991328, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.337 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 2508, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 1125, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 2508, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1125, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.338 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 2508, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 2508, 2024-04-25T14:11:37.339 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 2508, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2508, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:37.340 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x55bc7a604750": { 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:37.341 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:37.342 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T14:11:37.973 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T14:11:37.973 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T14:11:37.973 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T14:11:37.973 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T14:11:38.238 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:11:38.516 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:38 smithi112 ceph-mon[32617]: pgmap v711: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:38.655 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:38 smithi129 ceph-mon[32569]: pgmap v711: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:38 smithi078 ceph-mon[29163]: pgmap v711: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 47, 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 54, 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 11474, 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 66048, 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 14, 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 4, 2024-04-25T14:11:39.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.020820333, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.004224184, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.010702087, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000594161, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 50, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.013958170, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000279163 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 47, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000025915, 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000551 2024-04-25T14:11:39.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 5312, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 7008 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 67, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 248, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 14125, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 961514, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 53, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 5, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.031193558, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.009132747, 2024-04-25T14:11:39.980 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.018222147, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000594229, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 245, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.024719377, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000100895 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 46, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000005732, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000124 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 399744 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.981 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1727, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 1253, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 891419, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 490540, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 6, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.148028819, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.059084839, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.091429377, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.011881121, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1256, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.064239230, 2024-04-25T14:11:39.982 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000051145 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2365, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000594741, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000251 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 544272, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 153248 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.983 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 41, 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.002376622, 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000057966 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000007059, 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000003529 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:39.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 6, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 4, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 4, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.001427263, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000356815 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 2, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 11, 2024-04-25T14:11:39.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 14, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 0, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 14, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 14, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 0, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 3, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 3, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 2, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 7, 2024-04-25T14:11:39.986 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 3, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 2, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-04-25T14:11:39.987 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 5, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 298, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 2, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 3, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:11:39.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 2, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 2 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 3, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:11:39.989 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-04-25T14:11:39.990 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 5, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 5, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 2, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 2, 2024-04-25T14:11:39.991 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 4195732, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 0, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 4, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.008320880, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.002080220 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 0 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 17, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 17, 2024-04-25T14:11:39.992 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 0, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 15, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 16, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 1, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 14, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 14, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 0, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 3, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 4, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 1, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 120488, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 223632 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 8, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-04-25T14:11:39.993 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 6, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 14, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.001161924, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000580962 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-04-25T14:11:39.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.995 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000265339, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000132669 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.998 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-04-25T14:11:39.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.998 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:39.999 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.000 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-04-25T14:11:40.002 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.004 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.005 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 1, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 2, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 1, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 1, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 3, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 3, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 748, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:40.006 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:40.007 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-25T14:11:40.008 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 239201, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 80, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 7632, 2024-04-25T14:11:40.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 161, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 68797, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 295, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-04-25T14:11:40.010 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 42, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 12195, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 33, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 42, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 42, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.244994435, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.005833200 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 42, 2024-04-25T14:11:40.011 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 66 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 42, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 0, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 42, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 0, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 10, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 0, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 4, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 7, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-04-25T14:11:40.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 6, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 20, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-04-25T14:11:40.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-04-25T14:11:40.014 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 77, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 2, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 8, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 8, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-04-25T14:11:40.015 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 18, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 0, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 1 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 1, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-04-25T14:11:40.016 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 1837, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 733464, 2024-04-25T14:11:40.017 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 1837, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 1837, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 733464, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.018 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 42, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 12195, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 42, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 12195, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.019 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 42, 2024-04-25T14:11:40.020 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 42, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 42, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 42, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:40.021 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:40.022 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x562917e9c680": { 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 5, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 1428, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 5, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 4, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1428, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:40.023 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:40.024 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:40.024 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:40.024 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:40.024 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T14:11:40.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:40 smithi129 ceph-mon[32569]: pgmap v712: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:40 smithi078 ceph-mon[29163]: pgmap v712: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:40.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:40 smithi112 ceph-mon[32617]: pgmap v712: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:40.773 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T14:11:40.773 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T14:11:40.773 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T14:11:40.773 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T14:11:41.012 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:11:42.365 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:42 smithi129 ceph-mon[32569]: pgmap v713: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:42.648 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 556, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 665, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 126248, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 133347, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 8, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.077833730, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.029107351, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.038724124, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.005676308, 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 663, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.056843141, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000085736 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1168, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000346549, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000296 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:42.650 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 355, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 347, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 573635, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 226912, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 12, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 4, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.051960417, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.019959775, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.025110571, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.006556781, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 346, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.028154493, 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000081371 2024-04-25T14:11:42.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 123, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000033163, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000269 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 549264, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 152544 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 59, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 237, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 11978, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 958724, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 53, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-04-25T14:11:42.652 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.030239309, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.008711116, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.018205385, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000282069, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 236, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.022115513, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000093709 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 46, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000006672, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000145 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-25T14:11:42.653 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 401184 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 47, 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.001863640, 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000039651 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000138745, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000069372 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 0, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 0, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.655 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 11, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 10, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 10, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 10, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 0, 2024-04-25T14:11:42.656 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 1, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-04-25T14:11:42.657 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 5, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 298, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T14:11:42.658 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 0, 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 0 2024-04-25T14:11:42.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 1, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-04-25T14:11:42.660 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.661 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 4, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 4, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 2, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 2, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-04-25T14:11:42.662 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 4195424, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 0, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.005832718, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.001944239 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 0 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 13, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 13, 2024-04-25T14:11:42.663 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 0, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 11, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 12, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 1, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 10, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 10, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 0, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 0, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 0, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 0, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 120368, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 223632 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 0, 2024-04-25T14:11:42.664 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 0, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 4, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-04-25T14:11:42.665 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-04-25T14:11:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-04-25T14:11:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.671 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.674 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-04-25T14:11:42.675 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 0, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 1, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 1, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 0, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 0, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 0, 2024-04-25T14:11:42.676 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 80, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:42.677 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:42.678 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 245831, 2024-04-25T14:11:42.679 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 75, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 7632, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 161, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:42.680 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 50872, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 198, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 38, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 11332, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 28, 2024-04-25T14:11:42.681 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 38, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 38, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.232994711, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.006131439 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 38, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 60 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 38, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 0, 2024-04-25T14:11:42.682 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 38, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 10, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 3, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 6, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 6, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-04-25T14:11:42.683 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 20, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-04-25T14:11:42.684 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-04-25T14:11:42.685 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 77, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 2, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 9, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 9, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 14, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 0, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 1 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 1, 2024-04-25T14:11:42.686 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-04-25T14:11:42.687 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 966, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 607651, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 966, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 966, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 607651, 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:42.688 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 38, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 11332, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 38, 2024-04-25T14:11:42.689 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 11332, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 38, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 38, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 38, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 38, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:42.691 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:42.692 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x560591c04750": { 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 4, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 1120, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 4, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 3, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1120, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:42.693 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:42.694 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T14:11:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:42 smithi078 ceph-mon[29163]: pgmap v713: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:42.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:42 smithi112 ceph-mon[32617]: pgmap v713: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T14:11:43.425 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T14:11:43.425 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T14:11:43.425 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T14:11:43.426 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T14:11:43.665 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:11:44.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:44 smithi129 ceph-mon[32569]: pgmap v714: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:44 smithi078 ceph-mon[29163]: pgmap v714: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:44.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:44 smithi112 ceph-mon[32617]: pgmap v714: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T14:11:45.250 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T14:11:45.250 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T14:11:45.250 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 4, 2024-04-25T14:11:45.250 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 4, 2024-04-25T14:11:45.250 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 2944, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 448, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.001387414, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000038407, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000595834, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000722596, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000240865 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:45.251 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 247, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 244, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 515488, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 150160, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-04-25T14:11:45.252 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.044553397, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.017183463, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.019938207, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.006847712, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 243, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.012025875, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000049489 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.253 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 515488, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 150160 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 2, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 187, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 320, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 190096, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.012930502, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.009598942, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001095071, 2024-04-25T14:11:45.254 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 185, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.011449260, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000061887 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 190096 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.255 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T14:11:45.256 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T14:11:45.257 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T14:11:45.258 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148721, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 30, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T14:11:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-04-25T14:11:45.260 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 249, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 480614, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 249, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 249, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 480614, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T14:11:45.261 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T14:11:45.996 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T14:11:45.996 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T14:11:45.996 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T14:11:46.008 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T14:11:46.009 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T14:11:46.009 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:46.028 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T14:11:46.028 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T14:11:46.088 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:46.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 -- 172.21.15.78:0/2223463229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c108940 msgr2=0x7f790c10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 --2- 172.21.15.78:0/2223463229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c108940 0x7f790c10ad60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7904009ca0 tx=0x7f790402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 -- 172.21.15.78:0/2223463229 shutdown_connections 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 --2- 172.21.15.78:0/2223463229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c108940 0x7f790c10ad60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 --2- 172.21.15.78:0/2223463229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f790c103c40 0x7f790c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 --2- 172.21.15.78:0/2223463229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c102a00 0x7f790c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 -- 172.21.15.78:0/2223463229 >> 172.21.15.78:0/2223463229 conn(0x7f790c0fdbf0 msgr2=0x7f790c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 -- 172.21.15.78:0/2223463229 shutdown_connections 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 -- 172.21.15.78:0/2223463229 wait complete. 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 Processor -- start 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.245+0000 7f7911f7a640 1 -- start start 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f7911f7a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c102a00 0x7f790c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f7911f7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c103c40 0x7f790c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f7911f7a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f790c108940 0x7f790c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f7911f7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10e530 con 0x7f790c103c40 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f7911f7a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10e3b0 con 0x7f790c102a00 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f7911f7a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10e6b0 con 0x7f790c108940 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c103c40 0x7f790c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c103c40 0x7f790c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46012/0 (socket says 172.21.15.78:46012) 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 -- 172.21.15.78:0/436527850 learned_addr learned my addr 172.21.15.78:0/436527850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790b7fe640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c102a00 0x7f790c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790bfff640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f790c108940 0x7f790c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 -- 172.21.15.78:0/436527850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f790c108940 msgr2=0x7f790c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f790c108940 0x7f790c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 -- 172.21.15.78:0/436527850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c102a00 msgr2=0x7f790c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c102a00 0x7f790c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 -- 172.21.15.78:0/436527850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c1a1ce0 con 0x7f790c103c40 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790b7fe640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c102a00 0x7f790c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790bfff640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f790c108940 0x7f790c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f790affd640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c103c40 0x7f790c19a6e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f790000e9f0 tx=0x7f790000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.246+0000 7f7908ff9640 1 -- 172.21.15.78:0/436527850 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790000cd80 con 0x7f790c103c40 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.247+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c1a1fd0 con 0x7f790c103c40 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.247+0000 7f7908ff9640 1 -- 172.21.15.78:0/436527850 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79000054d0 con 0x7f790c103c40 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.247+0000 7f7908ff9640 1 -- 172.21.15.78:0/436527850 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900010690 con 0x7f790c103c40 2024-04-25T14:11:46.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.247+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c1a2440 con 0x7f790c103c40 2024-04-25T14:11:46.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.248+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f790c06a190 con 0x7f790c103c40 2024-04-25T14:11:46.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.249+0000 7f7908ff9640 1 -- 172.21.15.78:0/436527850 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7900004e90 con 0x7f790c103c40 2024-04-25T14:11:46.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.250+0000 7f7908ff9640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78f40afaf0 0x7f78f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.250+0000 7f790b7fe640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78f40afaf0 0x7f78f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.250+0000 7f7908ff9640 1 -- 172.21.15.78:0/436527850 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f79000df020 con 0x7f790c103c40 2024-04-25T14:11:46.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.250+0000 7f790b7fe640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78f40afaf0 0x7f78f40b2000 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f790c103600 tx=0x7f78fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.254+0000 7f7908ff9640 1 -- 172.21.15.78:0/436527850 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79000a7bf0 con 0x7f790c103c40 2024-04-25T14:11:46.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.438+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f790c108060 con 0x7f790c103c40 2024-04-25T14:11:46.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.439+0000 7f7908ff9640 1 -- 172.21.15.78:0/436527850 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f79000a74f0 con 0x7f790c103c40 2024-04-25T14:11:46.440 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:11:46.440 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78f40afaf0 msgr2=0x7f78f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78f40afaf0 0x7f78f40b2000 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f790c103600 tx=0x7f78fc005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c103c40 msgr2=0x7f790c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c103c40 0x7f790c19a6e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f790000e9f0 tx=0x7f790000edf0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 shutdown_connections 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f790c108940 0x7f790c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78f40afaf0 0x7f78f40b2000 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790c103c40 0x7f790c19a6e0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 --2- 172.21.15.78:0/436527850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f790c102a00 0x7f790c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 >> 172.21.15.78:0/436527850 conn(0x7f790c0fdbf0 msgr2=0x7f790c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 shutdown_connections 2024-04-25T14:11:46.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.442+0000 7f7911f7a640 1 -- 172.21.15.78:0/436527850 wait complete. 2024-04-25T14:11:46.452 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:46 smithi078 ceph-mon[29163]: pgmap v715: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:46.452 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:46 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:46.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:52:32.127466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:57:09.766427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:57:10.463427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:11:46.454 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.580+0000 7eff245ce640 1 -- 172.21.15.78:0/1082311947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 msgr2=0x7eff1c102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.580+0000 7eff245ce640 1 --2- 172.21.15.78:0/1082311947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c102240 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7eff18009900 tx=0x7eff1802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.580+0000 7eff245ce640 1 -- 172.21.15.78:0/1082311947 shutdown_connections 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.580+0000 7eff245ce640 1 --2- 172.21.15.78:0/1082311947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff1c109780 0x7eff1c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 --2- 172.21.15.78:0/1082311947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c106c10 0x7eff1c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 --2- 172.21.15.78:0/1082311947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c102240 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 -- 172.21.15.78:0/1082311947 >> 172.21.15.78:0/1082311947 conn(0x7eff1c0fd340 msgr2=0x7eff1c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 -- 172.21.15.78:0/1082311947 shutdown_connections 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 -- 172.21.15.78:0/1082311947 wait complete. 2024-04-25T14:11:46.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 Processor -- start 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 -- start start 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c106c10 0x7eff1c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff1c109780 0x7eff1c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10f5e0 con 0x7eff1c101e30 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10f460 con 0x7eff1c106c10 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.581+0000 7eff245ce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10f760 con 0x7eff1c109780 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44776/0 (socket says 172.21.15.78:44776) 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 -- 172.21.15.78:0/1969796732 learned_addr learned my addr 172.21.15.78:0/1969796732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff21b69640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c106c10 0x7eff1c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff22b6b640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff1c109780 0x7eff1c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 -- 172.21.15.78:0/1969796732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff1c109780 msgr2=0x7eff1c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff1c109780 0x7eff1c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 -- 172.21.15.78:0/1969796732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c106c10 msgr2=0x7eff1c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c106c10 0x7eff1c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 -- 172.21.15.78:0/1969796732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c1a1d60 con 0x7eff1c101e30 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff2236a640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c19a220 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7eff18006010 tx=0x7eff18031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff137fe640 1 -- 172.21.15.78:0/1969796732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1803f030 con 0x7eff1c101e30 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c1a1ff0 con 0x7eff1c101e30 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff137fe640 1 -- 172.21.15.78:0/1969796732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff18003aa0 con 0x7eff1c101e30 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff137fe640 1 -- 172.21.15.78:0/1969796732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18035070 con 0x7eff1c101e30 2024-04-25T14:11:46.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.582+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1a2400 con 0x7eff1c101e30 2024-04-25T14:11:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.584+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c0fee60 con 0x7eff1c101e30 2024-04-25T14:11:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.585+0000 7eff137fe640 1 -- 172.21.15.78:0/1969796732 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7eff18042030 con 0x7eff1c101e30 2024-04-25T14:11:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.586+0000 7eff137fe640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff080afaf0 0x7eff080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.586+0000 7eff21b69640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff080afaf0 0x7eff080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.586+0000 7eff137fe640 1 -- 172.21.15.78:0/1969796732 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7eff180ff120 con 0x7eff1c101e30 2024-04-25T14:11:46.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.586+0000 7eff21b69640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff080afaf0 0x7eff080b2000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0058c0 tx=0x7eff0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.595+0000 7eff137fe640 1 -- 172.21.15.78:0/1969796732 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff180c76a0 con 0x7eff1c101e30 2024-04-25T14:11:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:46 smithi129 ceph-mon[32569]: pgmap v715: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:46.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:46 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:46 smithi112 ceph-mon[32617]: pgmap v715: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T14:11:46.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:46 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:11:46.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.780+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff1c063840 con 0x7eff1c101e30 2024-04-25T14:11:46.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.781+0000 7eff137fe640 1 -- 172.21.15.78:0/1969796732 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7eff180cb9a0 con 0x7eff1c101e30 2024-04-25T14:11:46.781 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T14:11:46.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff080afaf0 msgr2=0x7eff080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff080afaf0 0x7eff080b2000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0058c0 tx=0x7eff0c00a2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 msgr2=0x7eff1c19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c19a220 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7eff18006010 tx=0x7eff18031a50 comp rx=0 tx=0).stop 2024-04-25T14:11:46.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 shutdown_connections 2024-04-25T14:11:46.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff1c109780 0x7eff1c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c106c10 0x7eff1c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7eff080afaf0 0x7eff080b2000 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 --2- 172.21.15.78:0/1969796732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff1c101e30 0x7eff1c19a220 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 >> 172.21.15.78:0/1969796732 conn(0x7eff1c0fd340 msgr2=0x7eff1c108d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 shutdown_connections 2024-04-25T14:11:46.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.783+0000 7eff245ce640 1 -- 172.21.15.78:0/1969796732 wait complete. 2024-04-25T14:11:46.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T14:11:46.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 -- 172.21.15.78:0/3841626968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2194103c20 msgr2=0x7f2194104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 --2- 172.21.15.78:0/3841626968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2194103c20 0x7f2194104030 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f217c009860 tx=0x7f217c02a340 comp rx=0 tx=0).stop 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 -- 172.21.15.78:0/3841626968 shutdown_connections 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 --2- 172.21.15.78:0/3841626968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2194108920 0x7f219410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 --2- 172.21.15.78:0/3841626968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2194103c20 0x7f2194104030 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 --2- 172.21.15.78:0/3841626968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21941029e0 0x7f21941036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 -- 172.21.15.78:0/3841626968 >> 172.21.15.78:0/3841626968 conn(0x7f21940fdbd0 msgr2=0x7f2194100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 -- 172.21.15.78:0/3841626968 shutdown_connections 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.906+0000 7f219890d640 1 -- 172.21.15.78:0/3841626968 wait complete. 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 Processor -- start 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 -- start start 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21941029e0 0x7f219406a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2194103c20 0x7f219419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2194108920 0x7f21941a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f219410e510 con 0x7f2194108920 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f219410e390 con 0x7f21941029e0 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219890d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f219410e690 con 0x7f2194103c20 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2194108920 0x7f21941a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f218bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2194103c20 0x7f219419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2194108920 0x7f21941a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44786/0 (socket says 172.21.15.78:44786) 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 -- 172.21.15.78:0/3906451961 learned_addr learned my addr 172.21.15.78:0/3906451961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f219259c640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21941029e0 0x7f219406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 -- 172.21.15.78:0/3906451961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2194103c20 msgr2=0x7f219419f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2194103c20 0x7f219419f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 -- 172.21.15.78:0/3906451961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21941029e0 msgr2=0x7f219406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:46.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21941029e0 0x7f219406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f2192d9d640 1 -- 172.21.15.78:0/3906451961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21941a2050 con 0x7f2194108920 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.907+0000 7f218bfff640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2194103c20 0x7f219419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.908+0000 7f2192d9d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2194108920 0x7f21941a1830 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f218c00eb10 tx=0x7f218c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.908+0000 7f218b7fe640 1 -- 172.21.15.78:0/3906451961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c004b20 con 0x7f2194108920 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.908+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21941a2340 con 0x7f2194108920 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.908+0000 7f218b7fe640 1 -- 172.21.15.78:0/3906451961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218c004cc0 con 0x7f2194108920 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.908+0000 7f218b7fe640 1 -- 172.21.15.78:0/3906451961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c010670 con 0x7f2194108920 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.908+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21941a2890 con 0x7f2194108920 2024-04-25T14:11:46.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.909+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2194107730 con 0x7f2194108920 2024-04-25T14:11:46.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.910+0000 7f218b7fe640 1 -- 172.21.15.78:0/3906451961 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f218c005000 con 0x7f2194108920 2024-04-25T14:11:46.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.911+0000 7f218b7fe640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f21800afaf0 0x7f21800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:46.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.911+0000 7f219259c640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f21800afaf0 0x7f21800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:46.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.911+0000 7f218b7fe640 1 -- 172.21.15.78:0/3906451961 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f218c0df370 con 0x7f2194108920 2024-04-25T14:11:46.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.911+0000 7f219259c640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f21800afaf0 0x7f21800b2000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2184009530 tx=0x7f2184009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:46.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:46.914+0000 7f218b7fe640 1 -- 172.21.15.78:0/3906451961 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f218c053ff0 con 0x7f2194108920 2024-04-25T14:11:47.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.090+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f219402dd30 con 0x7f2194108920 2024-04-25T14:11:47.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.091+0000 7f218b7fe640 1 -- 172.21.15.78:0/3906451961 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f218c0a7f40 con 0x7f2194108920 2024-04-25T14:11:47.091 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f21800afaf0 msgr2=0x7f21800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f21800afaf0 0x7f21800b2000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2184009530 tx=0x7f2184009260 comp rx=0 tx=0).stop 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2194108920 msgr2=0x7f21941a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2194108920 0x7f21941a1830 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f218c00eb10 tx=0x7f218c00c2d0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 shutdown_connections 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f21800afaf0 0x7f21800b2000 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2194108920 0x7f21941a1830 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2194103c20 0x7f219419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 --2- 172.21.15.78:0/3906451961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21941029e0 0x7f219406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 >> 172.21.15.78:0/3906451961 conn(0x7f21940fdbd0 msgr2=0x7f2194104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:47.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 shutdown_connections 2024-04-25T14:11:47.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.093+0000 7f219890d640 1 -- 172.21.15.78:0/3906451961 wait complete. 2024-04-25T14:11:47.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T14:11:47.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 -- 172.21.15.78:0/2554559606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730108920 msgr2=0x7f773010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 --2- 172.21.15.78:0/2554559606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730108920 0x7f773010ad40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7728009ca0 tx=0x7f772802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 -- 172.21.15.78:0/2554559606 shutdown_connections 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 --2- 172.21.15.78:0/2554559606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730108920 0x7f773010ad40 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 --2- 172.21.15.78:0/2554559606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7730103c20 0x7f7730104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 --2- 172.21.15.78:0/2554559606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77301029e0 0x7f77301036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 -- 172.21.15.78:0/2554559606 >> 172.21.15.78:0/2554559606 conn(0x7f77300fdbd0 msgr2=0x7f7730100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 -- 172.21.15.78:0/2554559606 shutdown_connections 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.220+0000 7f7736197640 1 -- 172.21.15.78:0/2554559606 wait complete. 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 Processor -- start 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 -- start start 2024-04-25T14:11:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77301029e0 0x7f773006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730103c20 0x7f773019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7730108920 0x7f77301a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f773010e510 con 0x7f7730103c20 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f773010e390 con 0x7f7730108920 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7736197640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f773010e690 con 0x7f77301029e0 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730103c20 0x7f773019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730103c20 0x7f773019f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44804/0 (socket says 172.21.15.78:44804) 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 -- 172.21.15.78:0/1644257564 learned_addr learned my addr 172.21.15.78:0/1644257564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:47.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f772f7fe640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77301029e0 0x7f773006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f772ffff640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7730108920 0x7f77301a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 -- 172.21.15.78:0/1644257564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77301029e0 msgr2=0x7f773006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77301029e0 0x7f773006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 -- 172.21.15.78:0/1644257564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7730108920 msgr2=0x7f77301a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7730108920 0x7f77301a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f7727fff640 1 -- 172.21.15.78:0/1644257564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77301a1f70 con 0x7f7730103c20 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.221+0000 7f772ffff640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7730108920 0x7f77301a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.222+0000 7f7727fff640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730103c20 0x7f773019f270 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f771800e9f0 tx=0x7f771800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.222+0000 7f772d7fa640 1 -- 172.21.15.78:0/1644257564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771800cd80 con 0x7f7730103c20 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.222+0000 7f772d7fa640 1 -- 172.21.15.78:0/1644257564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77180054d0 con 0x7f7730103c20 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.222+0000 7f772d7fa640 1 -- 172.21.15.78:0/1644257564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718010690 con 0x7f7730103c20 2024-04-25T14:11:47.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.222+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77301a2260 con 0x7f7730103c20 2024-04-25T14:11:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.222+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77301a2760 con 0x7f7730103c20 2024-04-25T14:11:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.223+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7730107730 con 0x7f7730103c20 2024-04-25T14:11:47.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.224+0000 7f772d7fa640 1 -- 172.21.15.78:0/1644257564 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7718004e30 con 0x7f7730103c20 2024-04-25T14:11:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.225+0000 7f772d7fa640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.225+0000 7f772f7fe640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.225+0000 7f772d7fa640 1 -- 172.21.15.78:0/1644257564 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f77180df2d0 con 0x7f7730103c20 2024-04-25T14:11:47.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.226+0000 7f772f7fe640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f77301035e0 tx=0x7f7720005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:47.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.229+0000 7f772d7fa640 1 -- 172.21.15.78:0/1644257564 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7718053ff0 con 0x7f7730103c20 2024-04-25T14:11:47.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.409+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f773002dc30 con 0x7f7730103c20 2024-04-25T14:11:47.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.410+0000 7f772d7fa640 1 -- 172.21.15.78:0/1644257564 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f77180a7ea0 con 0x7f7730103c20 2024-04-25T14:11:47.410 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 msgr2=0x7f771c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f77301035e0 tx=0x7f7720005f90 comp rx=0 tx=0).stop 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730103c20 msgr2=0x7f773019f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730103c20 0x7f773019f270 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f771800e9f0 tx=0x7f771800edf0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 shutdown_connections 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7730108920 0x7f77301a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7730103c20 0x7f773019f270 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 --2- 172.21.15.78:0/1644257564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77301029e0 0x7f773006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 >> 172.21.15.78:0/1644257564 conn(0x7f77300fdbd0 msgr2=0x7f7730104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:47.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 shutdown_connections 2024-04-25T14:11:47.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.412+0000 7f7736197640 1 -- 172.21.15.78:0/1644257564 wait complete. 2024-04-25T14:11:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/436527850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1969796732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3906451961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:11:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:47.413 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:47.414 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:47 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/436527850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:47.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1969796732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:47.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3906451961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:11:47.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:47.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:47.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:47 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.424 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 -- 172.21.15.78:0/603921511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 msgr2=0x7f3e3c10bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/603921511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c10bcb0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009ca0 tx=0x7f3e3402a8e0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 -- 172.21.15.78:0/603921511 shutdown_connections 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/603921511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c10bcb0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/603921511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e3c101750 0x7f3e3c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/603921511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c1050a0 0x7f3e3c1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 -- 172.21.15.78:0/603921511 >> 172.21.15.78:0/603921511 conn(0x7f3e3c0fd350 msgr2=0x7f3e3c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 -- 172.21.15.78:0/603921511 shutdown_connections 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.533+0000 7f3e41b2e640 1 -- 172.21.15.78:0/603921511 wait complete. 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 Processor -- start 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 -- start start 2024-04-25T14:11:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e3c101750 0x7f3e3c19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c1050a0 0x7f3e3c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c10f690 con 0x7f3e3c102250 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c10f510 con 0x7f3e3c1050a0 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.534+0000 7f3e41b2e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c10f810 con 0x7f3e3c101750 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44826/0 (socket says 172.21.15.78:44826) 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 -- 172.21.15.78:0/834202190 learned_addr learned my addr 172.21.15.78:0/834202190 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3bfff640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c1050a0 0x7f3e3c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3b7fe640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e3c101750 0x7f3e3c19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 -- 172.21.15.78:0/834202190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e3c101750 msgr2=0x7f3e3c19a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e3c101750 0x7f3e3c19a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 -- 172.21.15.78:0/834202190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c1050a0 msgr2=0x7f3e3c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c1050a0 0x7f3e3c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 -- 172.21.15.78:0/834202190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c1a1d20 con 0x7f3e3c102250 2024-04-25T14:11:47.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e3affd640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c19a720 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d950 tx=0x7f3e3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:47.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e38ff9640 1 -- 172.21.15.78:0/834202190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30017070 con 0x7f3e3c102250 2024-04-25T14:11:47.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e38ff9640 1 -- 172.21.15.78:0/834202190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e30009c30 con 0x7f3e3c102250 2024-04-25T14:11:47.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e38ff9640 1 -- 172.21.15.78:0/834202190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000f620 con 0x7f3e3c102250 2024-04-25T14:11:47.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c1a1fb0 con 0x7f3e3c102250 2024-04-25T14:11:47.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.535+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1a2570 con 0x7f3e3c102250 2024-04-25T14:11:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.536+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c10aa40 con 0x7f3e3c102250 2024-04-25T14:11:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.538+0000 7f3e38ff9640 1 -- 172.21.15.78:0/834202190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3e30010460 con 0x7f3e3c102250 2024-04-25T14:11:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.538+0000 7f3e38ff9640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3e240afb70 0x7f3e240b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.538+0000 7f3e3b7fe640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3e240afb70 0x7f3e240b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.538+0000 7f3e38ff9640 1 -- 172.21.15.78:0/834202190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f3e300de820 con 0x7f3e3c102250 2024-04-25T14:11:47.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.539+0000 7f3e3b7fe640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3e240afb70 0x7f3e240b2080 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c06b290 tx=0x7f3e2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:47.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.543+0000 7f3e38ff9640 1 -- 172.21.15.78:0/834202190 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e300a73f0 con 0x7f3e3c102250 2024-04-25T14:11:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.730+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3e3c02db50 con 0x7f3e3c102250 2024-04-25T14:11:47.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.731+0000 7f3e38ff9640 1 -- 172.21.15.78:0/834202190 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v77) ==== 81+0+590 (secure 0 0 0) 0x7f3e30009de0 con 0x7f3e3c102250 2024-04-25T14:11:47.735 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/360002043 2024-04-26T13:50:08.037919+0000 2024-04-25T14:11:47.735 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1759244565 2024-04-26T13:50:08.037919+0000 2024-04-25T14:11:47.735 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2174092153 2024-04-26T13:49:41.756206+0000 2024-04-25T14:11:47.735 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/3823505283 2024-04-26T13:49:41.756206+0000 2024-04-25T14:11:47.735 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/3823505283 2024-04-26T13:49:41.756206+0000 2024-04-25T14:11:47.735 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3906244379 2024-04-26T13:50:08.037919+0000 2024-04-25T14:11:47.736 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2353565210 2024-04-26T13:50:08.037919+0000 2024-04-25T14:11:47.736 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/231757119 2024-04-26T13:49:41.756206+0000 2024-04-25T14:11:47.736 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1444951251 2024-04-26T13:49:41.756206+0000 2024-04-25T14:11:47.736 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1759244565 2024-04-26T13:50:08.037919+0000 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3e240afb70 msgr2=0x7f3e240b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3e240afb70 0x7f3e240b2080 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c06b290 tx=0x7f3e2c008040 comp rx=0 tx=0).stop 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 msgr2=0x7f3e3c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c19a720 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d950 tx=0x7f3e3000de10 comp rx=0 tx=0).stop 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 shutdown_connections 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e3c1050a0 0x7f3e3c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f3e240afb70 0x7f3e240b2080 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e3c102250 0x7f3e3c19a720 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 --2- 172.21.15.78:0/834202190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e3c101750 0x7f3e3c19a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 >> 172.21.15.78:0/834202190 conn(0x7f3e3c0fd350 msgr2=0x7f3e3c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 shutdown_connections 2024-04-25T14:11:47.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:47.733+0000 7f3e41b2e640 1 -- 172.21.15.78:0/834202190 wait complete. 2024-04-25T14:11:47.744 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T14:11:47.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:47.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T14:11:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/436527850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1969796732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:11:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3906451961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T14:11:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:11:47.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:11:47.767 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:47 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' 2024-04-25T14:11:47.794 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T14:11:47.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:47.794 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T14:11:47.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:47.849 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-25T14:11:47.863 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T14:11:47.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:47.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T14:11:47.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T14:11:47.927 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-04-25T14:11:48.019 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T14:11:48.029 INFO:tasks.cephadm:Teardown begin 2024-04-25T14:11:48.030 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T14:11:48.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 -- 172.21.15.78:0/3214688507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 msgr2=0x7f989010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 --2- 172.21.15.78:0/3214688507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f989010ad20 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9888009ca0 tx=0x7f988802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 -- 172.21.15.78:0/3214688507 shutdown_connections 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 --2- 172.21.15.78:0/3214688507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f989010ad20 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 --2- 172.21.15.78:0/3214688507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9890103c00 0x7f9890104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 --2- 172.21.15.78:0/3214688507 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98901029c0 0x7f98901036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 -- 172.21.15.78:0/3214688507 >> 172.21.15.78:0/3214688507 conn(0x7f98900fdbd0 msgr2=0x7f9890100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 -- 172.21.15.78:0/3214688507 shutdown_connections 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 -- 172.21.15.78:0/3214688507 wait complete. 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 Processor -- start 2024-04-25T14:11:48.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 -- start start 2024-04-25T14:11:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98901029c0 0x7f9890199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.141+0000 7f9896d46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9890103c00 0x7f9890199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9896d46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f98901a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9896d46640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f989010e540 con 0x7f9890108900 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9896d46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f989010e3c0 con 0x7f9890103c00 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9896d46640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f989010e6c0 con 0x7f98901029c0 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f98901a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f98901a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44840/0 (socket says 172.21.15.78:44840) 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 -- 172.21.15.78:0/50535132 learned_addr learned my addr 172.21.15.78:0/50535132 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9887fff640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9890103c00 0x7f9890199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9894ae2640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98901029c0 0x7f9890199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 -- 172.21.15.78:0/50535132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98901029c0 msgr2=0x7f9890199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98901029c0 0x7f9890199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 -- 172.21.15.78:0/50535132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9890103c00 msgr2=0x7f9890199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9890103c00 0x7f9890199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 -- 172.21.15.78:0/50535132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901a0c50 con 0x7f9890108900 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9894ae2640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98901029c0 0x7f9890199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9887fff640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9890103c00 0x7f9890199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f98952e3640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f98901a05e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f98880097b0 tx=0x7f9888003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9885ffb640 1 -- 172.21.15.78:0/50535132 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888004240 con 0x7f9890108900 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9885ffb640 1 -- 172.21.15.78:0/50535132 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98880317c0 con 0x7f9890108900 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901a0ee0 con 0x7f9890108900 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.142+0000 7f9885ffb640 1 -- 172.21.15.78:0/50535132 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988802fde0 con 0x7f9890108900 2024-04-25T14:11:48.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.143+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a13c0 con 0x7f9890108900 2024-04-25T14:11:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.144+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989006afa0 con 0x7f9890108900 2024-04-25T14:11:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.145+0000 7f9885ffb640 1 -- 172.21.15.78:0/50535132 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f98880043e0 con 0x7f9890108900 2024-04-25T14:11:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.146+0000 7f9885ffb640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f98780afaf0 0x7f98780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.146+0000 7f9894ae2640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f98780afaf0 0x7f98780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.146+0000 7f9885ffb640 1 -- 172.21.15.78:0/50535132 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f9888100900 con 0x7f9890108900 2024-04-25T14:11:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.146+0000 7f9894ae2640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f98780afaf0 0x7f98780b2000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f98901035c0 tx=0x7f988c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.149+0000 7f9885ffb640 1 -- 172.21.15.78:0/50535132 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98880c94d0 con 0x7f9890108900 2024-04-25T14:11:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.367+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9890107710 con 0x7f9890108900 2024-04-25T14:11:48.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.368+0000 7f9885ffb640 1 -- 172.21.15.78:0/50535132 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f98880c8dd0 con 0x7f9890108900 2024-04-25T14:11:48.368 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:11:48.368 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:11:48.368 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f98780afaf0 msgr2=0x7f98780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f98780afaf0 0x7f98780b2000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f98901035c0 tx=0x7f988c005d30 comp rx=0 tx=0).stop 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 msgr2=0x7f98901a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f98901a05e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f98880097b0 tx=0x7f9888003c20 comp rx=0 tx=0).stop 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 shutdown_connections 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f98780afaf0 0x7f98780b2000 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9890108900 0x7f98901a05e0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9890103c00 0x7f9890199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 --2- 172.21.15.78:0/50535132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98901029c0 0x7f9890199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 >> 172.21.15.78:0/50535132 conn(0x7f98900fdbd0 msgr2=0x7f98901048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 shutdown_connections 2024-04-25T14:11:48.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.370+0000 7f9896d46640 1 -- 172.21.15.78:0/50535132 wait complete. 2024-04-25T14:11:48.382 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.490+0000 7f7739772640 1 -- 172.21.15.78:0/1298971540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 msgr2=0x7f77341090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 --2- 172.21.15.78:0/1298971540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f77341090e0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f7728009900 tx=0x7f772802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 -- 172.21.15.78:0/1298971540 shutdown_connections 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 --2- 172.21.15.78:0/1298971540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7734109770 0x7f773410bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 --2- 172.21.15.78:0/1298971540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f77341090e0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 --2- 172.21.15.78:0/1298971540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734104210 0x7f7734106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 -- 172.21.15.78:0/1298971540 >> 172.21.15.78:0/1298971540 conn(0x7f77340fdbd0 msgr2=0x7f7734100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 -- 172.21.15.78:0/1298971540 shutdown_connections 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 -- 172.21.15.78:0/1298971540 wait complete. 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 Processor -- start 2024-04-25T14:11:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.491+0000 7f7739772640 1 -- start start 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734104210 0x7f773419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f773419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7734109770 0x7f77341a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f773410f590 con 0x7f7734106c00 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f773410f410 con 0x7f7734104210 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f773410f710 con 0x7f7734109770 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f773419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f773419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44852/0 (socket says 172.21.15.78:44852) 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 -- 172.21.15.78:0/3205479806 learned_addr learned my addr 172.21.15.78:0/3205479806 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7732ffd640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734104210 0x7f773419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77337fe640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7734109770 0x7f77341a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 -- 172.21.15.78:0/3205479806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7734109770 msgr2=0x7f77341a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7734109770 0x7f77341a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 -- 172.21.15.78:0/3205479806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734104210 msgr2=0x7f773419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734104210 0x7f773419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 -- 172.21.15.78:0/3205479806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77341a1c90 con 0x7f7734106c00 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77337fe640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7734109770 0x7f77341a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f77327fc640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f773419a740 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7728006010 tx=0x7f77280043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f771bfff640 1 -- 172.21.15.78:0/3205479806 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772803f030 con 0x7f7734106c00 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f771bfff640 1 -- 172.21.15.78:0/3205479806 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7728003cb0 con 0x7f7734106c00 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77341a1f20 con 0x7f7734106c00 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f771bfff640 1 -- 172.21.15.78:0/3205479806 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728035070 con 0x7f7734106c00 2024-04-25T14:11:48.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.492+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77341a2320 con 0x7f7734106c00 2024-04-25T14:11:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.494+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77340ff6f0 con 0x7f7734106c00 2024-04-25T14:11:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.495+0000 7f771bfff640 1 -- 172.21.15.78:0/3205479806 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7728042030 con 0x7f7734106c00 2024-04-25T14:11:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.496+0000 7f771bfff640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.496+0000 7f7732ffd640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.496+0000 7f771bfff640 1 -- 172.21.15.78:0/3205479806 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f77280ffab0 con 0x7f7734106c00 2024-04-25T14:11:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.496+0000 7f7732ffd640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7724005c40 tx=0x7f772400a780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.500+0000 7f771bfff640 1 -- 172.21.15.78:0/3205479806 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77280c8680 con 0x7f7734106c00 2024-04-25T14:11:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:48 smithi129 ceph-mon[32569]: pgmap v716: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:48.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:48 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1644257564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:11:48.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:48 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/834202190' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:11:48.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.715+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f773402db50 con 0x7f7734106c00 2024-04-25T14:11:48.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.716+0000 7f771bfff640 1 -- 172.21.15.78:0/3205479806 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f77280c7f80 con 0x7f7734106c00 2024-04-25T14:11:48.716 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:11:48.716 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/835077889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835077889},{"type":"v1","addr":"172.21.15.78:6835","nonce":835077889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3031062077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3031062077},{"type":"v1","addr":"172.21.15.78:6837","nonce":3031062077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2808782144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2808782144},{"type":"v1","addr":"172.21.15.78:6839","nonce":2808782144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1109968953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1109968953},{"type":"v1","addr":"172.21.15.78:6841","nonce":1109968953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/1065513259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1065513259},{"type":"v1","addr":"172.21.15.129:6833","nonce":1065513259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6833/3808802757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3808802757},{"type":"v1","addr":"172.21.15.112:6833","nonce":3808802757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:57:11.484343+0000","modified":"2024-04-25T13:59:11.553703+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24484,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6835/2622963442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2622963442},{"type":"v1","addr":"172.21.15.112:6835","nonce":2622963442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.129:6835/2365161980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2365161980},{"type":"v1","addr":"172.21.15.129:6835","nonce":2365161980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/750801105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":750801105},{"type":"v1","addr":"172.21.15.129:6837","nonce":750801105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6837/4152465978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":4152465978},{"type":"v1","addr":"172.21.15.112:6837","nonce":4152465978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/1046146051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":1046146051},{"type":"v1","addr":"172.21.15.129:6839","nonce":1046146051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6839/67760277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":67760277},{"type":"v1","addr":"172.21.15.112:6839","nonce":67760277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24454,24484,24512]},"id":1}]} 2024-04-25T14:11:48.716 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 22 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 msgr2=0x7f771c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7724005c40 tx=0x7f772400a780 comp rx=0 tx=0).stop 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 msgr2=0x7f773419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f773419a740 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7728006010 tx=0x7f77280043b0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 shutdown_connections 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7734109770 0x7f77341a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f771c0afaf0 0x7f771c0b2000 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7734106c00 0x7f773419a740 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 --2- 172.21.15.78:0/3205479806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7734104210 0x7f773419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 >> 172.21.15.78:0/3205479806 conn(0x7f77340fdbd0 msgr2=0x7f7734104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 shutdown_connections 2024-04-25T14:11:48.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.718+0000 7f7739772640 1 -- 172.21.15.78:0/3205479806 wait complete. 2024-04-25T14:11:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:48 smithi078 ceph-mon[29163]: pgmap v716: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:48 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1644257564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:11:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:48 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/834202190' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:11:48.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:11:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:48 smithi112 ceph-mon[32617]: pgmap v716: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-25T14:11:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:48 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1644257564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T14:11:48.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:48 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/834202190' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T14:11:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 -- 172.21.15.78:0/1387773725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 msgr2=0x7f789010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 --2- 172.21.15.78:0/1387773725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f789010ad20 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f7888009ca0 tx=0x7f788802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 -- 172.21.15.78:0/1387773725 shutdown_connections 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 --2- 172.21.15.78:0/1387773725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f789010ad20 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 --2- 172.21.15.78:0/1387773725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7890103c00 0x7f7890104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 --2- 172.21.15.78:0/1387773725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78901029c0 0x7f78901036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 -- 172.21.15.78:0/1387773725 >> 172.21.15.78:0/1387773725 conn(0x7f78900fdbd0 msgr2=0x7f7890100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 -- 172.21.15.78:0/1387773725 shutdown_connections 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 -- 172.21.15.78:0/1387773725 wait complete. 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 Processor -- start 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.838+0000 7f789656c640 1 -- start start 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901029c0 0x7f789019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7890103c00 0x7f789019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f78901a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f789010e4f0 con 0x7f7890108900 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f789010e370 con 0x7f78901029c0 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f789010e670 con 0x7f7890103c00 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f78901a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f78901a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44876/0 (socket says 172.21.15.78:44876) 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 -- 172.21.15.78:0/2778356517 learned_addr learned my addr 172.21.15.78:0/2778356517 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 -- 172.21.15.78:0/2778356517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7890103c00 msgr2=0x7f789019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f788ffff640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901029c0 0x7f789019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f788f7fe640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7890103c00 0x7f789019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7890103c00 0x7f789019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 -- 172.21.15.78:0/2778356517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901029c0 msgr2=0x7f789019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901029c0 0x7f789019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 -- 172.21.15.78:0/2778356517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78901a1d00 con 0x7f7890108900 2024-04-25T14:11:48.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f788ffff640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901029c0 0x7f789019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:11:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f7894b09640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f78901a1690 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f78880097b0 tx=0x7f7888003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f788d7fa640 1 -- 172.21.15.78:0/2778356517 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888004240 con 0x7f7890108900 2024-04-25T14:11:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78901a1f90 con 0x7f7890108900 2024-04-25T14:11:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f788d7fa640 1 -- 172.21.15.78:0/2778356517 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f788802f380 con 0x7f7890108900 2024-04-25T14:11:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f788d7fa640 1 -- 172.21.15.78:0/2778356517 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788802fc30 con 0x7f7890108900 2024-04-25T14:11:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.839+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901a2450 con 0x7f7890108900 2024-04-25T14:11:48.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.841+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78901a26b0 con 0x7f7890108900 2024-04-25T14:11:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.842+0000 7f788d7fa640 1 -- 172.21.15.78:0/2778356517 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f78880043e0 con 0x7f7890108900 2024-04-25T14:11:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.842+0000 7f788d7fa640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78780afaf0 0x7f78780b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.843+0000 7f788ffff640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78780afaf0 0x7f78780b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.843+0000 7f788d7fa640 1 -- 172.21.15.78:0/2778356517 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f7888100610 con 0x7f7890108900 2024-04-25T14:11:48.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.843+0000 7f788ffff640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78780afaf0 0x7f78780b2000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f78901035c0 tx=0x7f7880005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:48.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:48.847+0000 7f788d7fa640 1 -- 172.21.15.78:0/2778356517 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78880c91e0 con 0x7f7890108900 2024-04-25T14:11:49.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.023+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78901a29a0 con 0x7f7890108900 2024-04-25T14:11:49.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.025+0000 7f788d7fa640 1 -- 172.21.15.78:0/2778356517 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22741 (secure 0 0 0) 0x7f78880098b0 con 0x7f7890108900 2024-04-25T14:11:49.025 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T14:11:49.025 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"5cff6f4a-030a-11ef-bc93-c7b262605968","created":"2024-04-25T13:49:14.716038+0000","modified":"2024-04-25T13:58:09.203005+0000","last_up_change":"2024-04-25T13:56:21.311476+0000","last_in_change":"2024-04-25T13:56:01.400962+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:52:32.127466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:57:09.766427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:57:10.463427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"caa5f025-762a-4cf2-a61e-9be6a4dd8013","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6803","nonce":2736058336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6805","nonce":2736058336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6809","nonce":2736058336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":2736058336},{"type":"v1","addr":"172.21.15.78:6807","nonce":2736058336}]},"public_addr":"172.21.15.78:6803/2736058336","cluster_addr":"172.21.15.78:6805/2736058336","heartbeat_back_addr":"172.21.15.78:6809/2736058336","heartbeat_front_addr":"172.21.15.78:6807/2736058336","state":["exists","up"]},{"osd":1,"uuid":"a258ec36-259b-448b-8103-de5ddfbbcfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6801","nonce":1864123277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6803","nonce":1864123277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6807","nonce":1864123277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1864123277},{"type":"v1","addr":"172.21.15.112:6805","nonce":1864123277}]},"public_addr":"172.21.15.112:6801/1864123277","cluster_addr":"172.21.15.112:6803/1864123277","heartbeat_back_addr":"172.21.15.112:6807/1864123277","heartbeat_front_addr":"172.21.15.112:6805/1864123277","state":["exists","up"]},{"osd":2,"uuid":"25bc85c2-81d0-479f-8b08-e5c37a158b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6801","nonce":1698371012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6803","nonce":1698371012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6807","nonce":1698371012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1698371012},{"type":"v1","addr":"172.21.15.129:6805","nonce":1698371012}]},"public_addr":"172.21.15.129:6801/1698371012","cluster_addr":"172.21.15.129:6803/1698371012","heartbeat_back_addr":"172.21.15.129:6807/1698371012","heartbeat_front_addr":"172.21.15.129:6805/1698371012","state":["exists","up"]},{"osd":3,"uuid":"d9b02621-7cdb-480e-863d-09bfc670da4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6811","nonce":2856659666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6813","nonce":2856659666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6817","nonce":2856659666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2856659666},{"type":"v1","addr":"172.21.15.78:6815","nonce":2856659666}]},"public_addr":"172.21.15.78:6811/2856659666","cluster_addr":"172.21.15.78:6813/2856659666","heartbeat_back_addr":"172.21.15.78:6817/2856659666","heartbeat_front_addr":"172.21.15.78:6815/2856659666","state":["exists","up"]},{"osd":4,"uuid":"b298aa83-5099-4f31-a030-f3b449241d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6809","nonce":3169717920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6811","nonce":3169717920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6815","nonce":3169717920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3169717920},{"type":"v1","addr":"172.21.15.112:6813","nonce":3169717920}]},"public_addr":"172.21.15.112:6809/3169717920","cluster_addr":"172.21.15.112:6811/3169717920","heartbeat_back_addr":"172.21.15.112:6815/3169717920","heartbeat_front_addr":"172.21.15.112:6813/3169717920","state":["exists","up"]},{"osd":5,"uuid":"5283d1e6-8c10-4e92-8c1c-baf015823098","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6809","nonce":1391792585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6811","nonce":1391792585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6815","nonce":1391792585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":1391792585},{"type":"v1","addr":"172.21.15.129:6813","nonce":1391792585}]},"public_addr":"172.21.15.129:6809/1391792585","cluster_addr":"172.21.15.129:6811/1391792585","heartbeat_back_addr":"172.21.15.129:6815/1391792585","heartbeat_front_addr":"172.21.15.129:6813/1391792585","state":["exists","up"]},{"osd":6,"uuid":"bee711c4-03f1-463b-a61a-208445896ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6819","nonce":3216701822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6821","nonce":3216701822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6825","nonce":3216701822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":3216701822},{"type":"v1","addr":"172.21.15.78:6823","nonce":3216701822}]},"public_addr":"172.21.15.78:6819/3216701822","cluster_addr":"172.21.15.78:6821/3216701822","heartbeat_back_addr":"172.21.15.78:6825/3216701822","heartbeat_front_addr":"172.21.15.78:6823/3216701822","state":["exists","up"]},{"osd":7,"uuid":"74bfe646-13d1-471c-acf9-8f84969d2313","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6817","nonce":1198451048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6819","nonce":1198451048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6823","nonce":1198451048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1198451048},{"type":"v1","addr":"172.21.15.112:6821","nonce":1198451048}]},"public_addr":"172.21.15.112:6817/1198451048","cluster_addr":"172.21.15.112:6819/1198451048","heartbeat_back_addr":"172.21.15.112:6823/1198451048","heartbeat_front_addr":"172.21.15.112:6821/1198451048","state":["exists","up"]},{"osd":8,"uuid":"2220b155-081a-4bdd-96c2-e46f8e20b97d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6817","nonce":4162935801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6819","nonce":4162935801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6823","nonce":4162935801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":4162935801},{"type":"v1","addr":"172.21.15.129:6821","nonce":4162935801}]},"public_addr":"172.21.15.129:6817/4162935801","cluster_addr":"172.21.15.129:6819/4162935801","heartbeat_back_addr":"172.21.15.129:6823/4162935801","heartbeat_front_addr":"172.21.15.129:6821/4162935801","state":["exists","up"]},{"osd":9,"uuid":"6395fc39-cdba-4a65-bfd5-1dd36eaaa385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6827","nonce":144296298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6829","nonce":144296298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6833","nonce":144296298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":144296298},{"type":"v1","addr":"172.21.15.78:6831","nonce":144296298}]},"public_addr":"172.21.15.78:6827/144296298","cluster_addr":"172.21.15.78:6829/144296298","heartbeat_back_addr":"172.21.15.78:6833/144296298","heartbeat_front_addr":"172.21.15.78:6831/144296298","state":["exists","up"]},{"osd":10,"uuid":"a6e1b875-eeac-47df-a4e1-df92fa867c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6825","nonce":1454658808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6827","nonce":1454658808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6831","nonce":1454658808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":1454658808},{"type":"v1","addr":"172.21.15.112:6829","nonce":1454658808}]},"public_addr":"172.21.15.112:6825/1454658808","cluster_addr":"172.21.15.112:6827/1454658808","heartbeat_back_addr":"172.21.15.112:6831/1454658808","heartbeat_front_addr":"172.21.15.112:6829/1454658808","state":["exists","up"]},{"osd":11,"uuid":"78830a45-c9f6-4c81-93ae-dbb8a1dc61db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6825","nonce":2167746323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6827","nonce":2167746323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6831","nonce":2167746323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":2167746323},{"type":"v1","addr":"172.21.15.129:6829","nonce":2167746323}]},"public_addr":"172.21.15.129:6825/2167746323","cluster_addr":"172.21.15.129:6827/2167746323","heartbeat_back_addr":"172.21.15.129:6831/2167746323","heartbeat_front_addr":"172.21.15.129:6829/2167746323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:51:41.390234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:05.296937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:29.576234+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:52:54.752942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:18.975426+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:53:44.395734+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:10.462782+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:54:36.221177+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:01.094940+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:27.615549+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:55:53.586095+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:56:20.124468+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:0/360002043":"2024-04-26T13:50:08.037919+0000","172.21.15.78:6801/1759244565":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2174092153":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6801/3823505283":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/3906244379":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/2353565210":"2024-04-26T13:50:08.037919+0000","172.21.15.78:0/231757119":"2024-04-26T13:49:41.756206+0000","172.21.15.78:0/1444951251":"2024-04-26T13:49:41.756206+0000","172.21.15.78:6800/1759244565":"2024-04-26T13:50:08.037919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:11:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78780afaf0 msgr2=0x7f78780b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78780afaf0 0x7f78780b2000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f78901035c0 tx=0x7f7880005e70 comp rx=0 tx=0).stop 2024-04-25T14:11:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 msgr2=0x7f78901a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f78901a1690 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f78880097b0 tx=0x7f7888003c20 comp rx=0 tx=0).stop 2024-04-25T14:11:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 shutdown_connections 2024-04-25T14:11:49.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f78780afaf0 0x7f78780b2000 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7890108900 0x7f78901a1690 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7890103c00 0x7f789019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 --2- 172.21.15.78:0/2778356517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901029c0 0x7f789019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 >> 172.21.15.78:0/2778356517 conn(0x7f78900fdbd0 msgr2=0x7f7890109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:49.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 shutdown_connections 2024-04-25T14:11:49.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.027+0000 7f789656c640 1 -- 172.21.15.78:0/2778356517 wait complete. 2024-04-25T14:11:49.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T14:11:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.148+0000 7f63fba20640 1 -- 172.21.15.78:0/1460500815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4104470 msgr2=0x7f63f410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.148+0000 7f63fba20640 1 --2- 172.21.15.78:0/1460500815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4104470 0x7f63f410abb0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009ca0 tx=0x7f63ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- 172.21.15.78:0/1460500815 shutdown_connections 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 --2- 172.21.15.78:0/1460500815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4104470 0x7f63f410abb0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 --2- 172.21.15.78:0/1460500815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f4103930 0x7f63f4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 --2- 172.21.15.78:0/1460500815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f4102f50 0x7f63f4103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- 172.21.15.78:0/1460500815 >> 172.21.15.78:0/1460500815 conn(0x7f63f40fd350 msgr2=0x7f63f40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- 172.21.15.78:0/1460500815 shutdown_connections 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- 172.21.15.78:0/1460500815 wait complete. 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 Processor -- start 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- start start 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4102f50 0x7f63f41021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f4103930 0x7f63f419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f4104470 0x7f63f41a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63f410e540 con 0x7f63f4102f50 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63f410e3c0 con 0x7f63f4104470 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.149+0000 7f63fba20640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63f410e6c0 con 0x7f63f4103930 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4102f50 0x7f63f41021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4102f50 0x7f63f41021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44886/0 (socket says 172.21.15.78:44886) 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 -- 172.21.15.78:0/2948437010 learned_addr learned my addr 172.21.15.78:0/2948437010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f9fbd640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f4104470 0x7f63f41a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f8fbb640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f4103930 0x7f63f419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 -- 172.21.15.78:0/2948437010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f4103930 msgr2=0x7f63f419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f4103930 0x7f63f419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 -- 172.21.15.78:0/2948437010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f4104470 msgr2=0x7f63f41a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f4104470 0x7f63f41a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 -- 172.21.15.78:0/2948437010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f41a1cd0 con 0x7f63f4102f50 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63f97bc640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4102f50 0x7f63f41021f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c970 tx=0x7f63f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63ea7fc640 1 -- 172.21.15.78:0/2948437010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0004220 con 0x7f63f4102f50 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63ea7fc640 1 -- 172.21.15.78:0/2948437010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f00043c0 con 0x7f63f4102f50 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f41a1fc0 con 0x7f63f4102f50 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63ea7fc640 1 -- 172.21.15.78:0/2948437010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0004a00 con 0x7f63f4102f50 2024-04-25T14:11:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.150+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f41a23d0 con 0x7f63f4102f50 2024-04-25T14:11:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.152+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f406a190 con 0x7f63f4102f50 2024-04-25T14:11:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.153+0000 7f63ea7fc640 1 -- 172.21.15.78:0/2948437010 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f63f00079b0 con 0x7f63f4102f50 2024-04-25T14:11:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.153+0000 7f63ea7fc640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f63e00afaf0 0x7f63e00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.154+0000 7f63f8fbb640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f63e00afaf0 0x7f63e00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.154+0000 7f63ea7fc640 1 -- 172.21.15.78:0/2948437010 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8248+0+0 (secure 0 0 0) 0x7f63f00de9a0 con 0x7f63f4102f50 2024-04-25T14:11:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.154+0000 7f63f8fbb640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f63e00afaf0 0x7f63e00b2000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f63f4101b90 tx=0x7f63e400c4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:49.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.158+0000 7f63ea7fc640 1 -- 172.21.15.78:0/2948437010 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63f406a190 con 0x7f63f4102f50 2024-04-25T14:11:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.378+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f63f4063840 con 0x7f63f4102f50 2024-04-25T14:11:49.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.381+0000 7f63ea7fc640 1 -- 172.21.15.78:0/2948437010 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7f63f0014030 con 0x7f63f4102f50 2024-04-25T14:11:49.381 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f63e00afaf0 msgr2=0x7f63e00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f63e00afaf0 0x7f63e00b2000 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f63f4101b90 tx=0x7f63e400c4a0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4102f50 msgr2=0x7f63f41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4102f50 0x7f63f41021f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c970 tx=0x7f63f000ce30 comp rx=0 tx=0).stop 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 shutdown_connections 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63f4104470 0x7f63f41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f4103930 0x7f63f419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f63e00afaf0 0x7f63e00b2000 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 --2- 172.21.15.78:0/2948437010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f4102f50 0x7f63f41021f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 >> 172.21.15.78:0/2948437010 conn(0x7f63f40fd350 msgr2=0x7f63f4106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 shutdown_connections 2024-04-25T14:11:49.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.384+0000 7f63fba20640 1 -- 172.21.15.78:0/2948437010 wait complete. 2024-04-25T14:11:49.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 -- 172.21.15.78:0/2857038565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 msgr2=0x7f36c81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 --2- 172.21.15.78:0/2857038565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c81047f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009900 tx=0x7f36b802a410 comp rx=0 tx=0).stop 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 -- 172.21.15.78:0/2857038565 shutdown_connections 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 --2- 172.21.15.78:0/2857038565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c8109a30 0x7f36c810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 --2- 172.21.15.78:0/2857038565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c8104d30 0x7f36c8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 --2- 172.21.15.78:0/2857038565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c81047f0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 -- 172.21.15.78:0/2857038565 >> 172.21.15.78:0/2857038565 conn(0x7f36c80fece0 msgr2=0x7f36c8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 -- 172.21.15.78:0/2857038565 shutdown_connections 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 -- 172.21.15.78:0/2857038565 wait complete. 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 Processor -- start 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.507+0000 7f36ce61a640 1 -- start start 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36ce61a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36ce61a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c8104d30 0x7f36c819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36ce61a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c8109a30 0x7f36c81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36ce61a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810f5f0 con 0x7f36c8103af0 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36ce61a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810f470 con 0x7f36c8104d30 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36ce61a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36c810f770 con 0x7f36c8109a30 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44896/0 (socket says 172.21.15.78:44896) 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 -- 172.21.15.78:0/1829625726 learned_addr learned my addr 172.21.15.78:0/1829625726 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c77fe640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c8104d30 0x7f36c819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36ccbb7640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c8109a30 0x7f36c81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 -- 172.21.15.78:0/1829625726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c8109a30 msgr2=0x7f36c81a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c8109a30 0x7f36c81a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 -- 172.21.15.78:0/1829625726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c8104d30 msgr2=0x7f36c819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c8104d30 0x7f36c819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 -- 172.21.15.78:0/1829625726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c81a1d90 con 0x7f36c8103af0 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c7fff640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c819a250 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f36b8006010 tx=0x7f36b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.508+0000 7f36c57fa640 1 -- 172.21.15.78:0/1829625726 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b803f030 con 0x7f36c8103af0 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.509+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c81a2020 con 0x7f36c8103af0 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.509+0000 7f36c57fa640 1 -- 172.21.15.78:0/1829625726 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36b8003aa0 con 0x7f36c8103af0 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.509+0000 7f36c57fa640 1 -- 172.21.15.78:0/1829625726 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8035070 con 0x7f36c8103af0 2024-04-25T14:11:49.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.509+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81a2560 con 0x7f36c8103af0 2024-04-25T14:11:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.510+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c8108840 con 0x7f36c8103af0 2024-04-25T14:11:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.511+0000 7f36c57fa640 1 -- 172.21.15.78:0/1829625726 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f36b8042030 con 0x7f36c8103af0 2024-04-25T14:11:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.512+0000 7f36c57fa640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f36b00afaf0 0x7f36b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.512+0000 7f36c77fe640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f36b00afaf0 0x7f36b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.512+0000 7f36c57fa640 1 -- 172.21.15.78:0/1829625726 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8506+0+0 (secure 0 0 0) 0x7f36b80ff7e0 con 0x7f36c8103af0 2024-04-25T14:11:49.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.512+0000 7f36c77fe640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f36b00afaf0 0x7f36b00b2000 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f36c819b2f0 tx=0x7f36bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:49.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.515+0000 7f36c57fa640 1 -- 172.21.15.78:0/1829625726 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36b80c82b0 con 0x7f36c8103af0 2024-04-25T14:11:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/50535132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3205479806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2778356517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.664 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i[63005]: ignoring --setuser ceph since I am not root 2024-04-25T14:11:49.664 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i[63005]: ignoring --setgroup ceph since I am not root 2024-04-25T14:11:49.664 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-mds[63009]: -- 172.21.15.129:0/224265213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad9c425a40 con 0x55ad9c46d180 2024-04-25T14:11:49.664 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i[63005]: starting mds.i at 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c[62695]: ignoring --setuser ceph since I am not root 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c[62695]: ignoring --setgroup ceph since I am not root 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-mds[62699]: -- 172.21.15.129:0/3179355616 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a84aa09a40 con 0x55a84aa51a80 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c[62695]: starting mds.c at 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f[62385]: ignoring --setuser ceph since I am not root 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f[62385]: ignoring --setgroup ceph since I am not root 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-mds[62389]: -- 172.21.15.129:0/889465964 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5572dc3f64e0 con 0x5572dc441180 2024-04-25T14:11:49.665 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-mds[62389]: -- 172.21.15.129:0/889465964 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572dc3f9a40 con 0x5572dc441180 2024-04-25T14:11:49.666 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:11:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f[62385]: starting mds.f at 2024-04-25T14:11:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:49 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/50535132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:49 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3205479806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:49 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2778356517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a[29159]: 2024-04-25T14:11:49.379+0000 7f57a261f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:11:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:49.733+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f36c80637a0 con 0x7f36c8103af0 2024-04-25T14:11:49.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/50535132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3205479806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2778356517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:11:49.766 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: ignoring --setuser ceph since I am not root 2024-04-25T14:11:49.767 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: ignoring --setgroup ceph since I am not root 2024-04-25T14:11:49.767 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-mds[63620]: -- 172.21.15.112:0/2875741232 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d27e397a40 con 0x55d27e3df600 2024-04-25T14:11:49.767 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: starting mds.b at 2024-04-25T14:11:49.767 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h[63305]: ignoring --setuser ceph since I am not root 2024-04-25T14:11:49.767 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h[63305]: ignoring --setgroup ceph since I am not root 2024-04-25T14:11:49.767 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-mds[63309]: -- 172.21.15.112:0/2749878111 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555cb4e4ba40 con 0x555cb4e93600 2024-04-25T14:11:49.767 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h[63305]: starting mds.h at 2024-04-25T14:11:49.768 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e[62989]: ignoring --setuser ceph since I am not root 2024-04-25T14:11:49.768 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e[62989]: ignoring --setgroup ceph since I am not root 2024-04-25T14:11:49.768 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-mds[62993]: -- 172.21.15.112:0/1582301042 <== mon.2 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ad5185a40 con 0x559ad51cd600 2024-04-25T14:11:49.768 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:11:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e[62989]: starting mds.e at 2024-04-25T14:11:50.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.383+0000 7f36c57fa640 1 -- 172.21.15.78:0/1829625726 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f36b80d0330 con 0x7f36c8103af0 2024-04-25T14:11:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f36b00afaf0 msgr2=0x7f36b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f36b00afaf0 0x7f36b00b2000 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f36c819b2f0 tx=0x7f36bc00b390 comp rx=0 tx=0).stop 2024-04-25T14:11:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 msgr2=0x7f36c819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c819a250 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f36b8006010 tx=0x7f36b8004270 comp rx=0 tx=0).stop 2024-04-25T14:11:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 shutdown_connections 2024-04-25T14:11:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c8109a30 0x7f36c81a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36c8104d30 0x7f36c819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f36b00afaf0 0x7f36b00b2000 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 --2- 172.21.15.78:0/1829625726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36c8103af0 0x7f36c819a250 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 >> 172.21.15.78:0/1829625726 conn(0x7f36c80fece0 msgr2=0x7f36c8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 shutdown_connections 2024-04-25T14:11:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.385+0000 7f36ce61a640 1 -- 172.21.15.78:0/1829625726 wait complete. 2024-04-25T14:11:50.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T14:11:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.511+0000 7f648b534640 1 -- 172.21.15.78:0/335971881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 msgr2=0x7f648410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.511+0000 7f648b534640 1 --2- 172.21.15.78:0/335971881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f648410be50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f647c009ca0 tx=0x7f647c02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.511+0000 7f648b534640 1 -- 172.21.15.78:0/335971881 shutdown_connections 2024-04-25T14:11:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.511+0000 7f648b534640 1 --2- 172.21.15.78:0/335971881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f648410be50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.511+0000 7f648b534640 1 --2- 172.21.15.78:0/335971881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484104d30 0x7f6484105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.511+0000 7f648b534640 1 --2- 172.21.15.78:0/335971881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6484103af0 0x7f64841047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.511+0000 7f648b534640 1 -- 172.21.15.78:0/335971881 >> 172.21.15.78:0/335971881 conn(0x7f64840fece0 msgr2=0x7f6484101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 -- 172.21.15.78:0/335971881 shutdown_connections 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 -- 172.21.15.78:0/335971881 wait complete. 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 Processor -- start 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 -- start start 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484103af0 0x7f648419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6484104d30 0x7f648419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f64841a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f648410f620 con 0x7f6484109a30 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f648410f4a0 con 0x7f6484103af0 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.512+0000 7f648b534640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f648410f7a0 con 0x7f6484104d30 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f64841a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:50.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f64841a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44922/0 (socket says 172.21.15.78:44922) 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 -- 172.21.15.78:0/2911499984 learned_addr learned my addr 172.21.15.78:0/2911499984 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f64892d0640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484103af0 0x7f648419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6488acf640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6484104d30 0x7f648419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 -- 172.21.15.78:0/2911499984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6484104d30 msgr2=0x7f648419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6484104d30 0x7f648419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 -- 172.21.15.78:0/2911499984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484103af0 msgr2=0x7f648419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484103af0 0x7f648419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 -- 172.21.15.78:0/2911499984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64841a1d40 con 0x7f6484109a30 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f64892d0640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484103af0 0x7f648419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f6489ad1640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f64841a16d0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f647c0097b0 tx=0x7f647c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f647a7fc640 1 -- 172.21.15.78:0/2911499984 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c003bf0 con 0x7f6484109a30 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f647a7fc640 1 -- 172.21.15.78:0/2911499984 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f647c003d90 con 0x7f6484109a30 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64841a1fd0 con 0x7f6484109a30 2024-04-25T14:11:50.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f647a7fc640 1 -- 172.21.15.78:0/2911499984 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c02fc30 con 0x7f6484109a30 2024-04-25T14:11:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.513+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64841a2430 con 0x7f6484109a30 2024-04-25T14:11:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.514+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648406a140 con 0x7f6484109a30 2024-04-25T14:11:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.516+0000 7f647a7fc640 1 -- 172.21.15.78:0/2911499984 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f647c004040 con 0x7f6484109a30 2024-04-25T14:11:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.516+0000 7f647a7fc640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f64700afaf0 0x7f64700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.516+0000 7f64892d0640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f64700afaf0 0x7f64700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.516+0000 7f647a7fc640 1 -- 172.21.15.78:0/2911499984 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8506+0+0 (secure 0 0 0) 0x7f647c100660 con 0x7f6484109a30 2024-04-25T14:11:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.517+0000 7f64892d0640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f64700afaf0 0x7f64700b2000 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f64800058c0 tx=0x7f6480005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:50.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.520+0000 7f647a7fc640 1 -- 172.21.15.78:0/2911499984 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f647c0046a0 con 0x7f6484109a30 2024-04-25T14:11:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: pgmap v717: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T14:11:50.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2948437010' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:11:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:11:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:11:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2948437010' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:11:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T14:11:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T14:11:50.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:50 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1829625726' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:11:50.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:50.698+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6484108890 con 0x7f6484109a30 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: pgmap v717: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2948437010' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2948437010' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T14:11:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:50 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1829625726' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: pgmap v717: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2948437010' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2948437010' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T14:11:50.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:50 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1829625726' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:11:51.336 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: Cluster is now healthy 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/1829625726' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6834/4182285071,v1:172.21.15.129:6835/4182285071] up:boot 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6834/1872434359,v1:172.21.15.112:6835/1872434359] up:boot 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6836/3907051683,v1:172.21.15.112:6838/3907051683] up:boot 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.112:6837/1907149811,v1:172.21.15.112:6839/1907149811] up:boot 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6836/2194285541,v1:172.21.15.129:6838/2194285541] up:boot 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: mds.? [v2:172.21.15.129:6837/2143354647,v1:172.21.15.129:6839/2143354647] up:boot 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: fsmap 12 up:standby 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:11:51.337 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:11:51.338 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:11:51.338 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:11:51.338 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: pgmap v719: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:11:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.390+0000 7f647a7fc640 1 -- 172.21.15.78:0/2911499984 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) ==== 181+0+0 (secure 0 0 0) 0x7f6484108890 con 0x7f6484109a30 2024-04-25T14:11:51.390 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-04-25T14:11:51.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.393+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f64700afaf0 msgr2=0x7f64700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:51.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.393+0000 7f648b534640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f64700afaf0 0x7f64700b2000 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f64800058c0 tx=0x7f6480005850 comp rx=0 tx=0).stop 2024-04-25T14:11:51.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.393+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 msgr2=0x7f64841a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:51.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.393+0000 7f648b534640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f64841a16d0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f647c0097b0 tx=0x7f647c031840 comp rx=0 tx=0).stop 2024-04-25T14:11:51.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.393+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 shutdown_connections 2024-04-25T14:11:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.393+0000 7f648b534640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f64700afaf0 0x7f64700b2000 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.393+0000 7f648b534640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6484109a30 0x7f64841a16d0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.394+0000 7f648b534640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6484104d30 0x7f648419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.394+0000 7f648b534640 1 --2- 172.21.15.78:0/2911499984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6484103af0 0x7f648419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.394+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 >> 172.21.15.78:0/2911499984 conn(0x7f64840fece0 msgr2=0x7f6484105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.394+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 shutdown_connections 2024-04-25T14:11:51.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.394+0000 7f648b534640 1 -- 172.21.15.78:0/2911499984 wait complete. 2024-04-25T14:11:51.408 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T14:11:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 -- 172.21.15.78:0/3188957586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec102a00 msgr2=0x7f9dec103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3188957586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec102a00 0x7f9dec103700 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009ca0 tx=0x7f9ddc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 -- 172.21.15.78:0/3188957586 shutdown_connections 2024-04-25T14:11:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3188957586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dec108940 0x7f9dec10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3188957586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dec103c40 0x7f9dec104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3188957586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec102a00 0x7f9dec103700 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 -- 172.21.15.78:0/3188957586 >> 172.21.15.78:0/3188957586 conn(0x7f9dec0fdbf0 msgr2=0x7f9dec1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 -- 172.21.15.78:0/3188957586 shutdown_connections 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.526+0000 7f9df0abf640 1 -- 172.21.15.78:0/3188957586 wait complete. 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 Processor -- start 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 -- start start 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dec102a00 0x7f9dec19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dec103c40 0x7f9dec19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec108940 0x7f9dec1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec10e500 con 0x7f9dec108940 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec10e380 con 0x7f9dec103c40 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9df0abf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec10e680 con 0x7f9dec102a00 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec108940 0x7f9dec1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:51.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec108940 0x7f9dec1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44940/0 (socket says 172.21.15.78:44940) 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 -- 172.21.15.78:0/3655432601 learned_addr learned my addr 172.21.15.78:0/3655432601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dea59c640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dec102a00 0x7f9dec19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 -- 172.21.15.78:0/3655432601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dec102a00 msgr2=0x7f9dec19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dec102a00 0x7f9dec19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9de9d9b640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dec103c40 0x7f9dec19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 -- 172.21.15.78:0/3655432601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dec103c40 msgr2=0x7f9dec19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dec103c40 0x7f9dec19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 -- 172.21.15.78:0/3655432601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec1a1d70 con 0x7f9dec108940 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dea59c640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dec102a00 0x7f9dec19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.527+0000 7f9dead9d640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec108940 0x7f9dec1a1700 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f9de400e970 tx=0x7f9de400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.528+0000 7f9ddb7fe640 1 -- 172.21.15.78:0/3655432601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de400ccc0 con 0x7f9dec108940 2024-04-25T14:11:51.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.528+0000 7f9ddb7fe640 1 -- 172.21.15.78:0/3655432601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de400ce60 con 0x7f9dec108940 2024-04-25T14:11:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.528+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec1a2060 con 0x7f9dec108940 2024-04-25T14:11:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.528+0000 7f9ddb7fe640 1 -- 172.21.15.78:0/3655432601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4010430 con 0x7f9dec108940 2024-04-25T14:11:51.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.528+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec1a2600 con 0x7f9dec108940 2024-04-25T14:11:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.529+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dec1a28c0 con 0x7f9dec108940 2024-04-25T14:11:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.530+0000 7f9ddb7fe640 1 -- 172.21.15.78:0/3655432601 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f9de4004f70 con 0x7f9dec108940 2024-04-25T14:11:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.531+0000 7f9ddb7fe640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9dd40afb70 0x7f9dd40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:11:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.531+0000 7f9ddb7fe640 1 -- 172.21.15.78:0/3655432601 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7952+0+0 (secure 0 0 0) 0x7f9de40df070 con 0x7f9dec108940 2024-04-25T14:11:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.531+0000 7f9dea59c640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9dd40afb70 0x7f9dd40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:11:51.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.531+0000 7f9dea59c640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9dd40afb70 0x7f9dd40b2080 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc000f80 tx=0x7f9ddc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:11:51.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.534+0000 7f9ddb7fe640 1 -- 172.21.15.78:0/3655432601 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9de4053ff0 con 0x7f9dec108940 2024-04-25T14:11:51.663 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: Cluster is now healthy 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/1829625726' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6834/4182285071,v1:172.21.15.129:6835/4182285071] up:boot 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6834/1872434359,v1:172.21.15.112:6835/1872434359] up:boot 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6836/3907051683,v1:172.21.15.112:6838/3907051683] up:boot 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.112:6837/1907149811,v1:172.21.15.112:6839/1907149811] up:boot 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6836/2194285541,v1:172.21.15.129:6838/2194285541] up:boot 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: mds.? [v2:172.21.15.129:6837/2143354647,v1:172.21.15.129:6839/2143354647] up:boot 2024-04-25T14:11:51.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: fsmap 12 up:standby 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: pgmap v719: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:11:51.665 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:51 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2911499984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:11:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:51.712+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9dec1a2a90 con 0x7f9dec108940 2024-04-25T14:11:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: Cluster is now healthy 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/1829625726' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6834/4182285071,v1:172.21.15.129:6835/4182285071] up:boot 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6834/1872434359,v1:172.21.15.112:6835/1872434359] up:boot 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6836/3907051683,v1:172.21.15.112:6838/3907051683] up:boot 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.112:6837/1907149811,v1:172.21.15.112:6839/1907149811] up:boot 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6836/2194285541,v1:172.21.15.129:6838/2194285541] up:boot 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: mds.? [v2:172.21.15.129:6837/2143354647,v1:172.21.15.129:6839/2143354647] up:boot 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: fsmap 12 up:standby 2024-04-25T14:11:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:11:51.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:11:51.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:11:51.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:11:51.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:11:51.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='mgr.14152 172.21.15.78:0/4212647739' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:11:51.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: pgmap v719: 97 pgs: 97 active+clean; 4.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T14:11:51.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:51 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2911499984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:11:51.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:51 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2911499984' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:11:52.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.392+0000 7f9ddb7fe640 1 -- 172.21.15.78:0/3655432601 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) ==== 169+0+0 (secure 0 0 0) 0x7f9dec1a2a90 con 0x7f9dec108940 2024-04-25T14:11:52.392 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-04-25T14:11:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9dd40afb70 msgr2=0x7f9dd40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9dd40afb70 0x7f9dd40b2080 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc000f80 tx=0x7f9ddc004a40 comp rx=0 tx=0).stop 2024-04-25T14:11:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec108940 msgr2=0x7f9dec1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:11:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec108940 0x7f9dec1a1700 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f9de400e970 tx=0x7f9de400ed70 comp rx=0 tx=0).stop 2024-04-25T14:11:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 shutdown_connections 2024-04-25T14:11:52.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:6800/3547239980,v1:172.21.15.78:6801/3547239980] conn(0x7f9dd40afb70 0x7f9dd40b2080 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dec108940 0x7f9dec1a1700 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dec103c40 0x7f9dec19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 --2- 172.21.15.78:0/3655432601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dec102a00 0x7f9dec19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:11:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 >> 172.21.15.78:0/3655432601 conn(0x7f9dec0fdbf0 msgr2=0x7f9dec109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:11:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.395+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 shutdown_connections 2024-04-25T14:11:52.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T14:11:52.396+0000 7f9df0abf640 1 -- 172.21.15.78:0/3655432601 wait complete. 2024-04-25T14:11:52.414 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:11:52.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:11:52.471 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:11:52.502 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T14:11:52.538 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:11:52.554 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:11:52.571 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:11:52.588 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T14:11:52.589 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T14:11:52.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:11:52.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:52 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/2911499984' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:11:52.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:52 smithi129 ceph-mon[32569]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T14:11:52.664 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:52 smithi129 ceph-mon[32569]: from='client.? 172.21.15.78:0/3655432601' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:11:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:52 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/2911499984' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:11:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:52 smithi078 ceph-mon[29163]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T14:11:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:52 smithi078 ceph-mon[29163]: from='client.? 172.21.15.78:0/3655432601' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:11:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:52 smithi078 systemd[1]: Stopping Ceph mon.a for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:11:52.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:52 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/2911499984' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T14:11:52.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:52 smithi112 ceph-mon[32617]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T14:11:52.766 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:52 smithi112 ceph-mon[32617]: from='client.? 172.21.15.78:0/3655432601' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T14:11:53.195 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a[29159]: 2024-04-25T14:11:52.886+0000 7f57a7e2a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:11:53.195 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a[29159]: 2024-04-25T14:11:52.886+0000 7f57a7e2a700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T14:11:53.195 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:52 smithi078 podman[97792]: 2024-04-25 14:11:52.928118799 +0000 UTC m=+0.138939412 container died 70d96a743c37bd08859a2a5ee84654eca3cf862a27547d231188572f64ba1e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:11:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:53 smithi078 podman[97792]: 2024-04-25 14:11:53.320941399 +0000 UTC m=+0.531762013 container cleanup 70d96a743c37bd08859a2a5ee84654eca3cf862a27547d231188572f64ba1e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:11:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 14:11:53 smithi078 bash[97792]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:11:53.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T14:11:53.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:11:53.803 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T14:11:53.803 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T14:11:53.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.b 2024-04-25T14:11:54.111 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:53 smithi112 systemd[1]: Stopping Ceph mon.b for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:11:54.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-b[32613]: 2024-04-25T14:11:54.111+0000 7f8f82a58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:11:54.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-b[32613]: 2024-04-25T14:11:54.111+0000 7f8f82a58700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:11:54.376 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:54 smithi112 podman[68881]: 2024-04-25 14:11:54.346142246 +0000 UTC m=+0.347427552 container died 874de52201275d945cb2a13a96b79afdf5d24169a2b4f4c70aaa5c7c3927004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:11:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:54 smithi112 podman[68881]: 2024-04-25 14:11:54.737622551 +0000 UTC m=+0.738907864 container cleanup 874de52201275d945cb2a13a96b79afdf5d24169a2b4f4c70aaa5c7c3927004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:11:55.016 INFO:journalctl@ceph.mon.b.smithi112.stdout:Apr 25 14:11:54 smithi112 bash[68881]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-b 2024-04-25T14:11:55.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T14:11:55.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:11:55.222 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T14:11:55.223 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T14:11:55.223 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.c 2024-04-25T14:11:55.505 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:55 smithi129 systemd[1]: Stopping Ceph mon.c for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:11:55.906 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-c[32565]: 2024-04-25T14:11:55.504+0000 7fea719eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:11:55.906 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-c[32565]: 2024-04-25T14:11:55.504+0000 7fea719eb700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T14:11:55.906 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:55 smithi129 podman[67743]: 2024-04-25 14:11:55.597746407 +0000 UTC m=+0.179543459 container died 004acab6f6315a39960c33efe84b15d4eaffa935ebef7f5910b43048115aa415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T14:11:56.163 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:56 smithi129 podman[67743]: 2024-04-25 14:11:56.006924546 +0000 UTC m=+0.588721597 container cleanup 004acab6f6315a39960c33efe84b15d4eaffa935ebef7f5910b43048115aa415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T14:11:56.164 INFO:journalctl@ceph.mon.c.smithi129.stdout:Apr 25 14:11:56 smithi129 bash[67743]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mon-c 2024-04-25T14:11:56.424 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T14:11:56.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:11:56.466 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T14:11:56.466 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T14:11:56.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:11:56.866 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 14:11:56 smithi078 systemd[1]: Stopping Ceph mgr.x for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:11:57.193 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 14:11:56 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T14:11:56.866+0000 7f28187bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:11:57.193 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 14:11:56 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x[29391]: 2024-04-25T14:11:56.866+0000 7f28187bc700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T14:11:57.193 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 14:11:56 smithi078 podman[97913]: 2024-04-25 14:11:56.897901567 +0000 UTC m=+0.240042536 container died b162b50cfff90d495e0274c0c4c9cc5b649107faa35714e95c6b54530f83a93a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T14:11:57.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 14:11:57 smithi078 podman[97913]: 2024-04-25 14:11:57.269088498 +0000 UTC m=+0.611229453 container cleanup b162b50cfff90d495e0274c0c4c9cc5b649107faa35714e95c6b54530f83a93a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T14:11:57.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 14:11:57 smithi078 bash[97913]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-x 2024-04-25T14:11:57.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T14:11:57.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:11:57.717 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T14:11:57.717 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T14:11:57.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.y 2024-04-25T14:11:58.016 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 14:11:57 smithi112 systemd[1]: Stopping Ceph mgr.y for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:11:58.385 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 14:11:58 smithi112 podman[69014]: 2024-04-25 14:11:58.096686342 +0000 UTC m=+0.186853576 container died 93c7a9abd610d6956018b9e7730b8f537cc265f5633d3e0a21f944ade0ba2232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T14:11:58.766 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 14:11:58 smithi112 podman[69014]: 2024-04-25 14:11:58.493769292 +0000 UTC m=+0.583936535 container cleanup 93c7a9abd610d6956018b9e7730b8f537cc265f5633d3e0a21f944ade0ba2232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:11:58.766 INFO:journalctl@ceph.mgr.y.smithi112.stdout:Apr 25 14:11:58 smithi112 bash[69014]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-y 2024-04-25T14:11:58.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T14:11:58.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:11:58.999 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T14:11:59.000 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T14:11:59.000 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.z 2024-04-25T14:11:59.323 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 14:11:59 smithi129 systemd[1]: Stopping Ceph mgr.z for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:11:59.605 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 14:11:59 smithi129 podman[67868]: 2024-04-25 14:11:59.324531521 +0000 UTC m=+0.127288147 container died 5a240cc8953ee241528b7e036c3f69fb65eb2ff48a506f770ac50272319d2ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T14:11:59.913 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 14:11:59 smithi129 podman[67868]: 2024-04-25 14:11:59.713260292 +0000 UTC m=+0.516016961 container cleanup 5a240cc8953ee241528b7e036c3f69fb65eb2ff48a506f770ac50272319d2ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:11:59.914 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Apr 25 14:11:59 smithi129 bash[67868]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mgr-z 2024-04-25T14:12:00.076 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T14:12:00.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:00.109 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T14:12:00.109 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T14:12:00.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.0 2024-04-25T14:12:00.469 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:00 smithi078 systemd[1]: Stopping Ceph osd.0 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:00.969 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0[43933]: 2024-04-25T14:12:00.527+0000 7f0e55f9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:00.969 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0[43933]: 2024-04-25T14:12:00.527+0000 7f0e55f9d700 -1 osd.0 80 *** Got signal Terminated *** 2024-04-25T14:12:00.969 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0[43933]: 2024-04-25T14:12:00.527+0000 7f0e55f9d700 -1 osd.0 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:06.182 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:05 smithi078 podman[98036]: 2024-04-25 14:12:05.858034957 +0000 UTC m=+5.553830489 container died 17e7001d85f55f98f15dba83a1291dd090a23ad0e900312d232ef942b6efb753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:12:06.469 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:06 smithi078 podman[98036]: 2024-04-25 14:12:06.283101367 +0000 UTC m=+5.978896910 container cleanup 17e7001d85f55f98f15dba83a1291dd090a23ad0e900312d232ef942b6efb753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:12:06.469 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:06 smithi078 bash[98036]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0 2024-04-25T14:12:06.952 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:06 smithi078 podman[98053]: 2024-04-25 14:12:06.51826886 +0000 UTC m=+0.658260459 container remove 17e7001d85f55f98f15dba83a1291dd090a23ad0e900312d232ef942b6efb753 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:12:07.219 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:06 smithi078 podman[98142]: 2024-04-25 14:12:06.856031794 +0000 UTC m=+0.018031780 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:12:07.219 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:06 smithi078 podman[98142]: 2024-04-25 14:12:06.961157366 +0000 UTC m=+0.123157379 container create 7a9cdeb2274d1ef44a457055deecd0805406a105a2f95482689ccbaaa8bfcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True) 2024-04-25T14:12:07.540 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:07 smithi078 podman[98142]: 2024-04-25 14:12:07.239323072 +0000 UTC m=+0.401323039 container init 7a9cdeb2274d1ef44a457055deecd0805406a105a2f95482689ccbaaa8bfcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:12:07.540 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:07 smithi078 podman[98142]: 2024-04-25 14:12:07.24256013 +0000 UTC m=+0.404560100 container start 7a9cdeb2274d1ef44a457055deecd0805406a105a2f95482689ccbaaa8bfcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T14:12:07.540 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:07 smithi078 podman[98142]: 2024-04-25 14:12:07.347890036 +0000 UTC m=+0.509890010 container attach 7a9cdeb2274d1ef44a457055deecd0805406a105a2f95482689ccbaaa8bfcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T14:12:07.909 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 14:12:07 smithi078 podman[98142]: 2024-04-25 14:12:07.541598683 +0000 UTC m=+0.703598651 container died 7a9cdeb2274d1ef44a457055deecd0805406a105a2f95482689ccbaaa8bfcf7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T14:12:08.186 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T14:12:08.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:08.227 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T14:12:08.227 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T14:12:08.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.1 2024-04-25T14:12:08.516 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:08 smithi112 systemd[1]: Stopping Ceph osd.1 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:09.016 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1[37221]: 2024-04-25T14:12:08.539+0000 7fe4749c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:09.016 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1[37221]: 2024-04-25T14:12:08.539+0000 7fe4749c7700 -1 osd.1 80 *** Got signal Terminated *** 2024-04-25T14:12:09.016 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1[37221]: 2024-04-25T14:12:08.539+0000 7fe4749c7700 -1 osd.1 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:14.325 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:14 smithi112 podman[69138]: 2024-04-25 14:12:14.029988822 +0000 UTC m=+5.617358028 container died 86eff2d8504d9ac7cc04725af97c9bfe3f1f48e0ff4aa934f10db3321ecf4df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T14:12:14.683 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:14 smithi112 podman[69138]: 2024-04-25 14:12:14.423588712 +0000 UTC m=+6.010957921 container cleanup 86eff2d8504d9ac7cc04725af97c9bfe3f1f48e0ff4aa934f10db3321ecf4df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T14:12:14.683 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:14 smithi112 bash[69138]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1 2024-04-25T14:12:14.991 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:14 smithi112 podman[69155]: 2024-04-25 14:12:14.683713308 +0000 UTC m=+0.650287679 container remove 86eff2d8504d9ac7cc04725af97c9bfe3f1f48e0ff4aa934f10db3321ecf4df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T14:12:15.253 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:14 smithi112 podman[69244]: 2024-04-25 14:12:14.895489252 +0000 UTC m=+0.017588168 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:12:15.253 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:15 smithi112 podman[69244]: 2024-04-25 14:12:15.009792728 +0000 UTC m=+0.131891691 container create 623f8c49ed2b88a87524e865f3162b10c8318452271a5f49f37ab3a7ec7d222e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T14:12:15.253 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:15 smithi112 podman[69244]: 2024-04-25 14:12:15.25412581 +0000 UTC m=+0.376224728 container init 623f8c49ed2b88a87524e865f3162b10c8318452271a5f49f37ab3a7ec7d222e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:12:15.516 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:15 smithi112 podman[69244]: 2024-04-25 14:12:15.25737668 +0000 UTC m=+0.379475598 container start 623f8c49ed2b88a87524e865f3162b10c8318452271a5f49f37ab3a7ec7d222e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T14:12:15.516 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:15 smithi112 podman[69244]: 2024-04-25 14:12:15.329004987 +0000 UTC m=+0.451103921 container attach 623f8c49ed2b88a87524e865f3162b10c8318452271a5f49f37ab3a7ec7d222e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T14:12:15.831 INFO:journalctl@ceph.osd.1.smithi112.stdout:Apr 25 14:12:15 smithi112 podman[69244]: 2024-04-25 14:12:15.551434898 +0000 UTC m=+0.673533809 container died 623f8c49ed2b88a87524e865f3162b10c8318452271a5f49f37ab3a7ec7d222e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:12:16.155 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T14:12:16.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:16.193 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T14:12:16.193 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T14:12:16.193 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.2 2024-04-25T14:12:16.517 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:16 smithi129 systemd[1]: Stopping Ceph osd.2 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:16.913 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2[37676]: 2024-04-25T14:12:16.517+0000 7f631c05d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:16.914 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2[37676]: 2024-04-25T14:12:16.517+0000 7f631c05d700 -1 osd.2 80 *** Got signal Terminated *** 2024-04-25T14:12:16.914 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2[37676]: 2024-04-25T14:12:16.517+0000 7f631c05d700 -1 osd.2 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:22.302 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:22 smithi129 podman[67992]: 2024-04-25 14:12:22.011373545 +0000 UTC m=+5.625875532 container died 8ce11e25e84b0b1a105011e9f5e04c2070c7accaab60b349e45c0535d8baa0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:12:22.644 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:22 smithi129 podman[67992]: 2024-04-25 14:12:22.402137129 +0000 UTC m=+6.016639117 container cleanup 8ce11e25e84b0b1a105011e9f5e04c2070c7accaab60b349e45c0535d8baa0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T14:12:22.644 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:22 smithi129 bash[67992]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2 2024-04-25T14:12:22.913 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:22 smithi129 podman[68022]: 2024-04-25 14:12:22.64533809 +0000 UTC m=+0.632588744 container remove 8ce11e25e84b0b1a105011e9f5e04c2070c7accaab60b349e45c0535d8baa0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T14:12:23.331 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:22 smithi129 podman[68112]: 2024-04-25 14:12:22.877772586 +0000 UTC m=+0.018188728 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:12:23.331 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:23 smithi129 podman[68112]: 2024-04-25 14:12:23.01302317 +0000 UTC m=+0.153439280 container create 32cc0dda3d653246b9c0ebcd48fdf5443e6b3fd1b519f6c26d2dacb53d9a3869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T14:12:23.331 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:23 smithi129 podman[68112]: 2024-04-25 14:12:23.215354675 +0000 UTC m=+0.355770786 container init 32cc0dda3d653246b9c0ebcd48fdf5443e6b3fd1b519f6c26d2dacb53d9a3869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-25T14:12:23.331 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:23 smithi129 podman[68112]: 2024-04-25 14:12:23.218331896 +0000 UTC m=+0.358748011 container start 32cc0dda3d653246b9c0ebcd48fdf5443e6b3fd1b519f6c26d2dacb53d9a3869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T14:12:23.331 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:23 smithi129 podman[68112]: 2024-04-25 14:12:23.332088557 +0000 UTC m=+0.472504677 container attach 32cc0dda3d653246b9c0ebcd48fdf5443e6b3fd1b519f6c26d2dacb53d9a3869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:12:23.664 INFO:journalctl@ceph.osd.2.smithi129.stdout:Apr 25 14:12:23 smithi129 podman[68112]: 2024-04-25 14:12:23.513624061 +0000 UTC m=+0.654040182 container died 32cc0dda3d653246b9c0ebcd48fdf5443e6b3fd1b519f6c26d2dacb53d9a3869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:12:24.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T14:12:24.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:24.152 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T14:12:24.152 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T14:12:24.152 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.3 2024-04-25T14:12:24.469 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:24 smithi078 systemd[1]: Stopping Ceph osd.3 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:24.969 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3[50449]: 2024-04-25T14:12:24.479+0000 7f6a58ab0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:24.969 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3[50449]: 2024-04-25T14:12:24.479+0000 7f6a58ab0700 -1 osd.3 80 *** Got signal Terminated *** 2024-04-25T14:12:24.969 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:24 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3[50449]: 2024-04-25T14:12:24.479+0000 7f6a58ab0700 -1 osd.3 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:26.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:26 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:26.376+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:27.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:27 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:27.347+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:28.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:27.726+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:28.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:28.226+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:28.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:28 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:28.395+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:28.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:28 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:28.422+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:29.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:28.758+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:29.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:29.234+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:29.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:29 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:29.384+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:29.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:29 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:29.430+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:30.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:29.763+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:30.219 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:29 smithi078 podman[98259]: 2024-04-25 14:12:29.922533373 +0000 UTC m=+5.573563068 container died 7caab565cf9315288087ab26a0ebcba90fd866e29f48bc8bdadb6dd5709bbbfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:12:30.547 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:30 smithi078 podman[98259]: 2024-04-25 14:12:30.314023219 +0000 UTC m=+5.965052939 container cleanup 7caab565cf9315288087ab26a0ebcba90fd866e29f48bc8bdadb6dd5709bbbfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T14:12:30.547 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:30 smithi078 bash[98259]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3 2024-04-25T14:12:30.548 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:30 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:30.438+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:30.548 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:30 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:30.403+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:30.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:30.264+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:30.882 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:30 smithi078 podman[98278]: 2024-04-25 14:12:30.54872008 +0000 UTC m=+0.626059627 container remove 7caab565cf9315288087ab26a0ebcba90fd866e29f48bc8bdadb6dd5709bbbfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T14:12:31.151 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:30 smithi078 podman[98374]: 2024-04-25 14:12:30.785879346 +0000 UTC m=+0.017777542 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:12:31.151 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:30 smithi078 podman[98374]: 2024-04-25 14:12:30.908193625 +0000 UTC m=+0.140091808 container create 2e140815a4299dcd7841746da1bc2f93336766ef6dae2f18e189de643c27cd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:12:31.151 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:31 smithi078 podman[98374]: 2024-04-25 14:12:31.152241114 +0000 UTC m=+0.384139297 container init 2e140815a4299dcd7841746da1bc2f93336766ef6dae2f18e189de643c27cd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:12:31.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:30.775+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:31.445 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:31 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:31.387+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:31.445 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:31 smithi078 podman[98374]: 2024-04-25 14:12:31.155386053 +0000 UTC m=+0.387284246 container start 2e140815a4299dcd7841746da1bc2f93336766ef6dae2f18e189de643c27cd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:12:31.445 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:31 smithi078 podman[98374]: 2024-04-25 14:12:31.252700505 +0000 UTC m=+0.484598705 container attach 2e140815a4299dcd7841746da1bc2f93336766ef6dae2f18e189de643c27cd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T14:12:31.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:31 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:31.113+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:31.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:31.261+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:31.719 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 14:12:31 smithi078 podman[98374]: 2024-04-25 14:12:31.446425513 +0000 UTC m=+0.678323716 container died 2e140815a4299dcd7841746da1bc2f93336766ef6dae2f18e189de643c27cd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T14:12:31.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:31 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:31.451+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:32.123 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T14:12:32.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:31.775+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:32.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:32.167 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T14:12:32.167 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T14:12:32.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.4 2024-04-25T14:12:32.485 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:32 smithi112 systemd[1]: Stopping Ceph osd.4 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:32.486 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:32 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:32.137+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:32.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:32.260+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:32.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:32 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:32.409+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:32.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:32 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:32.455+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:32.766 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:32 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4[44177]: 2024-04-25T14:12:32.485+0000 7f6929673700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:32.766 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:32 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4[44177]: 2024-04-25T14:12:32.485+0000 7f6929673700 -1 osd.4 80 *** Got signal Terminated *** 2024-04-25T14:12:32.766 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:32 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4[44177]: 2024-04-25T14:12:32.485+0000 7f6929673700 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:33.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:32.801+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:33.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:33 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:33.140+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:33.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:33 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:33.225+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:33.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:33 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:33.424+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:33.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:33 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:33.372+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:34.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:33 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:33.825+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:34.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:34 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:34.150+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:34.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:34 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:34.219+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:34.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:34 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:34.413+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:34.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:34 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:34.359+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:35.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:34 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:34.850+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:35.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:35 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:35.178+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:35.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:35 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:35.206+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:35.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:35.415+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:35.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:35.324+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:35.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:35.324+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:36.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:35 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:35.893+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:36.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:36 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:36.128+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:36.516 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:36 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4[44177]: 2024-04-25T14:12:36.094+0000 7f6921c21700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.321433+0000 front 2024-04-25T14:12:11.321465+0000 (oldest deadline 2024-04-25T14:12:36.021613+0000) 2024-04-25T14:12:36.620 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:36 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:36.399+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:36.621 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:36 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:36.301+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:36.621 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:36 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:36.301+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:36.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:36 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:36.164+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:37.413 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:37 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:37.161+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:37.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:36 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:36.922+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:37.516 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:37 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4[44177]: 2024-04-25T14:12:37.125+0000 7f6921c21700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.321433+0000 front 2024-04-25T14:12:11.321465+0000 (oldest deadline 2024-04-25T14:12:36.021613+0000) 2024-04-25T14:12:37.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:37 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:37.126+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:37.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:37 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:37.126+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:37.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:37.441+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:37.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:37.441+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:37.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:37.282+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:37.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:37 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:37.282+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:38.102 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:37 smithi112 podman[69360]: 2024-04-25 14:12:37.827425859 +0000 UTC m=+5.465997590 container died f5253fa989a552be07e2ea85aa28ae5c7fcecf290f9bde05eef1c8b3461fef5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T14:12:38.413 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:37 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:37.953+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:38.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:38 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:38.162+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:38.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:38 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:38.162+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:38.461 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:38 smithi112 podman[69360]: 2024-04-25 14:12:38.211016342 +0000 UTC m=+5.849588074 container cleanup f5253fa989a552be07e2ea85aa28ae5c7fcecf290f9bde05eef1c8b3461fef5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:12:38.461 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:38 smithi112 bash[69360]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4 2024-04-25T14:12:38.462 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:38 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:38.150+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:38.462 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:38 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:38.150+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:38.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:38 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:38.275+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:38.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:38 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:38.276+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:38.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:38 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:38.411+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:38.720 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:38 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:38.411+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:38.766 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:38 smithi112 podman[69378]: 2024-04-25 14:12:38.462625739 +0000 UTC m=+0.634913323 container remove f5253fa989a552be07e2ea85aa28ae5c7fcecf290f9bde05eef1c8b3461fef5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:12:39.035 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:38 smithi112 podman[69471]: 2024-04-25 14:12:38.6827784 +0000 UTC m=+0.018086171 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:12:39.035 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:38 smithi112 podman[69471]: 2024-04-25 14:12:38.805219007 +0000 UTC m=+0.140526755 container create fb29621c9b19910da32d50fcf2b07a9215cad05aae6475d826f2728f0705d238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:12:39.213 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:38 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:38.959+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:39.213 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:38 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:38.959+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:39.213 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:39 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:39.125+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:39.214 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:39 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:39.125+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:39.382 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:39 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:39.035+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:39.383 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:39 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:39.107+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:39.383 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:39 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:39.107+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:39.383 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:39 smithi112 podman[69471]: 2024-04-25 14:12:39.091200124 +0000 UTC m=+0.426507878 container init fb29621c9b19910da32d50fcf2b07a9215cad05aae6475d826f2728f0705d238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:12:39.383 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:39 smithi112 podman[69471]: 2024-04-25 14:12:39.09418616 +0000 UTC m=+0.429493908 container start fb29621c9b19910da32d50fcf2b07a9215cad05aae6475d826f2728f0705d238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:12:39.383 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:39 smithi112 podman[69471]: 2024-04-25 14:12:39.19129248 +0000 UTC m=+0.526600244 container attach fb29621c9b19910da32d50fcf2b07a9215cad05aae6475d826f2728f0705d238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T14:12:39.663 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:39 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:39.213+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:39.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:39.457+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:39.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:39.457+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:39.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:39.288+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:39.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:39 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:39.288+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:39.734 INFO:journalctl@ceph.osd.4.smithi112.stdout:Apr 25 14:12:39 smithi112 podman[69471]: 2024-04-25 14:12:39.384141506 +0000 UTC m=+0.719449259 container died fb29621c9b19910da32d50fcf2b07a9215cad05aae6475d826f2728f0705d238 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T14:12:40.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T14:12:40.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:40.077 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T14:12:40.077 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T14:12:40.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.5 2024-04-25T14:12:40.222 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:39 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:39.929+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:40.222 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:39 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:39.929+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:40.223 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:40.078+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:40.223 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:40.078+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:40.223 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:40 smithi129 systemd[1]: Stopping Ceph osd.5 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:40.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:40 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:40.033+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:40.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:40 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:40.150+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:40.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:40 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:40.150+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:40.663 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:40.221+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:40.663 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:40.371+0000 7ff9d564f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:40.664 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:40.371+0000 7ff9d564f700 -1 osd.5 80 *** Got signal Terminated *** 2024-04-25T14:12:40.664 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:40.371+0000 7ff9d564f700 -1 osd.5 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:40.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:40.444+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:40.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:40.444+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:40.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:40.276+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:40.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:40.276+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:41.258 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:41 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:41.102+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:41.259 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:41 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:41.102+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:41.259 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:40.936+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:41.259 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:40 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:40.936+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:41.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:40 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:40.995+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:41.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:41 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:41.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:41.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:41 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:41.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:41.663 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:41 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:41.258+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:41.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:41 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:41.424+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:41.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:41 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:41.424+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:41.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:41 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:41.238+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:41.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:41 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:41.238+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:42.231 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:42 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:42.115+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:42.231 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:42 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:42.115+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:42.231 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:41 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:41.974+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:42.232 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:41 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:41.974+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:42.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:41 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:41.972+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:42.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:42 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:42.184+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:42.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:42 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:42.184+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:42.663 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:42 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:42.231+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:42.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:42 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:42.409+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:42.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:42 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:42.409+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:42.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:42 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:42.270+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:42.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:42 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:42.270+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:43.230 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:42 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:42.941+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:43.231 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:42 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:42.941+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:43.413 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:43 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:43.082+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:43.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:43 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:43.082+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:43.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:43 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:43.192+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:43.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:42 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:42.987+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:43.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:42 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:42.987+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:43.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:43 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:43.231+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:43.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:43 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:43.231+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:43.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:43 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:43.402+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:43.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:43 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:43.402+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:43.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:43 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:43.299+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:43.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:43 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:43.299+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:44.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:43 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:43.942+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:44.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:43 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:43.942+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:44.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:44 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:44.183+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:44.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:44 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:44.183+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:44.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:44 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:44.183+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:44.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:44 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:44.221+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:44.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:44 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:44.111+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:44.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:44 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:44.111+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:44.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:43 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:43.982+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:44.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:43 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:43.982+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:44.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:43 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:43.982+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:44.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:44 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:44.377+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:44.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:44 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:44.377+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:44.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:44 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:44.257+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:44.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:44 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:44.257+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:45.215 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:44 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:44.934+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:45.215 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:44 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:44.934+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:45.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:45.208+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:45.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:44 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:44.960+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:45.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:44 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:44.960+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:45.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:44 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:44.960+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:45.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:45.137+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.606177+0000 front 2024-04-25T14:12:04.605981+0000 (oldest deadline 2024-04-25T14:12:27.506029+0000) 2024-04-25T14:12:45.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:45.137+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.108119+0000 front 2024-04-25T14:12:12.108170+0000 (oldest deadline 2024-04-25T14:12:37.408082+0000) 2024-04-25T14:12:45.414 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5[44175]: 2024-04-25T14:12:45.137+0000 7ff9cdbfd700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.909074+0000 front 2024-04-25T14:12:20.909134+0000 (oldest deadline 2024-04-25T14:12:44.409334+0000) 2024-04-25T14:12:45.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:45 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:45.215+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:45.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:45 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:45.215+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:45.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:45 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:45.215+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:45.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:45 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:45.328+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:45.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:45 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:45.328+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:45.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:45 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:45.328+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:45.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:45 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:45.278+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:45.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:45 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:45.278+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:45.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:45 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:45.278+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:46.015 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:45 smithi129 podman[68216]: 2024-04-25 14:12:45.726202031 +0000 UTC m=+5.455542409 container died 37b05d34c6fc2c60e62a04e75ec5112d3792a78206a4371e9c45e3530ab3206f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T14:12:46.015 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:45.939+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:46.015 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:45.939+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:46.015 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:45 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:45.939+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:46.268 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:45 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:45.961+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:46.268 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:45 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:45.961+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:46.268 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:46 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:46.188+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:46.268 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:46 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:46.188+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:46.268 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:46 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:46.188+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:46.356 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:46 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:46.198+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:46.356 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:46 smithi129 podman[68216]: 2024-04-25 14:12:46.114576735 +0000 UTC m=+5.843917113 container cleanup 37b05d34c6fc2c60e62a04e75ec5112d3792a78206a4371e9c45e3530ab3206f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T14:12:46.356 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:46 smithi129 bash[68216]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5 2024-04-25T14:12:46.650 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:46 smithi129 podman[68233]: 2024-04-25 14:12:46.357535994 +0000 UTC m=+0.631371784 container remove 37b05d34c6fc2c60e62a04e75ec5112d3792a78206a4371e9c45e3530ab3206f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:12:46.677 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:46.283+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:46.677 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:46.283+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:46.677 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:46.283+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:46.678 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:46.294+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:46.678 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:46.294+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:46.678 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:46 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:46.294+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:46.914 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:46 smithi129 podman[68322]: 2024-04-25 14:12:46.554817117 +0000 UTC m=+0.017420095 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:12:46.914 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:46 smithi129 podman[68322]: 2024-04-25 14:12:46.6838234 +0000 UTC m=+0.146426405 container create 6a2370e9a46b351e7a259ab1f058a7cd57d8909fc38bf33c039baa60dda7ae27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T14:12:46.914 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:46 smithi129 podman[68322]: 2024-04-25 14:12:46.894712101 +0000 UTC m=+0.357315077 container init 6a2370e9a46b351e7a259ab1f058a7cd57d8909fc38bf33c039baa60dda7ae27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T14:12:46.914 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:46 smithi129 podman[68322]: 2024-04-25 14:12:46.897859673 +0000 UTC m=+0.360462637 container start 6a2370e9a46b351e7a259ab1f058a7cd57d8909fc38bf33c039baa60dda7ae27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:12:47.186 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:47 smithi129 podman[68322]: 2024-04-25 14:12:47.003136421 +0000 UTC m=+0.465739400 container attach 6a2370e9a46b351e7a259ab1f058a7cd57d8909fc38bf33c039baa60dda7ae27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T14:12:47.187 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:47 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:47.154+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:47.187 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:47 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:47.154+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:47.187 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:46 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:46.932+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:47.187 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:46 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:46.932+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:47.187 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:46 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:46.932+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:47.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:46 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:46.977+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:47.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:46 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:46.977+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:47.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:47 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:47.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:47.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:47 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:47.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:47.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:47 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:47.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:47.499 INFO:journalctl@ceph.osd.5.smithi129.stdout:Apr 25 14:12:47 smithi129 podman[68322]: 2024-04-25 14:12:47.187914352 +0000 UTC m=+0.650517307 container died 6a2370e9a46b351e7a259ab1f058a7cd57d8909fc38bf33c039baa60dda7ae27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T14:12:47.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:47 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:47.276+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:47.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:47 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:47.276+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:47.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:47 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:47.276+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:47.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:47 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:47.242+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:47.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:47 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:47.242+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:47.720 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:47 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:47.242+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:47.799 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T14:12:47.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:47.838 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T14:12:47.838 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T14:12:47.838 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.6 2024-04-25T14:12:48.142 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:47 smithi078 systemd[1]: Stopping Ceph osd.6 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:48.144 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:47 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:47.921+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:48.144 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:47 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:47.921+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:48.144 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:47 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:47.921+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:48.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:48 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:47.999+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:48.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:48 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:47.999+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:48.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:48 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:48.133+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:48.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:48 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:48.133+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:48.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:48 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:48.133+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:48.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:48 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:48.144+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:48.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:48 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:48.144+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:48.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:48.272+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:48.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:48.272+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:48.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:48.272+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:48.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:48.143+0000 7f35fd6c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:48.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:48.143+0000 7f35fd6c2700 -1 osd.6 80 *** Got signal Terminated *** 2024-04-25T14:12:48.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:48.143+0000 7f35fd6c2700 -1 osd.6 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:48.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:48.291+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:48.470 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:48.291+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:48.470 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:48 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:48.291+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:49.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:49.010+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:49.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:49.010+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:49.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:49.145+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:49.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:49.145+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:49.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:49.145+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:49.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:49.096+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:49.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:49.096+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:49.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:48 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:48.962+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:49.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:48 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:48.962+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:49.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:48 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:48.962+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:49.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:49.274+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:49.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:49.274+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:49.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:49.274+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:49.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:49.311+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:49.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:49.311+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:49.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:49.311+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:50.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:49.966+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:50.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:49 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:49.966+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:50.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:50 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:50.179+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:50.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:50 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:50.179+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:50.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:50 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:50.179+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:50.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:50 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:50.092+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:50.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:50 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:50.092+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:50.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:49.977+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:50.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:49.977+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:50.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:49 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:49.977+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:50.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:50 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:50.248+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:50.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:50 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:50.248+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:50.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:50 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:50.248+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:50.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:50 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:50.353+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:50.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:50 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:50.353+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:50.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:50 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:50.353+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:51.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:50 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:50.985+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:51.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:50 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:50.985+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:51.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:51 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:51.153+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:51.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:51 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:51.153+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:51.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:51 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:51.153+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:51.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:51 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:51.153+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:51.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:51 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:51.114+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:51.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:51 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:51.114+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:51.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:50 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:50.989+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:51.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:50 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:50.989+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:51.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:50 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:50.989+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:51.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:51 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:51.236+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:51.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:51 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:51.236+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:51.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:51 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:51.236+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:51.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:51 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:51.382+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:51.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:51 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:51.382+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:51.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:51 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:51.382+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:52.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:52 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:52.007+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:52.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:52 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:52.007+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:52.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:52 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:52.106+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:52.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:52 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:52.106+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:52.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:52 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:52.106+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:52.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:52 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:52.106+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:52.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:52 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:52.084+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:52.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:52 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:52.084+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:52.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:51 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:51.983+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:52.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:51 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:51.983+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:52.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:51 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:51.983+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:52.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:52.196+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.754576+0000 front 2024-04-25T14:12:03.754619+0000 (oldest deadline 2024-04-25T14:12:26.054883+0000) 2024-04-25T14:12:52.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:52.196+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:10.756220+0000 front 2024-04-25T14:12:10.756122+0000 (oldest deadline 2024-04-25T14:12:36.656216+0000) 2024-04-25T14:12:52.469 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6[57032]: 2024-04-25T14:12:52.196+0000 7f35f9c78700 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.757381+0000 front 2024-04-25T14:12:20.757287+0000 (oldest deadline 2024-04-25T14:12:44.857237+0000) 2024-04-25T14:12:52.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:52.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:52.470 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:52.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:52.470 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:52 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:52.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:53.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:53 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:53.063+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:53.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:53 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:53.063+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:53.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:52 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:52.973+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:53.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:52 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:52.973+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:53.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:52 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:52.973+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:53.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:53.029+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:53.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:53.029+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:53.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:53.135+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:53.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:53.135+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:53.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:53.135+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:53.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:53.135+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:53.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:53 smithi078 podman[98494]: 2024-04-25 14:12:53.633195782 +0000 UTC m=+5.595255833 container died d70e3f488e01e87172c033aab0cdd69ab8a552a20f0605d59f4718ed552a8e19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T14:12:53.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:53 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:53.390+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:53.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:53 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:53.390+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:53.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:53 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:53.390+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:54.219 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 podman[98494]: 2024-04-25 14:12:54.00182169 +0000 UTC m=+5.963881740 container cleanup d70e3f488e01e87172c033aab0cdd69ab8a552a20f0605d59f4718ed552a8e19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T14:12:54.219 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 bash[98494]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6 2024-04-25T14:12:54.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:53.996+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:54.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:53.996+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:54.266 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:53.996+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:12:54.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:54.176+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:54.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:54.176+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:54.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:54.176+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:54.267 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:54 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:54.176+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:54.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:54 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:54.077+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:54.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:54 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:54.077+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:54.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:53 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:53.928+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:54.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:53 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:53.928+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:54.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:53 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:53.928+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:54.569 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 podman[98511]: 2024-04-25 14:12:54.253127419 +0000 UTC m=+0.615170854 container remove d70e3f488e01e87172c033aab0cdd69ab8a552a20f0605d59f4718ed552a8e19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:12:54.569 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:54 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:54.343+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:54.570 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:54 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:54.343+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:54.570 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:54 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:54.343+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:54.570 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:54 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:54.343+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:12:54.822 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 podman[98599]: 2024-04-25 14:12:54.473734782 +0000 UTC m=+0.017904999 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:12:54.822 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 podman[98599]: 2024-04-25 14:12:54.587299708 +0000 UTC m=+0.131469892 container create 0864892af80a3690df71c737e6733a4eea2c3049f6d7cdb29ede7510ac431abf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T14:12:54.822 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 podman[98599]: 2024-04-25 14:12:54.823357559 +0000 UTC m=+0.367527756 container init 0864892af80a3690df71c737e6733a4eea2c3049f6d7cdb29ede7510ac431abf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:12:55.114 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 podman[98599]: 2024-04-25 14:12:54.826515731 +0000 UTC m=+0.370685916 container start 0864892af80a3690df71c737e6733a4eea2c3049f6d7cdb29ede7510ac431abf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T14:12:55.114 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:54 smithi078 podman[98599]: 2024-04-25 14:12:54.90672373 +0000 UTC m=+0.450893943 container attach 0864892af80a3690df71c737e6733a4eea2c3049f6d7cdb29ede7510ac431abf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:12:55.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:54 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:54.898+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:55.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:54 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:54.898+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:55.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:54 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:54.898+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:55.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:55.054+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:55.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:55.054+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:55.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:55.054+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:12:55.459 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:55 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:55.331+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:55.459 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:55 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:55.331+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:55.459 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:55 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:55.331+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:55.459 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:55 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:55.331+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:12:55.459 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 14:12:55 smithi078 podman[98599]: 2024-04-25 14:12:55.115656962 +0000 UTC m=+0.659827162 container died 0864892af80a3690df71c737e6733a4eea2c3049f6d7cdb29ede7510ac431abf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T14:12:55.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:55 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:55.022+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:55.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:55 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:55.022+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:55.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:55 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:55.022+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:12:55.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:55 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:55.194+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:55.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:55 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:55.194+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:55.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:55 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:55.194+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:55.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:55 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:55.194+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:55.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T14:12:55.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:12:55.817 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T14:12:55.817 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T14:12:55.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.7 2024-04-25T14:12:56.130 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:55 smithi112 systemd[1]: Stopping Ceph osd.7 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:12:56.130 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:56.070+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:56.130 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:56.070+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:56.130 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:56.070+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:12:56.130 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:56.130+0000 7fbe27b91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:12:56.131 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:56.130+0000 7fbe27b91700 -1 osd.7 80 *** Got signal Terminated *** 2024-04-25T14:12:56.131 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:56.130+0000 7fbe27b91700 -1 osd.7 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:12:56.163 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:56 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:56.005+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:56.163 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:56 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:56.005+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:56.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:56 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:56.005+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:12:56.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:55.894+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:56.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:55.894+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:56.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:55 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:55.894+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:56.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:56.220+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:56.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:56.220+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:56.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:56.220+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:56.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:56 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:56.220+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:56.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:56 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:56.353+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:56.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:56 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:56.353+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:56.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:56 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:56.353+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:56.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:56 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:56.353+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:12:57.163 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:57 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:57.033+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:57.163 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:57 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:57.033+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:57.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:57 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:57.034+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:12:57.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:56 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:56.897+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:57.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:56 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:56.897+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:57.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:56 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:56.897+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:57.418 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:57 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:57.092+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:57.418 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:57 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:57.092+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:57.418 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:57 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:57.092+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:12:57.418 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:57 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:57.265+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:57.419 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:57 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:57.265+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:57.419 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:57 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:57.265+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:57.419 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:57 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:57.265+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:57.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:57 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:57.403+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:57.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:57 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:57.403+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:57.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:57 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:57.403+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:57.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:57 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:57.403+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:12:58.413 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:57 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:57.926+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:58.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:57 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:57.926+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:58.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:57 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:57.926+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:58.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:58 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:58.037+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:58.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:58 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:58.037+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:58.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:58 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:58.037+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:12:58.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:58.081+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:58.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:58.081+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:58.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:58.081+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:12:58.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:58.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:58.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:58.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:58.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:58.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:58.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:58.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:58.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:58 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:58.410+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:58.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:58 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:58.410+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:58.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:58 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:58.410+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:58.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:58 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:58.410+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:12:59.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:58 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:58.879+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:12:59.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:58 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:58.879+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:12:59.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:58 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:58.879+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:12:59.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:58 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:58.879+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:12:59.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:59 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:59.066+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:12:59.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:59 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:59.066+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:12:59.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:12:59 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:12:59.066+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:12:59.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:59.044+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:12:59.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:59.044+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:12:59.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:59.044+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:12:59.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:12:59.044+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:33.701406+0000 front 2024-04-25T14:12:33.701353+0000 (oldest deadline 2024-04-25T14:12:58.401454+0000) 2024-04-25T14:12:59.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:59.260+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:12:59.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:59.260+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:12:59.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:59.260+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:12:59.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:12:59 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:12:59.260+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:12:59.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:59 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:59.363+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:12:59.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:59 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:59.363+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:12:59.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:59 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:59.363+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:12:59.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:59 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:59.363+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:12:59.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:12:59 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:12:59.363+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:00.413 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:59 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:59.927+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:00.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:59 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:59.927+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:00.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:59 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:59.927+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:00.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:12:59 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:12:59.927+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:00.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:00.098+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:00.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:00.098+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:00.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:00.098+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:00.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:00.098+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:00.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:00.092+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:13:00.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:00.092+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:13:00.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:00.092+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:13:00.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:00.092+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:33.701406+0000 front 2024-04-25T14:12:33.701353+0000 (oldest deadline 2024-04-25T14:12:58.401454+0000) 2024-04-25T14:13:00.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:00.300+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:00.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:00.300+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:00.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:00.300+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:00.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:00 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:00.300+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:00.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:00.404+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:00.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:00.404+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:00.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:00.404+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:00.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:00.404+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:00.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:00 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:00.404+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:01.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:01.134+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:01.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:01.134+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:01.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:01.134+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:01.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:01.134+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:01.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:00.957+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:01.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:00.957+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:01.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:00.957+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:01.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:00 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:00.957+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:01.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:01.111+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.098183+0000 front 2024-04-25T14:12:12.098214+0000 (oldest deadline 2024-04-25T14:12:37.998279+0000) 2024-04-25T14:13:01.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:01.111+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.998779+0000 front 2024-04-25T14:12:17.998806+0000 (oldest deadline 2024-04-25T14:12:42.098725+0000) 2024-04-25T14:13:01.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:01.111+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:29.100365+0000 front 2024-04-25T14:12:29.100399+0000 (oldest deadline 2024-04-25T14:12:53.700999+0000) 2024-04-25T14:13:01.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7[51186]: 2024-04-25T14:13:01.111+0000 7fbe2013f700 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:33.701406+0000 front 2024-04-25T14:12:33.701353+0000 (oldest deadline 2024-04-25T14:12:58.401454+0000) 2024-04-25T14:13:01.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:01.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:01.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:01.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:01.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:01.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:01.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:01.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:01.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:01 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:01.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:01.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:01.413+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:01.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:01.413+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:01.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:01.413+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:01.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:01.413+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:01.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:01 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:01.413+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:01.814 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:01 smithi112 podman[69578]: 2024-04-25 14:13:01.528406597 +0000 UTC m=+5.518403480 container died b8ea8386350aa23df030323f362e4d904f8fb55ce910ae8dc9d8769bdcd1958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:13:02.198 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:01 smithi112 podman[69578]: 2024-04-25 14:13:01.897946982 +0000 UTC m=+5.887943874 container cleanup b8ea8386350aa23df030323f362e4d904f8fb55ce910ae8dc9d8769bdcd1958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T14:13:02.198 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:01 smithi112 bash[69578]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7 2024-04-25T14:13:02.413 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:01.977+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:02.413 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:01.977+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:02.413 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:01.977+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:02.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:01 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:01.977+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:02.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:02 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:02.139+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:02.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:02 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:02.139+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:02.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:02 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:02.139+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:02.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:02 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:02.139+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:02.516 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:02 smithi112 podman[69608]: 2024-04-25 14:13:02.199125788 +0000 UTC m=+0.664696295 container remove b8ea8386350aa23df030323f362e4d904f8fb55ce910ae8dc9d8769bdcd1958e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T14:13:02.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:02 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:02.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:02.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:02 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:02.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:02.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:02 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:02.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:02.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:02 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:02.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:02.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:02 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:02.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:02.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:02 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:02.438+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:02.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:02 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:02.438+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:02.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:02 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:02.438+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:02.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:02 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:02.438+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:02.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:02 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:02.438+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:02.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:02 smithi112 podman[69698]: 2024-04-25 14:13:02.423088906 +0000 UTC m=+0.017850579 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:13:02.794 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:02 smithi112 podman[69698]: 2024-04-25 14:13:02.541708467 +0000 UTC m=+0.136470124 container create 3c0009f5012fcb6d7e397216f9904223bd30979b85df2960d4df484cfa50f565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T14:13:03.083 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:02 smithi112 podman[69698]: 2024-04-25 14:13:02.794771544 +0000 UTC m=+0.389533214 container init 3c0009f5012fcb6d7e397216f9904223bd30979b85df2960d4df484cfa50f565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:13:03.084 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:02 smithi112 podman[69698]: 2024-04-25 14:13:02.798547643 +0000 UTC m=+0.393309301 container start 3c0009f5012fcb6d7e397216f9904223bd30979b85df2960d4df484cfa50f565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:13:03.084 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:02 smithi112 podman[69698]: 2024-04-25 14:13:02.869338173 +0000 UTC m=+0.464099840 container attach 3c0009f5012fcb6d7e397216f9904223bd30979b85df2960d4df484cfa50f565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:13:03.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:03.090+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:03.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:03.090+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:03.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:03.090+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:03.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:03.090+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:03.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:03.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:03.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:03.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:03.446 INFO:journalctl@ceph.osd.7.smithi112.stdout:Apr 25 14:13:03 smithi112 podman[69698]: 2024-04-25 14:13:03.084509352 +0000 UTC m=+0.679271001 container died 3c0009f5012fcb6d7e397216f9904223bd30979b85df2960d4df484cfa50f565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:13:03.447 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:03 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:03.238+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:03.447 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:03 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:03.238+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:03.447 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:03 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:03.238+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:03.447 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:03 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:03.238+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:03.447 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:03 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:03.238+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:03.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:03.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:03.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:03.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:03.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:03.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:03.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:03.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:03.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:03 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:03.392+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:03.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T14:13:03.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:03.816 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T14:13:03.816 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T14:13:03.816 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.8 2024-04-25T14:13:04.109 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:03 smithi129 systemd[1]: Stopping Ceph osd.8 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:04.109 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.998+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:04.109 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.998+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:04.109 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.998+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:04.109 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:03 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:03.998+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:04.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:04 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:04.109+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:04.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:04 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:04.109+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:04.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:04 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:04.109+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:04.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:04 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:04.109+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:04.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:04 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:04.126+0000 7fd253968700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:04.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:04 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:04.126+0000 7fd253968700 -1 osd.8 80 *** Got signal Terminated *** 2024-04-25T14:13:04.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:04 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:04.126+0000 7fd253968700 -1 osd.8 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:13:04.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:04 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:04.234+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:04.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:04 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:04.234+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:04.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:04 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:04.234+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:04.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:04 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:04.234+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:04.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:04 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:04.234+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:04.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:04.376+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:04.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:04.376+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:04.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:04.376+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:04.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:04.376+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:04.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:04 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:04.376+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:05.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:05.084+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:05.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:05.084+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:05.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:05.084+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:05.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:05.084+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:05.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:05.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:05.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:05.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:05.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:05.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:05.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:05 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:05.016+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:05.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:05 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:05.187+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:05.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:05 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:05.187+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:05.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:05 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:05.187+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:05.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:05 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:05.187+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:05.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:05 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:05.187+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:05.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:05.420+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:05.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:05.420+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:05.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:05.420+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:05.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:05.420+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:05.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:05 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:05.420+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:06.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:06.048+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:06.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:06.048+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:06.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:06.048+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:06.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:06.048+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:06.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:06.066+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:06.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:06.066+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:06.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:06.066+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:06.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:06 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:06.066+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:06.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:06.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:06.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:06.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:06.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:06.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:06.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:06.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:06.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:06.171+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:06.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:06.450+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:06.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:06.450+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:06.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:06.450+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:06.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:06.450+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:06.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:06 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:06.450+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:07.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:07.066+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:07.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:07.066+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:07.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:07.066+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:07.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:07.066+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:07.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:07.061+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:07.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:07.061+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:07.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:07.061+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:07.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:07 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:07.061+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:07.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:07 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:07.172+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:07.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:07 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:07.172+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:07.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:07 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:07.172+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:07.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:07 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:07.172+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:07.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:07 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:07.172+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:07.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:07 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:07.436+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:07.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:07 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:07.436+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:07.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:07 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:07.436+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:07.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:07 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:07.436+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:07.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:07 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:07.436+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:08.413 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:08.092+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:08.092+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:08.092+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:08.092+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:08.092+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:43.182681+0000 front 2024-04-25T14:12:43.182706+0000 (oldest deadline 2024-04-25T14:13:07.282714+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:08.015+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:08.015+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:08.015+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:08.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:08.015+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:08.415 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:08 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:08.015+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:08.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:08.155+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:08.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:08.155+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:08.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:08.155+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:08.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:08.155+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:08.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:08.155+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:08.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:08 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:08.155+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:08.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:08 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:08.431+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:08.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:08 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:08.431+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:08.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:08 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:08.431+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:08.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:08 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:08.431+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:08.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:08 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:08.431+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:09.413 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:09.065+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:09.065+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:09.065+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:09.065+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:09.065+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:09.081+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.476732+0000 front 2024-04-25T14:12:12.476765+0000 (oldest deadline 2024-04-25T14:12:38.376739+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:09.081+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.677645+0000 front 2024-04-25T14:12:20.677779+0000 (oldest deadline 2024-04-25T14:12:46.577842+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:09.081+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:28.878856+0000 front 2024-04-25T14:12:28.878958+0000 (oldest deadline 2024-04-25T14:12:54.178951+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:09.081+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:37.079973+0000 front 2024-04-25T14:12:37.080157+0000 (oldest deadline 2024-04-25T14:12:59.280502+0000) 2024-04-25T14:13:09.414 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8[50742]: 2024-04-25T14:13:09.081+0000 7fd24bf16700 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:43.182681+0000 front 2024-04-25T14:12:43.182706+0000 (oldest deadline 2024-04-25T14:13:07.282714+0000) 2024-04-25T14:13:09.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:09 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:09.127+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:09.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:09 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:09.127+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:09.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:09 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:09.127+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:09.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:09 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:09.127+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:09.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:09 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:09.127+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:09.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:09 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:09.127+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:09.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:09 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:09.441+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:09.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:09 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:09.441+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:09.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:09 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:09.441+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:09.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:09 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:09.441+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:09.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:09 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:09.441+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:09.859 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 podman[68427]: 2024-04-25 14:13:09.582421753 +0000 UTC m=+5.579617864 container died 49d499df7f57219de98f59a2c9f25e5e6a1feb15ef5f8f3148470f23398aaeb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T14:13:10.163 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 podman[68427]: 2024-04-25 14:13:09.959329023 +0000 UTC m=+5.956525133 container cleanup 49d499df7f57219de98f59a2c9f25e5e6a1feb15ef5f8f3148470f23398aaeb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T14:13:10.163 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:09 smithi129 bash[68427]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8 2024-04-25T14:13:10.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:10 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:10.107+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:10.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:10 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:10.107+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:10.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:10 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:10.107+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:10.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:10 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:10.107+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:10.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:10 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:10.107+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:10.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:10 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:10.128+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:10.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:10 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:10.128+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:10.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:10 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:10.128+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:10.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:10 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:10.128+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:10.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:10 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:10.128+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:10.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:10 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:10.128+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:10.574 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:10 smithi129 podman[68444]: 2024-04-25 14:13:10.235744799 +0000 UTC m=+0.646792669 container remove 49d499df7f57219de98f59a2c9f25e5e6a1feb15ef5f8f3148470f23398aaeb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T14:13:10.896 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:10 smithi129 podman[68540]: 2024-04-25 14:13:10.478709727 +0000 UTC m=+0.018240618 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:13:10.896 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:10 smithi129 podman[68540]: 2024-04-25 14:13:10.586688994 +0000 UTC m=+0.126219895 container create 09c54a7371efcd56dde5e94ed96b1a6031132a666b566d836e7cb16f4a9d5df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:13:10.896 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:10 smithi129 podman[68540]: 2024-04-25 14:13:10.798093083 +0000 UTC m=+0.337623950 container init 09c54a7371efcd56dde5e94ed96b1a6031132a666b566d836e7cb16f4a9d5df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T14:13:10.896 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:10 smithi129 podman[68540]: 2024-04-25 14:13:10.801226068 +0000 UTC m=+0.340756927 container start 09c54a7371efcd56dde5e94ed96b1a6031132a666b566d836e7cb16f4a9d5df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T14:13:10.896 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:10 smithi129 podman[68540]: 2024-04-25 14:13:10.897241138 +0000 UTC m=+0.436772000 container attach 09c54a7371efcd56dde5e94ed96b1a6031132a666b566d836e7cb16f4a9d5df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T14:13:10.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:10 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:10.469+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:10.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:10 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:10.469+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:10.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:10 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:10.469+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:10.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:10 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:10.469+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:10.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:10 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:10.469+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:11.164 INFO:journalctl@ceph.osd.8.smithi129.stdout:Apr 25 14:13:11 smithi129 podman[68540]: 2024-04-25 14:13:11.099220092 +0000 UTC m=+0.638750953 container died 09c54a7371efcd56dde5e94ed96b1a6031132a666b566d836e7cb16f4a9d5df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:11.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:11 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:11.141+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:11.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:11 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:11.141+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:11.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:11 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:11.141+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:11.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:11 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:11.141+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:11.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:11 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:11.141+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:11.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:11 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:11.139+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:11.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:11 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:11.139+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:11.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:11 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:11.139+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:11.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:11 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:11.139+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:11.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:11 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:11.139+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:11.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:11 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:11.139+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:11.660 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T14:13:11.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:11.697 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T14:13:11.697 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T14:13:11.697 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.9 2024-04-25T14:13:11.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:11.508+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:11.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:11.508+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:11.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:11.508+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:11.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:11.508+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:11.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:11.508+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:11.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:11 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:11.508+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:44.800196+0000 front 2024-04-25T14:12:44.800167+0000 (oldest deadline 2024-04-25T14:13:10.699999+0000) 2024-04-25T14:13:11.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:11 smithi078 systemd[1]: Stopping Ceph osd.9 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:12.468 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.004+0000 7fce61fd4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:12.468 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.004+0000 7fce61fd4700 -1 osd.9 80 *** Got signal Terminated *** 2024-04-25T14:13:12.468 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.004+0000 7fce61fd4700 -1 osd.9 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:13:12.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.468+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:12.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.468+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:12.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.468+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:12.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:12 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:12.167+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:12.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:12 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:12.167+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:12.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:12 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:12.167+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:12.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:12 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:12.167+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:12.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:12 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:12.167+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:12.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:12 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:12.167+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:12.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:12 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:12.173+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:12.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:12 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:12.173+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:12.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:12 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:12.173+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:12.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:12 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:12.173+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:12.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:12 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:12.173+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:12.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.468+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:12.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.468+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:12.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:12 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:12.468+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:44.800196+0000 front 2024-04-25T14:12:44.800167+0000 (oldest deadline 2024-04-25T14:13:10.699999+0000) 2024-04-25T14:13:13.518 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:13 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:13.205+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:13.518 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:13 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:13.205+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:13.518 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:13 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:13.205+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:13.518 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:13 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:13.205+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:13.518 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:13 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:13.205+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:13.519 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:13 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:13.205+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:13.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:13 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:13.187+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:13.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:13 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:13.187+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:13.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:13 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:13.187+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:13.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:13 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:13.187+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:13.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:13 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:13.187+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:13.721 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:13 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:13.432+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:13.721 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:13 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:13.432+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:13.721 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:13 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:13.432+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:13.722 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:13 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:13.432+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:13.722 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:13 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:13.432+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:13.722 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:13 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:13.432+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:44.800196+0000 front 2024-04-25T14:12:44.800167+0000 (oldest deadline 2024-04-25T14:13:10.699999+0000) 2024-04-25T14:13:14.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:14 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:14.225+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:14.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:14 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:14.225+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:14.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:14 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:14.225+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:14.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:14 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:14.225+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:14.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:14 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:14.225+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:14.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:14 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:14.225+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:14.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:14 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:14.216+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:14.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:14 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:14.216+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:14.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:14 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:14.216+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:14.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:14 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:14.216+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:14.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:14 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:14.216+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:14.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:14 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:14.462+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:14.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:14 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:14.462+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:14.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:14 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:14.462+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:14.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:14 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:14.462+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:14.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:14 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:14.462+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:14.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:14 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:14.462+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:44.800196+0000 front 2024-04-25T14:12:44.800167+0000 (oldest deadline 2024-04-25T14:13:10.699999+0000) 2024-04-25T14:13:15.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:15 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:15.240+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:15.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:15 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:15.240+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:15.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:15 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:15.240+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:15.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:15 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:15.240+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:15.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:15 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:15.240+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:15.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:15 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:15.240+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:15.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:15 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:15.240+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:15.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:15 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:15.239+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:15.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:15 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:15.239+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:15.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:15 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:15.239+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:15.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:15 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:15.239+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:15.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:15 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:15.239+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:15.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:15 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:15.239+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:15.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:15 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:15.504+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:15.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:15 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:15.504+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:15.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:15 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:15.504+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:15.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:15 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:15.504+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:15.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:15 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:15.504+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:15.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:15 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:15.504+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:44.800196+0000 front 2024-04-25T14:12:44.800167+0000 (oldest deadline 2024-04-25T14:13:10.699999+0000) 2024-04-25T14:13:16.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:16.228+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:16.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:16.228+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:16.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:16.228+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:16.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:16.228+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:16.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:16.228+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:16.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:16.228+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:16.517 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:16 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:16.228+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:16.565 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:16.247+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:16.565 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:16.247+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:16.565 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:16.247+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:16.565 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:16.247+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:16.565 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:16.247+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:16.565 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:16 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:16.247+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:16.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:16.488+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:03.492938+0000 front 2024-04-25T14:12:03.493080+0000 (oldest deadline 2024-04-25T14:12:28.193065+0000) 2024-04-25T14:13:16.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:16.488+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:09.893922+0000 front 2024-04-25T14:12:09.894017+0000 (oldest deadline 2024-04-25T14:12:34.593827+0000) 2024-04-25T14:13:16.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:16.488+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:18.595342+0000 front 2024-04-25T14:12:18.595318+0000 (oldest deadline 2024-04-25T14:12:44.495535+0000) 2024-04-25T14:13:16.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:16.488+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.495952+0000 front 2024-04-25T14:12:24.495989+0000 (oldest deadline 2024-04-25T14:12:53.896811+0000) 2024-04-25T14:13:16.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:16.488+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.597838+0000 front 2024-04-25T14:12:35.597966+0000 (oldest deadline 2024-04-25T14:12:58.497787+0000) 2024-04-25T14:13:16.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:16 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9[63611]: 2024-04-25T14:13:16.488+0000 7fce5a582700 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:44.800196+0000 front 2024-04-25T14:12:44.800167+0000 (oldest deadline 2024-04-25T14:13:10.699999+0000) 2024-04-25T14:13:17.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:17 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:17.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:17.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:17 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:17.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:17.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:17 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:17.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:17.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:17 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:17.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:17.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:17 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:17.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:17.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:17 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:17.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:17.516 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:17 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:17.249+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:17.576 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:17 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:17.281+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:17.576 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:17 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:17.281+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:17.577 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:17 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:17.281+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:17.577 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:17 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:17.281+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:17.577 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:17 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:17.281+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:17.577 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:17 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:17.281+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:17.672 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:17 smithi078 podman[98717]: 2024-04-25 14:13:17.401828649 +0000 UTC m=+5.511108402 container died ad91ba42e1888bcc76c77ad61b38495e770c6f371c99e8d03bb2faf4b37ce8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:13:17.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:17 smithi078 podman[98717]: 2024-04-25 14:13:17.780012174 +0000 UTC m=+5.889291972 container cleanup ad91ba42e1888bcc76c77ad61b38495e770c6f371c99e8d03bb2faf4b37ce8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T14:13:17.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:17 smithi078 bash[98717]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9 2024-04-25T14:13:18.329 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:18 smithi078 podman[98735]: 2024-04-25 14:13:18.023103442 +0000 UTC m=+0.616516591 container remove ad91ba42e1888bcc76c77ad61b38495e770c6f371c99e8d03bb2faf4b37ce8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:13:18.592 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:18 smithi078 podman[98826]: 2024-04-25 14:13:18.233099272 +0000 UTC m=+0.018208063 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:13:18.592 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:18 smithi078 podman[98826]: 2024-04-25 14:13:18.40732184 +0000 UTC m=+0.192430612 container create 9b88bf2e5e451fbed46782d3957e36e0ace59f14f6aeb8535c5482bc03ac7e80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T14:13:18.592 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:18 smithi078 podman[98826]: 2024-04-25 14:13:18.592820307 +0000 UTC m=+0.377929079 container init 9b88bf2e5e451fbed46782d3957e36e0ace59f14f6aeb8535c5482bc03ac7e80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:13:18.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:18.330+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:18.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:18.330+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:18.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:18.330+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:18.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:18.330+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:18.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:18.330+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:18.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:18.330+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:18.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:18 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:18.299+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:18.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:18 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:18.299+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:18.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:18 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:18.299+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:18.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:18 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:18.299+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:18.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:18 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:18.299+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:18.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:18 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:18.299+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:18.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:18 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:18.299+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:18.888 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:18 smithi078 podman[98826]: 2024-04-25 14:13:18.596821692 +0000 UTC m=+0.381930469 container start 9b88bf2e5e451fbed46782d3957e36e0ace59f14f6aeb8535c5482bc03ac7e80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T14:13:18.888 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:18 smithi078 podman[98826]: 2024-04-25 14:13:18.676281052 +0000 UTC m=+0.461389837 container attach 9b88bf2e5e451fbed46782d3957e36e0ace59f14f6aeb8535c5482bc03ac7e80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T14:13:19.178 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 14:13:18 smithi078 podman[98826]: 2024-04-25 14:13:18.889541263 +0000 UTC m=+0.674650038 container died 9b88bf2e5e451fbed46782d3957e36e0ace59f14f6aeb8535c5482bc03ac7e80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:13:19.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T14:13:19.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:19.508 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T14:13:19.509 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T14:13:19.509 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.10 2024-04-25T14:13:19.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:19.322+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:19.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:19.322+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:19.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:19.322+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:19.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:19.322+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:19.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:19.322+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:19.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:19 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:19.322+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:19.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:19.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:19.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:19.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:19.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:19.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:19.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:19.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 systemd[1]: Stopping Ceph osd.10 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:20.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.783+0000 7f1f341d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:20.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.783+0000 7f1f341d7700 -1 osd.10 80 *** Got signal Terminated *** 2024-04-25T14:13:20.266 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:19 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:19.783+0000 7f1f341d7700 -1 osd.10 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:13:20.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:20.302+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:20.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:20.302+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:20.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:20.302+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:20.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:20.302+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:20.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:20.302+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:20.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:20 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:20.302+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:20.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:20.315+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:20.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:20.315+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:20.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:20.315+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:20.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:20.315+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:20.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:20.315+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:20.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:20.315+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:20.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:20 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:20.315+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:21.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:21.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:21.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:21.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:21.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:21.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:21.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:21.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:21.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:21.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:21.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:21 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:21.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:21.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:21.358+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:21.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:21.358+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:21.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:21.358+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:21.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:21.358+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:21.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:21.358+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:21.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:21.358+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:21.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:21 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:21.358+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:22.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:22.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:22.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:22.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:22.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:22.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:22.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:22.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:22.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:22.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:22.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:22.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:22.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:22.274+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:22.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:22.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:22.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:22.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:22.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:22.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:22.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:22.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:22.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:22.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:22.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:22.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:22.767 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:22 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:22.319+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:23.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:23.291+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:23.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:23.291+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:23.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:23.291+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:23.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:23.291+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:23.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:23.291+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:23.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:23.291+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:23.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:23 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:23.291+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:23.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:23.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:23.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:23.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:23.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:23.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:23.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:23.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:23.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:23.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:23.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:23.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:23.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:23 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:23.317+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:24.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:24.290+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:24.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:24.290+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:24.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:24.290+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:24.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:24.290+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:24.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:24.290+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:24.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:24.290+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:24.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:24 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:24.290+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:24.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:24.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.966254+0000 front 2024-04-25T14:12:04.966111+0000 (oldest deadline 2024-04-25T14:12:30.866326+0000) 2024-04-25T14:13:24.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:24.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:11.367404+0000 front 2024-04-25T14:12:11.367549+0000 (oldest deadline 2024-04-25T14:12:36.667551+0000) 2024-04-25T14:13:24.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:24.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:17.768418+0000 front 2024-04-25T14:12:17.768335+0000 (oldest deadline 2024-04-25T14:12:43.668342+0000) 2024-04-25T14:13:24.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:24.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T14:12:24.769537+0000 front 2024-04-25T14:12:24.769474+0000 (oldest deadline 2024-04-25T14:12:50.669250+0000) 2024-04-25T14:13:24.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:24.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.370140+0000 front 2024-04-25T14:12:35.370171+0000 (oldest deadline 2024-04-25T14:13:00.570696+0000) 2024-04-25T14:13:24.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:24.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:45.172367+0000 front 2024-04-25T14:12:45.172311+0000 (oldest deadline 2024-04-25T14:13:07.472079+0000) 2024-04-25T14:13:24.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:24 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10[57922]: 2024-04-25T14:13:24.285+0000 7f1f2c785700 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:52.173272+0000 front 2024-04-25T14:12:52.173088+0000 (oldest deadline 2024-04-25T14:13:15.072848+0000) 2024-04-25T14:13:25.500 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:25 smithi112 podman[69803]: 2024-04-25 14:13:25.220508563 +0000 UTC m=+5.517340714 container died bd356e625fee1b0a44117c93404124a61ae911c361f29bc1a69ed05340a79f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:13:25.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:25 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:25.283+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:25.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:25 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:25.283+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:25.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:25 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:25.283+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:25.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:25 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:25.283+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:25.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:25 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:25.283+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:25.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:25 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:25.283+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:25.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:25 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:25.283+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:25.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:25 smithi112 podman[69803]: 2024-04-25 14:13:25.592683439 +0000 UTC m=+5.889515649 container cleanup bd356e625fee1b0a44117c93404124a61ae911c361f29bc1a69ed05340a79f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:25.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:25 smithi112 bash[69803]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10 2024-04-25T14:13:26.136 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:25 smithi112 podman[69820]: 2024-04-25 14:13:25.83604044 +0000 UTC m=+0.616845464 container remove bd356e625fee1b0a44117c93404124a61ae911c361f29bc1a69ed05340a79f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:26.388 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:26 smithi112 podman[69910]: 2024-04-25 14:13:26.137280915 +0000 UTC m=+0.098267207 container create 6d27820882c2c3c6ce71fd45bc4ee71a723dce7343bc75d7e2bc357400b4da98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:13:26.389 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:26 smithi112 podman[69910]: 2024-04-25 14:13:26.056921146 +0000 UTC m=+0.017907411 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:13:26.389 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:26 smithi112 podman[69910]: 2024-04-25 14:13:26.315005346 +0000 UTC m=+0.275991597 container init 6d27820882c2c3c6ce71fd45bc4ee71a723dce7343bc75d7e2bc357400b4da98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:26.389 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:26 smithi112 podman[69910]: 2024-04-25 14:13:26.318136894 +0000 UTC m=+0.279123151 container start 6d27820882c2c3c6ce71fd45bc4ee71a723dce7343bc75d7e2bc357400b4da98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:26.389 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:26 smithi112 podman[69910]: 2024-04-25 14:13:26.389756307 +0000 UTC m=+0.350742586 container attach 6d27820882c2c3c6ce71fd45bc4ee71a723dce7343bc75d7e2bc357400b4da98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:13:26.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:26 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:26.320+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:26.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:26 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:26.320+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:26.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:26 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:26.320+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:26.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:26 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:26.320+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:26.665 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:26 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:26.320+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:26.665 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:26 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:26.320+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:26.665 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:26 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:26.320+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:26.766 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:26 smithi112 podman[69910]: 2024-04-25 14:13:26.614612068 +0000 UTC m=+0.575598313 container died 6d27820882c2c3c6ce71fd45bc4ee71a723dce7343bc75d7e2bc357400b4da98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T14:13:27.158 INFO:journalctl@ceph.osd.10.smithi112.stdout:Apr 25 14:13:27 smithi112 podman[69936]: 2024-04-25 14:13:27.134994593 +0000 UTC m=+0.515634694 container remove 6d27820882c2c3c6ce71fd45bc4ee71a723dce7343bc75d7e2bc357400b4da98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:27.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T14:13:27.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:27.201 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T14:13:27.201 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T14:13:27.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.11 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 systemd[1]: Stopping Ceph osd.11 for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.271+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.271+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.271+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.271+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.271+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.271+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.271+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.502+0000 7fd1321f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:27.503 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.502+0000 7fd1321f1700 -1 osd.11 80 *** Got signal Terminated *** 2024-04-25T14:13:27.504 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:27 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:27.502+0000 7fd1321f1700 -1 osd.11 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T14:13:28.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:28.319+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:28.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:28.319+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:28.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:28.319+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:28.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:28.319+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:28.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:28.319+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:28.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:28.319+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:28.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:28 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:28.319+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:29.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:29.350+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:29.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:29.350+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:29.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:29.350+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:29.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:29.350+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:29.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:29.350+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:29.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:29.350+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:29.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:29 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:29.350+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:30.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:30.399+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:30.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:30.399+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:30.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:30.399+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:30.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:30.399+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:30.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:30.399+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:30.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:30.399+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:30.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:30 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:30.399+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:31.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:31.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:31.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:31.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:31.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:31.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:31.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:31.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:31 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:31.371+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6820 osd.8 since back 2024-04-25T14:13:06.222306+0000 front 2024-04-25T14:13:06.222333+0000 (oldest deadline 2024-04-25T14:13:30.922319+0000) 2024-04-25T14:13:32.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T14:12:04.012583+0000 front 2024-04-25T14:12:04.012619+0000 (oldest deadline 2024-04-25T14:12:27.512518+0000) 2024-04-25T14:13:32.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6804 osd.1 since back 2024-04-25T14:12:12.813720+0000 front 2024-04-25T14:12:12.813611+0000 (oldest deadline 2024-04-25T14:12:38.713645+0000) 2024-04-25T14:13:32.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-04-25T14:12:20.414363+0000 front 2024-04-25T14:12:20.414496+0000 (oldest deadline 2024-04-25T14:12:43.914456+0000) 2024-04-25T14:13:32.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6812 osd.4 since back 2024-04-25T14:12:35.516965+0000 front 2024-04-25T14:12:35.517008+0000 (oldest deadline 2024-04-25T14:12:58.417084+0000) 2024-04-25T14:13:32.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-04-25T14:12:41.818586+0000 front 2024-04-25T14:12:41.818612+0000 (oldest deadline 2024-04-25T14:13:07.118612+0000) 2024-04-25T14:13:32.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T14:12:51.819690+0000 front 2024-04-25T14:12:51.819751+0000 (oldest deadline 2024-04-25T14:13:14.719690+0000) 2024-04-25T14:13:32.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.112:6820 osd.7 since back 2024-04-25T14:12:55.721241+0000 front 2024-04-25T14:12:55.721196+0000 (oldest deadline 2024-04-25T14:13:21.621121+0000) 2024-04-25T14:13:32.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11[57359]: 2024-04-25T14:13:32.358+0000 7fd12e7a7700 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.129:6820 osd.8 since back 2024-04-25T14:13:06.222306+0000 front 2024-04-25T14:13:06.222333+0000 (oldest deadline 2024-04-25T14:13:30.922319+0000) 2024-04-25T14:13:33.219 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:32 smithi129 podman[68658]: 2024-04-25 14:13:32.951136925 +0000 UTC m=+5.558925984 container died f4776edfcf71cf4784af932fc43111c6c7fbf2bd3ab474e0f9c29cb35d484609 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T14:13:33.561 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:33 smithi129 podman[68658]: 2024-04-25 14:13:33.31977574 +0000 UTC m=+5.927564795 container cleanup f4776edfcf71cf4784af932fc43111c6c7fbf2bd3ab474e0f9c29cb35d484609 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-25T14:13:33.562 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:33 smithi129 bash[68658]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11 2024-04-25T14:13:33.873 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:33 smithi129 podman[68675]: 2024-04-25 14:13:33.562722773 +0000 UTC m=+0.612522600 container remove f4776edfcf71cf4784af932fc43111c6c7fbf2bd3ab474e0f9c29cb35d484609 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:13:34.163 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:33 smithi129 podman[68765]: 2024-04-25 14:13:33.777379335 +0000 UTC m=+0.017958250 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T14:13:34.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:33 smithi129 podman[68765]: 2024-04-25 14:13:33.888344116 +0000 UTC m=+0.128923059 container create 7fe515843c591e04c019a2548a8ca3c71265cba0ff9993d50082bcc07676b32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T14:13:34.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:34 smithi129 podman[68765]: 2024-04-25 14:13:34.09124776 +0000 UTC m=+0.331826662 container init 7fe515843c591e04c019a2548a8ca3c71265cba0ff9993d50082bcc07676b32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T14:13:34.164 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:34 smithi129 podman[68765]: 2024-04-25 14:13:34.094490463 +0000 UTC m=+0.335069364 container start 7fe515843c591e04c019a2548a8ca3c71265cba0ff9993d50082bcc07676b32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T14:13:34.663 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:34 smithi129 podman[68765]: 2024-04-25 14:13:34.182480481 +0000 UTC m=+0.423059394 container attach 7fe515843c591e04c019a2548a8ca3c71265cba0ff9993d50082bcc07676b32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:13:34.664 INFO:journalctl@ceph.osd.11.smithi129.stdout:Apr 25 14:13:34 smithi129 podman[68765]: 2024-04-25 14:13:34.382243322 +0000 UTC m=+0.622822224 container died 7fe515843c591e04c019a2548a8ca3c71265cba0ff9993d50082bcc07676b32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T14:13:34.970 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T14:13:35.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:35.008 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T14:13:35.008 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T14:13:35.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.a 2024-04-25T14:13:35.373 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 14:13:35 smithi078 systemd[1]: Stopping Ceph mds.a for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:35.719 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 14:13:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a[70153]: 2024-04-25T14:13:35.373+0000 7f079cc53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:35.719 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 14:13:35 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a[70153]: 2024-04-25T14:13:35.373+0000 7f079cc53700 -1 mds.a *** got signal Terminated *** 2024-04-25T14:13:39.219 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 14:13:38 smithi078 podman[98942]: 2024-04-25 14:13:38.961014502 +0000 UTC m=+3.754912375 container died 7ead8345f63bfed6c133acd66ed42557d27c044ac129e2948aab3b2e249a482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T14:13:39.502 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 14:13:39 smithi078 podman[98942]: 2024-04-25 14:13:39.406928441 +0000 UTC m=+4.200826316 container cleanup 7ead8345f63bfed6c133acd66ed42557d27c044ac129e2948aab3b2e249a482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T14:13:39.503 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 14:13:39 smithi078 bash[98942]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a 2024-04-25T14:13:39.817 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 14:13:39 smithi078 podman[98956]: 2024-04-25 14:13:39.608402282 +0000 UTC m=+0.642587853 container remove 7ead8345f63bfed6c133acd66ed42557d27c044ac129e2948aab3b2e249a482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:39.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T14:13:39.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:39.871 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T14:13:39.871 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T14:13:39.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.d 2024-04-25T14:13:40.219 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 14:13:39 smithi078 systemd[1]: Stopping Ceph mds.d for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:40.719 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 14:13:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d[69203]: 2024-04-25T14:13:40.254+0000 7f4dd5f4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:40.719 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 14:13:40 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d[69203]: 2024-04-25T14:13:40.254+0000 7f4dd5f4a700 -1 mds.d *** got signal Terminated *** 2024-04-25T14:13:43.527 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 14:13:43 smithi078 podman[99072]: 2024-04-25 14:13:43.22557506 +0000 UTC m=+3.094660329 container died 6a0ff26c9602e2963aafbd70ded55ae1ed55ba1fcf4dd684c4949c4f07799a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:13:43.969 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 14:13:43 smithi078 podman[99072]: 2024-04-25 14:13:43.602994443 +0000 UTC m=+3.472079705 container cleanup 6a0ff26c9602e2963aafbd70ded55ae1ed55ba1fcf4dd684c4949c4f07799a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T14:13:43.969 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 14:13:43 smithi078 bash[99072]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d 2024-04-25T14:13:43.969 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 14:13:43 smithi078 podman[99087]: 2024-04-25 14:13:43.770979974 +0000 UTC m=+0.537230847 container remove 6a0ff26c9602e2963aafbd70ded55ae1ed55ba1fcf4dd684c4949c4f07799a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:44.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T14:13:44.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:44.051 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T14:13:44.051 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T14:13:44.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.g 2024-04-25T14:13:44.364 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 14:13:44 smithi078 systemd[1]: Stopping Ceph mds.g for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:44.719 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 14:13:44 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g[69799]: 2024-04-25T14:13:44.364+0000 7ff64d1ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:44.719 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 14:13:44 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g[69799]: 2024-04-25T14:13:44.364+0000 7ff64d1ef700 -1 mds.g *** got signal Terminated *** 2024-04-25T14:13:48.248 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 14:13:47 smithi078 podman[99193]: 2024-04-25 14:13:47.994846134 +0000 UTC m=+3.731000019 container died 8a2adb1f743e339a518eb76e83581a49ca4d3287574889103f1cade2da3a269c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T14:13:48.656 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 14:13:48 smithi078 podman[99193]: 2024-04-25 14:13:48.348245558 +0000 UTC m=+4.084399448 container cleanup 8a2adb1f743e339a518eb76e83581a49ca4d3287574889103f1cade2da3a269c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T14:13:48.657 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 14:13:48 smithi078 bash[99193]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-g 2024-04-25T14:13:48.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T14:13:48.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:48.850 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T14:13:48.850 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T14:13:48.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.j 2024-04-25T14:13:49.160 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 14:13:48 smithi078 systemd[1]: Stopping Ceph mds.j for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:49.469 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 14:13:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j[68848]: 2024-04-25T14:13:49.160+0000 7faa7d5ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:49.469 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 14:13:49 smithi078 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j[68848]: 2024-04-25T14:13:49.160+0000 7faa7d5ee700 -1 mds.j *** got signal Terminated *** 2024-04-25T14:13:52.412 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 14:13:52 smithi078 podman[99315]: 2024-04-25 14:13:52.147026633 +0000 UTC m=+3.085995913 container died d98f796c6f6019ec2a3e0c0bef51fed334498c2c0f10651e9e55e19e47485585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T14:13:52.673 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 14:13:52 smithi078 podman[99315]: 2024-04-25 14:13:52.496155481 +0000 UTC m=+3.435124758 container cleanup d98f796c6f6019ec2a3e0c0bef51fed334498c2c0f10651e9e55e19e47485585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T14:13:52.674 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 14:13:52 smithi078 bash[99315]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-j 2024-04-25T14:13:52.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T14:13:52.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:52.923 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T14:13:52.923 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T14:13:52.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.b 2024-04-25T14:13:53.266 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:13:52 smithi112 systemd[1]: Stopping Ceph mds.b for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:54.016 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:13:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: 2024-04-25T14:13:53.526+0000 7fbfe514d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:54.016 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:13:53 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b[63616]: 2024-04-25T14:13:53.526+0000 7fbfe514d700 -1 mds.b *** got signal Terminated *** 2024-04-25T14:13:57.763 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:13:57 smithi112 podman[70026]: 2024-04-25 14:13:57.488458366 +0000 UTC m=+4.375119907 container died 832616ae2e58ed3ef979934d7b2c28f9a357fcd0443ecf520c9c3bb4496fb674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:13:58.016 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:13:57 smithi112 podman[70026]: 2024-04-25 14:13:57.886136821 +0000 UTC m=+4.772798362 container cleanup 832616ae2e58ed3ef979934d7b2c28f9a357fcd0443ecf520c9c3bb4496fb674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-25T14:13:58.016 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:13:57 smithi112 bash[70026]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b 2024-04-25T14:13:58.331 INFO:journalctl@ceph.mds.b.smithi112.stdout:Apr 25 14:13:58 smithi112 podman[70041]: 2024-04-25 14:13:58.221188403 +0000 UTC m=+0.731463362 container remove 832616ae2e58ed3ef979934d7b2c28f9a357fcd0443ecf520c9c3bb4496fb674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T14:13:58.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T14:13:58.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:13:58.389 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T14:13:58.390 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T14:13:58.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.e 2024-04-25T14:13:58.731 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:13:58 smithi112 systemd[1]: Stopping Ceph mds.e for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:13:59.016 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:13:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e[62989]: 2024-04-25T14:13:58.731+0000 7f033cefb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:13:59.016 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:13:58 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e[62989]: 2024-04-25T14:13:58.731+0000 7f033cefb700 -1 mds.e *** got signal Terminated *** 2024-04-25T14:14:01.741 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:14:01 smithi112 podman[70157]: 2024-04-25 14:14:01.474378513 +0000 UTC m=+2.851216405 container died 94e0751fdcd2b34eeeef1eab2662b7baf98c760426d52280d4e4a4b56f164ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:14:01.999 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:14:01 smithi112 podman[70157]: 2024-04-25 14:14:01.824206064 +0000 UTC m=+3.201043966 container cleanup 94e0751fdcd2b34eeeef1eab2662b7baf98c760426d52280d4e4a4b56f164ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:14:01.999 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:14:01 smithi112 bash[70157]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e 2024-04-25T14:14:02.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T14:14:02.258 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:14:02 smithi112 podman[70172]: 2024-04-25 14:14:02.000363121 +0000 UTC m=+0.517827758 container remove 94e0751fdcd2b34eeeef1eab2662b7baf98c760426d52280d4e4a4b56f164ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T14:14:02.258 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:14:02 smithi112 systemd[1]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-25T14:14:02.259 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:14:02 smithi112 systemd[1]: Stopped Ceph mds.e for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T14:14:02.259 INFO:journalctl@ceph.mds.e.smithi112.stdout:Apr 25 14:14:02 smithi112 systemd[1]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.e.service: Consumed 2.444s CPU time. 2024-04-25T14:14:02.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:14:02.270 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T14:14:02.270 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T14:14:02.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.h 2024-04-25T14:14:02.516 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:14:02 smithi112 systemd[1]: Stopping Ceph mds.h for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:14:03.016 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:14:02 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h[63305]: 2024-04-25T14:14:02.602+0000 7f709b63c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:14:03.016 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:14:02 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h[63305]: 2024-04-25T14:14:02.602+0000 7f709b63c700 -1 mds.h *** got signal Terminated *** 2024-04-25T14:14:05.736 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:14:05 smithi112 podman[70281]: 2024-04-25 14:14:05.48256625 +0000 UTC m=+2.993568026 container died 7871954402fe2c9296b63ae806739f234428d846b10718f6479e5d57cf8567a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T14:14:06.016 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:14:05 smithi112 podman[70281]: 2024-04-25 14:14:05.820482347 +0000 UTC m=+3.331484166 container cleanup 7871954402fe2c9296b63ae806739f234428d846b10718f6479e5d57cf8567a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:14:06.016 INFO:journalctl@ceph.mds.h.smithi112.stdout:Apr 25 14:14:05 smithi112 bash[70281]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-h 2024-04-25T14:14:06.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T14:14:06.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:14:06.278 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T14:14:06.278 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T14:14:06.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.k 2024-04-25T14:14:06.590 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 14:14:06 smithi112 systemd[1]: Stopping Ceph mds.k for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:14:07.016 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 14:14:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k[62666]: 2024-04-25T14:14:06.590+0000 7fc71b042700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:14:07.016 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 14:14:06 smithi112 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k[62666]: 2024-04-25T14:14:06.590+0000 7fc71b042700 -1 mds.k *** got signal Terminated *** 2024-04-25T14:14:09.507 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 14:14:09 smithi112 podman[70403]: 2024-04-25 14:14:09.233612984 +0000 UTC m=+2.747605221 container died e2927c9c209bc072541bef21aa11a705f873d868ebce8b0dad3bdb2271ac4dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T14:14:09.766 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 14:14:09 smithi112 podman[70403]: 2024-04-25 14:14:09.607674605 +0000 UTC m=+3.121666841 container cleanup e2927c9c209bc072541bef21aa11a705f873d868ebce8b0dad3bdb2271ac4dc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T14:14:09.766 INFO:journalctl@ceph.mds.k.smithi112.stdout:Apr 25 14:14:09 smithi112 bash[70403]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-k 2024-04-25T14:14:09.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T14:14:10.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:14:10.015 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T14:14:10.015 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T14:14:10.016 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.c 2024-04-25T14:14:10.310 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:10 smithi129 systemd[1]: Stopping Ceph mds.c for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:14:10.572 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:10 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c[62695]: 2024-04-25T14:14:10.310+0000 7f6be904f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:14:10.572 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:10 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c[62695]: 2024-04-25T14:14:10.310+0000 7f6be904f700 -1 mds.c *** got signal Terminated *** 2024-04-25T14:14:13.761 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:13 smithi129 podman[68870]: 2024-04-25 14:14:13.491102552 +0000 UTC m=+3.282932918 container died 62bd797bd311249dbd00765d888b58a23c45c1d750c2852c3b2c52db7381c229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T14:14:14.019 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:13 smithi129 podman[68870]: 2024-04-25 14:14:13.828067481 +0000 UTC m=+3.619897850 container cleanup 62bd797bd311249dbd00765d888b58a23c45c1d750c2852c3b2c52db7381c229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T14:14:14.020 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:13 smithi129 bash[68870]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c 2024-04-25T14:14:14.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T14:14:14.270 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:14 smithi129 podman[68896]: 2024-04-25 14:14:14.020722565 +0000 UTC m=+0.527944729 container remove 62bd797bd311249dbd00765d888b58a23c45c1d750c2852c3b2c52db7381c229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T14:14:14.270 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:14 smithi129 systemd[1]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-25T14:14:14.270 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:14 smithi129 systemd[1]: Stopped Ceph mds.c for 5cff6f4a-030a-11ef-bc93-c7b262605968. 2024-04-25T14:14:14.270 INFO:journalctl@ceph.mds.c.smithi129.stdout:Apr 25 14:14:14 smithi129 systemd[1]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.c.service: Consumed 2.459s CPU time. 2024-04-25T14:14:14.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:14:14.281 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T14:14:14.281 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T14:14:14.281 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.f 2024-04-25T14:14:14.606 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:14:14 smithi129 systemd[1]: Stopping Ceph mds.f for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:14:14.913 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:14:14 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f[62385]: 2024-04-25T14:14:14.605+0000 7f4b55fa0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:14:14.913 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:14:14 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f[62385]: 2024-04-25T14:14:14.605+0000 7f4b55fa0700 -1 mds.f *** got signal Terminated *** 2024-04-25T14:14:17.790 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:14:17 smithi129 podman[69004]: 2024-04-25 14:14:17.484014606 +0000 UTC m=+2.977817723 container died c887c63f1db75c467929f167b2f71e0059013c8337a52f2294f69ecfd48faa6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T14:14:18.098 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:14:17 smithi129 podman[69004]: 2024-04-25 14:14:17.85688835 +0000 UTC m=+3.350691473 container cleanup c887c63f1db75c467929f167b2f71e0059013c8337a52f2294f69ecfd48faa6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T14:14:18.098 INFO:journalctl@ceph.mds.f.smithi129.stdout:Apr 25 14:14:17 smithi129 bash[69004]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-f 2024-04-25T14:14:18.240 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T14:14:18.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:14:18.273 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T14:14:18.273 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T14:14:18.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.i 2024-04-25T14:14:18.663 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:14:18 smithi129 systemd[1]: Stopping Ceph mds.i for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:14:18.663 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:14:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i[63005]: 2024-04-25T14:14:18.592+0000 7f785a1c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:14:18.663 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:14:18 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i[63005]: 2024-04-25T14:14:18.592+0000 7f785a1c4700 -1 mds.i *** got signal Terminated *** 2024-04-25T14:14:21.776 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:14:21 smithi129 podman[69124]: 2024-04-25 14:14:21.473813119 +0000 UTC m=+2.983520628 container died 48581fb537166f55162f4880b02673fcb023f3b943d1d2503487971ac1a3fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T14:14:22.085 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:14:21 smithi129 podman[69124]: 2024-04-25 14:14:21.87696891 +0000 UTC m=+3.386676425 container cleanup 48581fb537166f55162f4880b02673fcb023f3b943d1d2503487971ac1a3fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T14:14:22.086 INFO:journalctl@ceph.mds.i.smithi129.stdout:Apr 25 14:14:21 smithi129 bash[69124]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-i 2024-04-25T14:14:22.294 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T14:14:22.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:14:22.326 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T14:14:22.326 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T14:14:22.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.l 2024-04-25T14:14:22.646 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 14:14:22 smithi129 systemd[1]: Stopping Ceph mds.l for 5cff6f4a-030a-11ef-bc93-c7b262605968... 2024-04-25T14:14:22.913 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 14:14:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l[62072]: 2024-04-25T14:14:22.646+0000 7f3ce4619700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:14:22.913 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 14:14:22 smithi129 ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l[62072]: 2024-04-25T14:14:22.646+0000 7f3ce4619700 -1 mds.l *** got signal Terminated *** 2024-04-25T14:14:26.657 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 14:14:26 smithi129 podman[69244]: 2024-04-25 14:14:26.337760856 +0000 UTC m=+3.797800152 container died 659f89faefcbf33ba6ab8a55ff4f014048be7a28c88f09147bdbda17824bc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:14:26.658 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 14:14:26 smithi129 podman[69244]: 2024-04-25 14:14:26.658626202 +0000 UTC m=+4.118665496 container cleanup 659f89faefcbf33ba6ab8a55ff4f014048be7a28c88f09147bdbda17824bc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:14:26.913 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 14:14:26 smithi129 bash[69244]: ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l 2024-04-25T14:14:26.913 INFO:journalctl@ceph.mds.l.smithi129.stdout:Apr 25 14:14:26 smithi129 podman[69258]: 2024-04-25 14:14:26.868090152 +0000 UTC m=+0.527583982 container remove 659f89faefcbf33ba6ab8a55ff4f014048be7a28c88f09147bdbda17824bc24c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-5cff6f4a-030a-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:14:27.081 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cff6f4a-030a-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T14:14:27.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T14:14:27.111 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T14:14:27.111 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:14:27.305 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T14:14:29.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:14:29.963 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T14:14:32.433 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T14:14:32.623 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T14:14:34.922 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:14:34.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:14:34.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:14:35.010 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T14:14:35.011 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi078/crash 2024-04-25T14:14:35.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:14:35.070 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:14:35.070 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:14:35.072 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi112/crash 2024-04-25T14:14:35.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:14:35.117 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:14:35.117 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:14:35.119 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi129/crash 2024-04-25T14:14:35.120 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash -- . 2024-04-25T14:14:35.171 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T14:14:35.172 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-04-25T14:14:35.173 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T14:14:35.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T14:14:35.218 INFO:tasks.cephadm:Compressing logs... 2024-04-25T14:14:35.218 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:14:35.221 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:14:35.223 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T14:14:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:14:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2024-04-25T14:14:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:14:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T14:14:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T14:14:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T14:14:35.251 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T14:14:35.251 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T14:14:35.251 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T14:14:35.251 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.x.log: /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T14:14:35.251 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:14:35.251 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:14:35.251 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:14:35.252 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T14:14:35.252 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.8% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T14:14:35.252 INFO:teuthology.orchestra.run.smithi078.stderr: 83.0% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:14:35.252 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:14:35.252 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.a.log: 94.7% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T14:14:35.252 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T14:14:35.252 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T14:14:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T14:14:35.257 INFO:teuthology.orchestra.run.smithi078.stderr: 94.7% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T14:14:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:14:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:: No such file or directory 2024-04-25T14:14:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T14:14:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T14:14:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:14:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T14:14:35.260 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.8% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr: 90.5% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr: 93.0% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T14:14:35.261 INFO:teuthology.orchestra.run.smithi129.stderr: 82.6% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:: No such file or directory 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T14:14:35.264 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:14:35.265 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.9% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T14:14:35.266 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:14:35.266 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:14:35.271 INFO:teuthology.orchestra.run.smithi078.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:14:35.271 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T14:14:35.272 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T14:14:35.282 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T14:14:35.285 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T14:14:35.286 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:14:35.291 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:14:35.302 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T14:14:35.306 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:14:35.307 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T14:14:35.309 INFO:teuthology.orchestra.run.smithi112.stderr: 94.6% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T14:14:35.313 INFO:teuthology.orchestra.run.smithi129.stderr: 94.7% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T14:14:35.314 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T14:14:35.315 INFO:teuthology.orchestra.run.smithi129.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T14:14:35.322 INFO:teuthology.orchestra.run.smithi129.stderr: 92.9% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T14:14:35.330 INFO:teuthology.orchestra.run.smithi112.stderr: 94.5% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T14:14:35.333 INFO:teuthology.orchestra.run.smithi129.stderr: 92.5% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T14:14:35.444 INFO:teuthology.orchestra.run.smithi078.stderr: 90.3% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T14:14:35.536 INFO:teuthology.orchestra.run.smithi112.stderr: 94.1% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T14:14:35.659 INFO:teuthology.orchestra.run.smithi129.stderr: 91.9% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T14:14:35.707 INFO:teuthology.orchestra.run.smithi112.stderr: 92.0% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T14:14:35.805 INFO:teuthology.orchestra.run.smithi129.stderr: 92.6% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T14:14:35.959 INFO:teuthology.orchestra.run.smithi078.stderr: 92.1% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T14:14:36.073 INFO:teuthology.orchestra.run.smithi078.stderr: 92.5% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T14:14:36.186 INFO:teuthology.orchestra.run.smithi078.stderr: 90.6% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T14:14:36.490 INFO:teuthology.orchestra.run.smithi129.stderr: 92.4% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T14:14:36.543 INFO:teuthology.orchestra.run.smithi078.stderr: 92.2% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T14:14:36.563 INFO:teuthology.orchestra.run.smithi078.stderr: 93.0% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T14:14:36.564 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-04-25T14:14:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m1.332s 2024-04-25T14:14:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m5.150s 2024-04-25T14:14:36.565 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m0.202s 2024-04-25T14:14:36.607 INFO:teuthology.orchestra.run.smithi112.stderr: 92.6% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T14:14:36.649 INFO:teuthology.orchestra.run.smithi129.stderr: 92.6% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T14:14:36.982 INFO:teuthology.orchestra.run.smithi112.stderr: 92.8% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T14:14:37.060 INFO:teuthology.orchestra.run.smithi112.stderr: 92.9% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T14:14:37.117 INFO:teuthology.orchestra.run.smithi129.stderr: 92.1% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T14:14:37.119 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T14:14:37.119 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m1.882s 2024-04-25T14:14:37.119 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m5.489s 2024-04-25T14:14:37.119 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m0.212s 2024-04-25T14:14:37.223 INFO:teuthology.orchestra.run.smithi112.stderr: 92.3% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T14:14:37.273 INFO:teuthology.orchestra.run.smithi112.stderr: 92.1% -- replaced with /var/log/ceph/5cff6f4a-030a-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T14:14:37.275 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T14:14:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m2.038s 2024-04-25T14:14:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m9.376s 2024-04-25T14:14:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.340s 2024-04-25T14:14:37.276 INFO:tasks.cephadm:Archiving logs... 2024-04-25T14:14:37.276 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi078/log 2024-04-25T14:14:37.277 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:14:38.130 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi112/log 2024-04-25T14:14:38.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:14:39.521 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi129/log 2024-04-25T14:14:39.522 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T14:14:40.438 INFO:tasks.cephadm:Removing cluster... 2024-04-25T14:14:40.438 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 --force 2024-04-25T14:14:40.626 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T14:14:40.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 --force 2024-04-25T14:14:41.086 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T14:14:41.344 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cff6f4a-030a-11ef-bc93-c7b262605968 --force 2024-04-25T14:14:41.529 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: 5cff6f4a-030a-11ef-bc93-c7b262605968 2024-04-25T14:14:41.794 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T14:14:41.794 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:14:41.812 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:14:41.827 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T14:14:41.843 INFO:tasks.cephadm:Teardown complete 2024-04-25T14:14:41.843 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T14:14:41.855 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T14:14:41.855 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:14:41.858 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:14:41.870 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T14:14:41.982 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T14:14:41.998 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T14:14:41.998 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:14:42.000 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:14:42.003 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:14:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:14:42.014 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:14:42.016 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +1167us[+1157us] +/- 69ms 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -1490us[-1490us] +/- 60ms 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +711us[ +711us] +/- 85ms 2024-04-25T14:14:42.018 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 7 377 126 +1172us[+1091us] +/- 68ms 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 1 -1468us[-1468us] +/- 61ms 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 65 +677us[ +677us] +/- 85ms 2024-04-25T14:14:42.020 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:14:42.021 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:14:42.021 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-25T14:14:42.021 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +1165us[+1175us] +/- 69ms 2024-04-25T14:14:42.021 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -1499us[-1499us] +/- 60ms 2024-04-25T14:14:42.021 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +682us[ +691us] +/- 85ms 2024-04-25T14:14:42.022 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:14:42.022 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T14:14:42.033 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T14:14:42.033 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T14:14:42.058 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:14:42.137 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:14:42.200 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T14:14:42.265 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:14:42.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:14:42.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:14:42.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:14:42.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:14:42.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:14:42.347 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T14:14:42.359 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T14:14:42.409 INFO:teuthology.task.internal:Duration was 1984.950067 seconds 2024-04-25T14:14:42.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T14:14:42.442 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T14:14:42.442 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:14:42.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:14:42.447 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T14:14:42.480 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:14:42.481 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:14:42.483 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:14:42.961 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T14:14:42.961 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T14:14:42.961 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:14:42.981 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T14:14:42.981 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:14:43.008 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T14:14:43.008 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T14:14:43.032 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T14:14:43.032 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:14:43.035 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:14:43.051 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T14:14:43.098 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T14:14:43.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:14:43.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:14:43.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T14:14:43.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T14:14:43.492 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T14:14:43.493 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:14:43.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:14:43.540 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T14:14:43.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T14:14:43.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:14:43.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:14:43.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:14:43.599 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-04-25T14:14:43.602 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-25T14:14:43.625 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-04-25T14:14:43.651 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:14:43.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:14:43.665 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:14:43.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:14:43.679 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T14:14:43.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:14:43.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T14:14:43.717 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T14:14:43.718 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi078 2024-04-25T14:14:43.718 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:14:43.760 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi112 2024-04-25T14:14:43.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:14:43.795 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673184/remote/smithi129 2024-04-25T14:14:43.796 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T14:14:43.831 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T14:14:43.831 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:14:43.833 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:14:43.836 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T14:14:43.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T14:14:43.893 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T14:14:43.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T14:14:43.919 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T14:14:43.919 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:14:43.921 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:14:43.924 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T14:14:43.933 INFO:teuthology.orchestra.run.smithi078.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:14 /home/ubuntu/cephtest 2024-04-25T14:14:43.935 INFO:teuthology.orchestra.run.smithi112.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:14 /home/ubuntu/cephtest 2024-04-25T14:14:43.937 INFO:teuthology.orchestra.run.smithi129.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 14:14 /home/ubuntu/cephtest 2024-04-25T14:14:43.938 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T14:14:43.950 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T14:14:44.023 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 1984.9500668048859 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T14:14:44.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:14:44.111 INFO:teuthology.run:pass